Seunghyun Joo

Seunghyun Joo

Research Associate at Seoul National University | Looking for a PhD position

CuAD: Source-to-Source Automatic Differentiation of CUDA Kernels for Discrete Adjoints in Computational Fluid Dynamics

Published in arXiv (physics.flu-dyn), 2026

arXiv GitHub

🔍 Overview

Discrete adjoints give the sensitivities that aerodynamic design needs, but they are hard to obtain for flow solvers that run on GPUs. CuAD reads CUDA kernels and writes their tangent and adjoint versions as ordinary CUDA source (project page).

Its reverse mode is organised around the GPU thread. It adopts the convention of CUDA kernels that inputs and outputs do not alias, so intermediate values are recomputed instead of stored. Pure accumulations are differentiated without a tape, and the scalars that a loop carries from one iteration to the next are kept in a fixed thread-local array rather than in the device heap.

âś… Highlights

  • What decides the cost of a GPU adjoint. On a Biot–Savart summation compiled by one compiler, the CuAD adjoint costs about four times the primal at every size up to 16384 vortices. Enzyme reaches the same cost once the source declares __restrict__. On the unchanged source, Enzyme’s heap cache and Clad’s heap tape cost 9 to 4400 times the primal, vary between runs, and Clad fails beyond 8192 vortices.
  • Exact Jacobians in single precision. Jacobians assembled from the generated tangents keep the adjoint gradients accurate to 1e-6 on a transonic airfoil and 4e-5 on a stiff Burgers problem. Single-precision finite differences give errors of 5e-4 to 4e-2 and of 45 to 75 per cent.
  • A complete 2-D Euler adjoint. For a transonic NACA 0012, the residual, the wall and far-field boundary conditions and the wall force are differentiated without change to the source. The lift and drag shape sensitivities from the adjoint and from the tangent agree to 1e-13.

🧪 Selected Results

Comparison with Clad and Enzyme

Biot-Savart adjoint cost and device heap of four tools
Biot–Savart adjoint of four tools under one compiler: median cost of three runs relative to the primal kernel, and the device heap that the tape or cache of Clad and Enzyme requires.

Transonic NACA 0012 (2-D Euler)

Transonic NACA 0012 Mach number and surface pressure
Mach number with the sonic line and surface pressure at M = 0.8 and α = 1.25° on a 160 × 64 O-grid.
Lift adjoint and shape sensitivities
Lift adjoint and the lift and drag sensitivities to eight Hicks–Henne parameters, from the adjoint (bars) and from the tangent (circles).

Single-precision Jacobians

Adjoint gradient error against condition number
Steady Burgers problem: relative error of the adjoint gradient against the condition number, for single-precision Jacobians from the CuAD tangent and from central differences.

Further applications

Nozzle inverse design
Inverse design of a transonic nozzle: area, Mach number and pressure of the initial, target and designed geometry, and the adjoint at the initial design.
Vortex sheet roll-up and sensitivity
Unsteady adjoint of a rolling-up vortex sheet behind an elliptically loaded wing: roll-up and the sensitivity of the wake descent.

Recommended citation: Seunghyun Joo. (2026). “CuAD: Source-to-Source Automatic Differentiation of CUDA Kernels for Discrete Adjoints in Computational Fluid Dynamics.” arXiv preprint arXiv:2610.26694.
Download Paper