II · THE IDEA · ARTIFICIAL INTELLIGENCE
Diffusion Model ODE/SDE Frameworks
▶ Listen · narrated
Once the score of the noisy data distribution is known, generation is no longer only a discrete denoising chain. It becomes the numerical solution of a reverse-time differential equation.
At a glance
- Forward process
- Gradually corrupts data toward a simple prior, often Gaussian
- Key estimate
- The score, the gradient of the log density of noisy data
- Reverse SDE
- Stochastic reverse-time dynamics that generate samples from noise
- Probability flow
- A deterministic ODE with the same marginal densities as the SDE
- Discrete ancestor
- Denoising diffusion as a finite Markov chain of noise steps
Think of a photograph left out in a sandstorm. Grain by grain the image disappears until only random sand remains. If you knew, at every stage, which way the remaining grains were still leaning toward the lost picture, you could push the sand back and rebuild the image.
A diffusion model learns that push. The forward process is the sandstorm: data are steadily mixed with noise. The learned quantity is the score — roughly, the direction in which the noisy sample is still more likely under the true noisy distribution. Sampling starts from pure noise and follows that direction backward in time.
You can follow it in two ways. One adds a little fresh randomness at each step (a reverse SDE). The other follows a carefully chosen deterministic path (the probability flow ODE). Both are built from the same learned score, and both are arranged so that the cloud of possible samples looks the same at every moment of the journey, even though a single run is jittery in one case and smooth in the other.
Forward noising is a Markov process that sends data x(0) toward a tractable prior. In DDPM this is a discrete Gaussian chain; in the score-based framework it is an Itô SDE d x = f(x,t) dt + g(t) d w. The corresponding reverse-time SDE runs from t = T to t = 0 and involves the score ∇_x log p_t(x) in its drift. Estimating that score (denoising score matching or noise prediction) is the learning problem.
The probability flow ODE replaces the reverse SDE with a deterministic drift whose Fokker–Planck evolution yields the same marginals p_t as the SDE. Consequently one score network supports both stochastic integrators and ODE solvers. Discrete DDPM reverse steps can be read as particular discretisations of these continuous dynamics; continuous time makes schedules into choices of f and g and makes deterministic sampling an ODE integration problem. Limitations remain numerical: discretisation error, score approximation error, and the gap between ideal reverse dynamics and the finite-step procedures used in production.
Look closer
Noise first, then time-reversal
In the continuous framing, a forward SDE slowly replaces structure in the data with noise until the distribution is close to a tractable prior. Sampling is not a separate invention: it is the reverse-time process associated with that same forward dynamics. The reverse SDE needs the score of the intermediate noisy distributions, which is what the network is trained to supply.
One family of marginals, two samplers
The probability flow ODE is constructed so that its marginal densities match those of the reverse SDE at every time, yet its trajectories are deterministic. The same trained score can therefore drive either a stochastic sampler or an ODE solver. What changes is pathwise behaviour and the numerical method, not the learned density model itself.
From finite chains to differentials
Denoising diffusion probabilistic models work with a discrete forward Markov chain that adds Gaussian noise over a fixed number of steps and a learned reverse chain. The SDE view treats that construction as a discretisation of continuous dynamics, which makes the role of the score explicit and opens the door to ODE samplers and continuous-time analysis without discarding the discrete picture.
The story
Diffusion models begin with a deliberately destructive process. Clean data are progressively mixed with noise until almost nothing of the original sample remains and the distribution is close to something easy to draw from, typically a Gaussian. In denoising diffusion probabilistic models this is written as a finite Markov chain: each step adds a controlled amount of Gaussian noise, and a neural network is trained so that a reverse chain can undo those steps, one after another, starting from pure noise.
The score-based SDE framework reframes the same idea in continuous time. A forward stochastic differential equation continuously transports the data distribution toward the prior. Under standard conditions that forward process has a reverse-time SDE whose trajectories, run from noise back toward data time, are samples from the original distribution. The reverse dynamics are not free inventions. They are determined by the forward drift and diffusion coefficients together with the score — the gradient of the log density of the state at each intermediate time. Training therefore reduces to estimating that score, often by denoising score matching or by objectives that are closely related to the noise-prediction losses used in discrete diffusion.
Once a score model is available, sampling is numerical integration. One option is to discretise the reverse-time SDE and inject randomness at each step, in the spirit of the original discrete reverse chain. Another option is the probability flow ordinary differential equation associated with the same forward process. That ODE is built so that its marginal densities coincide with those of the SDE at every time, while individual trajectories remain deterministic. The trained score appears in the drift of both systems; the difference is whether a diffusion term is retained.
This dual view does not replace the discrete Markov-chain formulation so much as situate it. A finite noise schedule can be read as a discretisation of a continuous forward SDE, and the learned reverse transitions as a numerical scheme for the reverse dynamics. Continuous time makes certain design choices clearer: alternative noise schedules become choices of drift and diffusion coefficients; deterministic samplers become ODE solvers; and the shared role of the score across stochastic and deterministic paths is explicit. In practice, both families of samplers are used, sometimes mixed within a single pipeline, all driven by one score model trained on the forward corruption process.
Why it mattered then
Denoising diffusion probabilistic models had already shown that a learned reverse of a simple Gaussian noising chain could generate high-quality samples. The SDE formulation placed that empirical success inside a broader continuous-time theory. It identified the score as the central learned object, linked discrete reverse chains to reverse-time stochastic dynamics, and produced the probability flow ODE as a deterministic counterpart with matching marginals. That clarified why several training objectives and sampler designs were related, and it gave a single language in which noise schedules, stochastic samplers and ODE samplers could be compared rather than invented separately.
Why it matters now
Most practical diffusion systems still train a network that is effectively a score or noise predictor and then sample by walking from noise toward data. The ODE/SDE framing is what justifies treating that walk as either a stochastic integrator or a deterministic ODE solve, and what explains why the same checkpoint can support both. It remains the conceptual bridge between discrete training recipes and continuous-time sampler design, including methods that trade randomness for fewer or adaptive function evaluations while aiming to preserve the learned marginal path.
The surprising detail
The probability flow ODE and the reverse SDE are not two competing models of the data. They are two processes engineered to share the same marginal density at every time, so that either can generate samples once the score is known. Path by path they differ — one is deterministic, the other is not — yet at the level of distributions evolving in time they agree. A single trained score therefore licenses both a noisy reverse trajectory and a smooth deterministic one without a second training run.
What is disputed
The discrete DDPM chain and the continuous SDE/ODE framework are tightly related but not identical object-for-object in every paper; continuous formulations recover discrete training and sampling as particular discretisations, while practical systems often blend schedule choices and integrators that only approximately realise the ideal reverse dynamics.
Remember this
A diffusion sampler is reverse-time integration of an SDE or its probability-flow ODE, both driven by the same learned score.
Test yourself
You have one trained score model for a given forward noising process. Why can you sample with either a reverse-time SDE or the probability flow ODE without retraining, and what is guaranteed to match between those two procedures?
Both the reverse SDE and the probability flow ODE are written in terms of the same forward coefficients and the same score of the noisy marginals. The ODE is constructed so that its marginal densities equal those of the SDE at every time. Training estimates that shared score once; either numerical process may then be integrated from the prior. What is guaranteed to match is the time-evolving marginal distributions, not the individual trajectories, which remain stochastic under the SDE and deterministic under the ODE.
Go deeper
- [2011.13456] Score-Based Generative Modeling through Stochastic Differential Equations · arxiv.org
- [2006.11239] Denoising Diffusion Probabilistic Models · arxiv.org
Image: Original diagram, The Daily Triptych. Licence: Original work. Source.