Junoh Kang

Working Effectively with AI Agents

Thu, 23 Jul 2026 00:00:00 GMT

Summary

Remember what productivity means. It is not more output, but more verified outcomes with less human attention.
Stay attached to the Plan → Do → Verify loop. Your mental model must evolve with the work; otherwise, you cannot plan or verify what comes next.
Remember the limits of being human. Your bio-token budget is finite, cannot be parallelized, and is replenished only by time.

LOB-Bench: Orderbook Generative Model Evaluation Benchmark

Wed, 24 Jun 2026 00:00:00 GMT

Overview

LOB-Bench starts from a critique of how generative limit order book models are usually evaluated. One-step prediction loss and a few stylized facts can be useful local checks, but they do not show whether a model remains realistic after it samples a trajectory. A model can pass those narrower checks and still drift away from realistic market behavior once it generates a sequence autoregressively.

The benchmark therefore evaluates generated LOBSTER-compatible order book rollouts by comparing real and generated score distributions, conditional behavior, market-impact response curves, and discriminator separability. This makes it useful for diagnosing where a generated market trajectory fails: quote state, event timing, cancellation behavior, event placement, order flow, price response, or trajectory-level artifacts.

The reported results make the benchmark's role concrete. LOBS5 is the strongest tested model overall, but LOB-Bench still exposes growing horizon error, conditional microstructure failures, market-impact response gaps, and discriminator separability. The useful output is therefore not a single leaderboard score, but a failure profile for generated order book trajectories.

LOB Generation Needs a Rollout Benchmark

Limit order book data is hard to generate because it mixes discrete events, continuous prices and quantities, irregular timing, strategic interaction, and market microstructure constraints. A generator that emits message-level data must keep the book coherent over time. A generator that emits book states directly must still preserve spread, depth, liquidity, return, and order-flow behavior across a rollout.

The evaluation problem is that common checks answer narrower questions. Held-out next-token cross-entropy tests whether the model predicts the next event under real history, but it does not test the distribution of sampled trajectories. Stylized facts test selected marginal patterns, but they can miss failures in timing, cancellation behavior, event placement, or response dynamics. Fragmented realism metrics can show isolated failures, but they do not by themselves produce a benchmark-level failure profile.

LOB-Bench is the paper's answer to that gap. It evaluates generated samples after rollout, when the conditioning history has started to contain model-generated outputs. The target question is not just "did the model predict the next event?" The target question is "does the generated market trajectory still look like real LOB data after the model runs?"

What LOB-Bench Measures

LOB-Bench starts from generated, real, and conditioning sequences stored in LOBSTER-compatible CSV format. This input contract matters. If a model generates messages, those messages need to be exported or replayed into comparable message and book-state sequences. If a model generates another representation, it needs a conversion step before the default benchmark metrics apply.

At a high level, the benchmark has three measurement groups.

| Group | Flow | What it catches | | --- | --- | --- | | Group 1: score-distribution metrics | Sequence $d$ -> scalar score $\Phi(d)$ -> real/generated distribution comparison | Broad microstructure realism across book state, timing, lifecycle, event placement, and order flow. | | Group 2: market-impact response functions | Event class $\pi$ -> sign-adjusted mid-price response curve $R_{\pi}(l)$ -> real/generated curve comparison | Whether generated trajectories preserve average price response after market events. | | Group 3: adversarial measurement | Orderbook-state trajectory -> compact state-change representation -> discriminator score | Whether a learned classifier can still separate generated trajectories from real ones. |

Group 1: Score-distribution Metrics

Group 1 is the main distributional evaluation layer.

Project a sequence to a scalar. A scoring function $\Phi$ maps a generated or real sequence $d$ into a scalar score $\Phi(d)$.
Compare real and generated score distributions. LOB-Bench uses histogram L1 distance for binned distribution mismatch and Wasserstein-1 distance for how far score values move.
Repeat the comparison conditionally. The benchmark can bucket by a context variable or conditioning score, compare the target score distribution inside each bucket, and average the bucket-level errors.
Track the same error by rollout horizon. Conditioning the comparison on rollout step shows whether a model stays realistic as it samples farther away from the real conditioning history.

Default Score Families

The concrete $\Phi$ functions are best read as a menu of probes. They are not the whole benchmark; they are the scalar projections used by Group 1.

| Metric family | Examples | What it checks | | --- | --- | --- | | Quote and book state | Bid-ask spread, order-book imbalance, bid and ask volume, best-level volume | Whether top-of-book tightness, balance, and liquidity look realistic. | | Message timing and lifecycle | Inter-arrival time, time-to-cancel | Whether event timing and cancellation lifetimes match real data. | | Event placement | Limit-order depth, cancellation depth, limit-order level, cancellation level | Whether new limits and cancellations occur at realistic distances from the mid-price or book levels. | | Trading and order flow | Volume per minute, order-flow imbalance, OFI conditional on next mid-price move | Whether trading pressure and directional order-flow patterns are preserved. |

Group 2: Market-impact Response Functions

Group 2 measures whether generated trajectories preserve average price responses around market events.

Choose an event class. LOB-Bench groups events such as market orders, limit orders, and cancellations, and separates them by whether they change the mid-price.
Compute a sign-adjusted response curve. For each event class, it tracks the future mid-price response across a lag grid and aligns the sign with the event's expected price-pressure direction. If $p_t$ is the mid-price at event time $t$ and $\epsilon_t$ is this event-aligned sign, the response curve for event class $\pi$ is

$$ R_{\pi}(l) = \left\langle (p_{t+l} - p_t)\epsilon_t \mid \pi_t = \pi \right\rangle_T. $$

Compare real and generated curves. The benchmark measures the gap between the real response curve and the generated response curve. For one event class, this is the mean absolute curve gap across lags:

$$ \Delta R_{\pi} = \frac{1}{L}\sum_l \left|R_{\pi}^{\mathrm{real}}(l)-R_{\pi}^{\mathrm{gen}}(l)\right|. $$

This is a stronger test than a marginal statistic. A model that matches spreads and volumes can still fail if it does not reproduce how prices react after order-flow shocks. The paper reports that LOBS5 reproduces GOOG response curves much better than a stochastic baseline in the discussed comparison.

Group 3: Adversarial Measurement

Group 3 asks whether a learned classifier can still find trajectory-level artifacts that hand-written metrics miss.

Start from orderbook-state trajectories. The adversarial check uses real and generated book-state sequences.
Map state changes into a compact representation. LOB-Bench represents sparse book updates through features such as mid-price change, relative price level, and quantity change.
Train a discriminator. The discriminator tries to classify whether a trajectory is real or generated.
Read separability as an artifact signal. If the discriminator separates real and generated samples well, then the generated data still contains detectable structure that the interpretable metrics may not fully describe.

Positioning Current LOB Generators

LOB-Bench is most useful as a map of where current LOB generators stand. LOBS5 is the strongest tested model overall, but it is not treated as solved market simulation: the benchmark still exposes horizon error, conditional microstructure failures, response-curve gaps, and discriminator separability.

The comparisons also show why the generated object matters. Message-only RWKV variants diverge quickly on price levels and book-volume-related statistics, which suggests that plausible event tokens are not enough when the book state drifts. Hand-coded or parametric baselines can match some placement statistics, such as depths and levels, while still missing imbalance, volume, timing, and market-impact behavior.

This makes LOB-Bench a positioning tool rather than only a leaderboard. It helps say whether a model is mainly good at static score distributions, conditional behavior, event-response dynamics, or adversarial trajectory realism. It still does not validate a single inserted action, queue position, fill probability, order identity, or matching-engine correctness; those claims need simulator- and execution-specific checks.

References

Nagy et al., LOB-Bench: Benchmarking Generative AI for Finance -- an Application to Limit Order Book Markets (2025), arXiv:2502.09172.

SFT Memorizes, RL Generalizes

Sat, 20 Jun 2026 00:00:00 GMT

Overview

This post reviews SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training by Chu et al. The paper asks a simple but important question: when we post-train a foundation model, does the model learn a transferable rule, or does it mainly become better at reproducing the rule distribution seen during training?

The paper finds that verifier-based RL generalizes better than SFT when the evaluation rule changes. In the tested GeneralPoints and V-IRL environments, supervised finetuning improves or fits the training setting but often collapses under shifted rules. PPO-style reinforcement learning with verifier feedback improves out-of-distribution rule and visual performance from the same SFT-initialized checkpoint.

This does not mean that SFT is useless. In the paper's setup, RL works well after SFT has taught the model a usable answer format; without that SFT initialization, RL does not obtain reliable structured rewards.

|| |:--:| | Figure 1 from Chu et al. illustrates the paper's core pattern: under an OOD rule shift, RL improves while SFT collapses. |

The Question: Does Post-training Memorize or Generalize?

Post-training is often described as the stage that makes a pretrained model useful. SFT teaches the model to follow instructions and produce the desired answer format. RL then optimizes the model against a reward signal, which can come from human preference models, verifiers, or task-specific outcome checks.

The paper reframes this familiar pipeline as a generalization question. If a model is trained on one rule and then evaluated on a related but different rule, what does each post-training method preserve? A method that really learns the underlying principle should adapt to the new instruction. A method that mainly fits the training distribution may keep applying the old rule even when the prompt states a new one.

Experiment Design: Train on One Rule, Test on Another

This is why the paper compares SFT and RL under shifted rules rather than only in-distribution accuracy. In-distribution improvement alone cannot distinguish rule learning from rule memorization. The interesting test is whether the trained behavior survives when the rule, visual input, or action convention changes.

The paper uses the same high-level experimental pattern across the main settings:

Start from the same SFT-initialized checkpoint.
Scale post-training in two different ways.
- SFT path: supervised examples under the training rule.
- RL path: PPO-style RL with verifier reward and feedback.
Evaluate after changing the rule or visual condition.

A small but important detail is that verification is iterative. The model can revise its answer after verifier feedback, and one GeneralPoints ablation shows that more verification iterations improve RL's OOD gains. This supports the idea that RL benefits from learning how to use feedback.

Experiments

Rule Generalization

The rule generalization results are the clearest evidence for the paper's title. In GeneralPoints, the training rule treats J, Q, and K as 10. The shifted rule evaluates them as 11, 12, and 13. In V-IRL, the training rule uses absolute orientation actions such as north and east. The shifted rule uses relative actions such as left and right.

| Task | RL OOD performance | SFT OOD performance | Metric | | --- | ---: | ---: | --- | | GP-L | 11.5% -> 15.0% | 11.5% -> 3.4% | Episode success | | GP-VL | 11.2% -> 14.2% | 11.2% -> 5.6% | Episode success | | V-IRL-L | 80.8% -> 91.8% | 80.8% -> 1.3% | Per-step accuracy | | V-IRL-VL | 35.7% -> 45.0% | 35.7% -> 2.5% | Per-step accuracy |

Under these shifts, RL improves OOD performance from the shared initial checkpoint. SFT moves in the opposite direction. The V-IRL numbers are especially striking. SFT does not merely fail to improve under the shifted action convention. It almost collapses. This supports the paper's interpretation that SFT can overfit to the training rule or output convention, while verifier-based RL can preserve more rule-following flexibility.

Visual Generalization

The paper also tests whether the RL advantage survives visual shifts. In GP-VL, the shift changes card suit colors. In V-IRL-VL, evaluation moves from New York training routes to the multi-city VLN mini benchmark. This matters because rule following alone does not guarantee robust visual recognition.

| Task | RL OOD change | SFT OOD change | Visual shift | | --- | ---: | ---: | --- | | GP-VL | 23.6% -> 41.2% | 23.6% -> 13.7% | Black suits to red suits | | V-IRL-VL | 16.7% -> 77.8% | 16.7% -> 11.1% | NYC routes to VLN mini benchmark |

The visual results suggest that RL is not only improving final task behavior. In GP-VL, RL improves both recognition accuracy and episode success, while SFT deteriorates both. The authors hypothesize that SFT may locally overfit to frequent reasoning tokens while neglecting recognition tokens, but they leave the exact mechanism unresolved. In V-IRL-VL, verifier-backed revision appears to help the model use visual and instruction information more robustly under a new route distribution.

Why SFT Still Matters

The paper's conclusion is not "skip SFT." Its main RL pipeline starts from an SFT-initialized model because the verifier needs structured outputs to assign useful rewards. When the authors apply RL directly to the base Llama-3.2-Vision-11B model on GP-L, the runs fail to improve: the base model often produces long, tangential, or unstructured answers, so the verifier cannot reliably extract task information.

SFT therefore acts as a format teacher. It teaches the model to answer in a way that the verifier can parse, creating the usable initialization window for RL. This also explains why the result does not directly contradict DeepSeek-R1-style claims that strong reasoning behavior may emerge with little or no cold-start SFT in other settings. The requirement depends on the backbone, task, verifier, and output format.

References

Chu et al., SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training (ICML 2025), arXiv:2501.17161v2.

Post-Training of Modern LLMs

Fri, 19 Jun 2026 00:00:00 GMT

Overview

Modern LLMs are usually built in stages. Pre-training gives the model broad next-token prediction capability. Instruction tuning teaches the model to follow user instructions. Post-training then shapes the model's behavior with preference feedback, reward optimization, or verifiable task rewards.

This post follows two post-training routes. The first is RLHF, where preference data is used through PPO-style reward optimization or direct preference optimization; the examples are InstructGPT and DPO. The second is RLVR, where the reward comes from checkable outcomes rather than a learned preference model; the examples are DeepSeekMath's GRPO, DeepSeek-R1-Zero, and the final DeepSeek-R1 pipeline.

From Next-token Prediction to RLHF

The limitation of next-token prediction is that it imitates text rather than directly optimizing assistant behavior. A base model can learn fluent language from web-scale data, but helpfulness, truthfulness, and harmlessness are not direct training targets. More importantly, next-token prediction does not provide direct negative feedback on what behavior to avoid.

RLHF adds a preference signal after supervised instruction tuning by increasing the probability of preferred behavior and making dispreferred behavior less likely.

InstructGPT

PPO-based RLHF is the explicit reward-model route. The RL objective is not just "maximize reward." It also constrains the updated policy to stay close to a reference policy. In InstructGPT, the PPO-ptx objective adds a pretraining loss term to reduce capability regressions:

$$ \max_{\phi}; \mathbb{E}{(x,y)\sim D{\pi_\phi^{\mathrm{RL}}}} \left[ r_\theta(x,y)-\beta \log \frac{\pi_\phi^{\mathrm{RL}}(y\mid x)} {\pi^{\mathrm{SFT}}(y\mid x)} \right] + \gamma, \mathbb{E}{x\sim D{\mathrm{pretrain}}} \left[ \log \pi_\phi^{\mathrm{RL}}(x) \right]. $$

The first term optimizes the reward model while penalizing deviation from the SFT policy. The second term is the ptx loss: it keeps part of the update anchored to pretraining data to reduce broad capability regressions.

Result

The result was strong: PPO and PPO-ptx outperformed SFT and prompted base models on labeler preference evaluations, and the 1.3B PPO-ptx model was preferred over the 175B GPT-3 baseline despite the parameter gap.

Limitation

The limitation is the reward model. It is costly to train, it is not a perfect proxy for human preference, and optimizing against it can create reward hacking or reward-model over-optimization problems.

DPO

DPO is the direct preference-optimization route inside the same preference-based framing. It starts from the same KL-regularized preference view, but removes the explicit reward-model and PPO stages. Its key observation is that the optimal reward can be represented by a policy/reference log-ratio:

$$ \hat r_\theta(x,y)

\beta \log \frac{\pi_\theta(y \mid x)} {\pi_{\mathrm{ref}}(y \mid x)}. $$

The resulting loss directly compares a preferred answer $y_w$ and a dispreferred answer $y_l$:

$$ L_{\mathrm{DPO}}(\pi_\theta;\pi_{\mathrm{ref}})

-\mathbb{E}{(x,y_w,y_l)} \left[ \log \sigma\left( \beta \log \frac{\pi\theta(y_w\mid x)}{\pi_{\mathrm{ref}}(y_w\mid x)}

\beta \log \frac{\pi_\theta(y_l\mid x)}{\pi_{\mathrm{ref}}(y_l\mid x)} \right) \right]. $$

This is why DPO is sometimes described as RL-free. That phrase is easy to misunderstand. DPO removes the explicit reward model and PPO optimization loop, but it still uses preference pairs, a reference policy, and a KL-regularized objective.

Limitations

The original DPO evidence is also scale-limited. Its strongest reported experiments reach models up to 6B parameters, so the paper should be read as evidence for the tested settings rather than as proof that DPO replaces PPO-based RLHF at frontier scale.

From Preference Rewards to Verifiable Rewards

Preference-based RLHF is useful, but the reward signal is expensive and subjective. Human labelers or reward models must judge completed responses. The learned reward model can also be over-optimized, especially when the policy discovers answers that score well under the reward model without being genuinely useful.

RLVR changes the reward source. In math, code, and some reasoning tasks, the final answer can be checked. A math answer can be compared against the known answer. A program can be run against tests. In these settings, the training loop does not need a learned preference model for every completed answer. This distinction is the bridge from preference alignment to reasoning RL. RLVR is attractive because the reward can be less subjective. The hope is that training on logical, checkable problems can also improve the model's broader reasoning capability.

| Setting | Reward signal | Natural domain | | --- | --- | --- | | RLHF / DPO | Human or model preference | Chat, helpfulness, safety | | RLVR | Rule-based outcome reward | Math, code, verifiable reasoning |

GRPO and DeepSeekMath

DeepSeekMath introduces Group Relative Policy Optimization, or GRPO, as a PPO-style method for mathematical reasoning. The main engineering problem is the learned value model, or critic. PPO normally uses a value function to estimate advantages. For LLM reasoning, that value model can be another large model, and math rewards often arrive only after a full solution is generated.

GRPO removes the learned critic. For each question, it samples a group of outputs, scores them, normalizes the rewards inside the group, and uses those group-relative values as advantages. The GRPO objective can be remembered as a grouped PPO clipped objective with a reference-policy KL penalty:

$$ \begin{aligned} J_{\mathrm{GRPO}}(\theta) &= \mathbb{E}{q,,{o_i}{i=1}^{G}} \Bigg[ \frac{1}{G}\sum_{i=1}^{G} \frac{1}{|o_i|}\sum_{t=1}^{|o_i|} \left( \min\left( \rho_{i,t}(\theta)A_i,, \operatorname{clip}\left(\rho_{i,t}(\theta),1-\epsilon,1+\epsilon\right)A_i \right) -\beta\mathbb{D}{\mathrm{KL}}\left(\pi\theta\Vert\pi_{\mathrm{ref}}\right) \right) \Bigg]. \end{aligned} $$

The group-relative advantage is:

$$ A_i

\frac{ r_i-\operatorname{mean}({r_1,r_2,\ldots,r_G}) }{ \operatorname{std}({r_1,r_2,\ldots,r_G}) }. $$

The core move is critic removal, not reward removal. DeepSeekMath-RL still scores sampled outputs and still uses a reference policy for KL regularization. GRPO should therefore be read as a PPO variant, not as a wholly separate reinforcement learning family.

DeepSeek-R1-Zero

R1-Zero is the cleaner RLVR case. It applies RL directly to DeepSeek-V3-Base with verifiable rewards. AIME 2024 is a good example: the final answer is an integer from 0 to 999, so correctness can be checked by exact match. On AIME 2024, the paper reports pass@1 rising from 15.6% to 77.9%, and self-consistency with 16 samples reaching 86.7%. It also reports longer generated reasoning during training.

DeepSeek-R1

R1-Zero shows pure RLVR can induce reasoning, but with usability issues such as CoT readability and language mixing. The final DeepSeek-R1 pipeline turns that idea into a more usable multi-stage training recipe:

Cold-start SFT fixes CoT readability before RL.
First RL improves reasoning while reducing language mixing.
Second SFT broadens the model beyond math and code reasoning.
Final RL aligns general assistant behavior with helpfulness and safety.

Limitations of RLVR and GRPO

GRPO normalization bias

GRPO removes the learned critic, but it does not remove all design choices from the objective. Two normalization terms are especially important:

$$ \frac{1}{|o_i|}\sum_{t=1}^{|o_i|}L^{\mathrm{PPO}}_{i,t} \qquad A_i = \frac{r_i-\operatorname{mean}(r)} {\operatorname{std}(r)}. $$

The first term is length normalization. Because the token-level loss is averaged by output length, a long incorrect output can receive a weaker per-token penalty. If a model expects to be wrong, lengthening the answer can dilute the penalty signal.

The second term is reward standard-deviation normalization. Dividing by $\operatorname{std}(r)$ can overweight questions where the sampled answers have low reward variance. This can happen for very easy questions where most samples are correct, or for very hard questions where most samples are wrong.

Dr. GRPO is motivated by these caveats: it removes both the length normalization and the reward standard-deviation normalization terms.

RLVR outcome-level reward

RLVR usually gives an outcome-level reward. It can check whether the final answer is right, but it does not automatically explain which reasoning step was wrong. It is still a sparse signal for improving the reasoning process itself.

References

Ernest Ryu, RL for LLMs.
Ouyang et al., Training language models to follow instructions with human feedback (2022).
Rafailov et al., Direct Preference Optimization: Your Language Model is Secretly a Reward Model (2023).
Shao et al., DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models (2024).
Guo et al., DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning (2025).
Liu et al., Understanding R1-Zero-Like Training: A Critical Perspective (2025).

One-step Generation in the Post Diffusion Era

Thu, 12 Mar 2026 00:00:00 GMT

Overview

Diffusion models are still one of the major paradigms in image and video generation. They produce high-quality samples and remain a strong default for many generative tasks. However, their sampling process is inherently iterative: generation usually requires many denoising steps or numerical solver evaluations. This inference-time iteration becomes a practical bottleneck when we care about fast generation, large-scale serving, or long-form generation.

This post gives a high-level review of training-based approaches toward one-step or few-step generation: methods that try to compress, replace, or bypass the inference-time trajectory through distillation, consistency constraints, average-velocity regression, distribution matching, or training-time distribution evolution.

Drifting Models are discussed in more detail than the others because they make this contrast especially explicit. Diffusion and flow models evolve samples during inference, while Drifting Models interpret training itself as a distribution-evolution process: as the generator parameters change, the generated distribution drifts toward the data distribution. The purpose of the post is still broader than Drifting Models alone: to organize several attempts at one-step generation and clarify what each method depends on.

Generative Modeling

Pushforward Formulation

At a high level, generative modeling can be written as a pushforward problem. We start from a simple prior distribution, such as a Gaussian noise distribution $p_\epsilon$, and learn a network $f_\theta$ that transforms it into a generated distribution

$$ q_\theta = (f_\theta)# p\epsilon \approx p_{\mathrm{data}}. $$

Diffusion / Flow Models: Inference-Time Iteration

Diffusion and flow models make the pushforward problem tractable by decomposing a complex transformation into many small steps. Instead of learning the full noise-to-data map in one shot, they introduce an intermediate time variable and learn a score, denoising direction, or velocity field that tells each sample which direction to move along the trajectory.

In both cases, sampling can be viewed as solving learned SDE or ODE dynamics that move each sample from noise toward data at inference time. Numerically solving these dynamics requires iterative updates. This multi-step inference-time sampling is the bottleneck that motivates one-step generation.

Training-Based Attempts Toward One-Step Generation

This section reviews training-based attempts to obtain one-step or few-step generators. The methods differ mainly in what training signal they use: teacher trajectories, trajectory consistency, average velocity, score-based distribution matching, or training-time distribution evolution.

Progressive Distillation

Progressive distillation trains a student one-step transition to match two consecutive teacher transitions:

$$ f_\theta(z_t, t \to s) \approx f_\eta\left(f_\eta(z_t, t \to u), u \to s\right), \qquad t > u > s. $$

Consistency Models

The key idea of Consistency Models is to learn a function whose output is consistent along the same probability-flow ODE trajectory. Instead of predicting only the next denoising step, the model maps noisy samples at different times to the same clean endpoint.

The consistency constraint is:

$$ f_\theta(x_t,t) \approx f_\theta(x_{t^\prime},t^\prime), \qquad x_t,;x_{t^\prime} \text{ on the same trajectory}. $$

The practical question is how to obtain such paired points. Consistency Models use two main training setups: Consistency Distillation (CD) and Consistency Training (CT). The pair construction differs as follows:

$$ \begin{cases} \text{Consistency Distillation (CD):} & x_{t^\prime} = \operatorname{ODESolver}(x_t, t \to t^\prime; s_\phi), \ \text{Consistency Training (CT):} & x_t = \alpha_t x_0 + \sigma_t \epsilon,\quad x_{t^\prime} = \alpha_{t^\prime} x_0 + \sigma_{t^\prime} \epsilon. \end{cases} $$

CD uses a pretrained score model $s_\phi$ and an ODE solver to move along the probability-flow trajectory, while CT constructs two noisy versions of the same clean sample using the same noise.

After training, one-step sampling is a single evaluation from terminal noise:

$$ x_0 = f_\theta(x_T,T). $$

Limitation

Unstable training. Consistency Models are highly sensitive to curriculum design, such as how the time discretization is scheduled and how difficult trajectory pairs are introduced during training.
Indirect training signal. The constraint $f_\theta(x_t,t) = f_\theta(x_{t^\prime},t^\prime)$ only says that two outputs should match. By itself, this admits trivial solutions, so practical training needs boundary conditions and other engineering tricks to make the shared output correspond to a real clean sample.
Dependence on diffusion teacher in CD. Consistency distillation still requires a pretrained diffusion or score model to construct reliable same-trajectory pairs.

CTM

Consistency Trajectory Models (CTM) extend Consistency Models from endpoint consistency to trajectory consistency. Instead of only mapping a noisy point to the clean endpoint, CTM learns a transition map

$$ G_\theta(x_t,t,s) $$

that moves a sample from time $t$ to another time $s$ along the probability-flow ODE trajectory.

The trajectory consistency constraint is:

$$ G_\theta(x_t,t,s) \approx G_\theta(G_\theta(x_t,t,u),u,s). $$

That is, a direct jump from $t$ to $s$ should match a composed path through an intermediate time $u$.

CTM also has to handle special cases of the transition map. For example,

$$ \begin{cases} G_\theta(x_t,t,t)=x_t, & \text{identity at the same time}, \ G_\theta(x_t,t,0)\approx x_0, & \text{endpoint prediction}. \end{cases} $$

After training, one-step sampling is:

$$ x_0 = G_\theta(x_T,T,0). $$

Because the target time $s$ is an input, the same model can also be used for multi-step sampling by composing shorter transitions. In that sense, CTM tries to learn the probability-flow ODE trajectory itself, not only the final endpoint.

Limitation

Additional constraints for special cases. CTM is more flexible than CM because it models transitions between arbitrary times, but that flexibility introduces boundary cases such as $s=t$ and $s=0$ that need to be handled by the parameterization and loss design.
Training stability. CTM still has an indirect training signal: the model is trained by making direct and composed paths agree, unlike MeanFlow's velocity-regression-style objective.
Not self-contained. Strong practical performance requires an additional GAN component.

MeanFlow

The key idea of MeanFlow is to learn the average displacement over a time interval, rather than the instantaneous velocity at each time. Flow Matching learns a local velocity field $v(x_t,t)$ and then integrates it with an ODE solver. MeanFlow asks whether the model can learn the interval-level velocity directly.

The average velocity is defined as:

$$ u(z_t,r,t)

\frac{1}{t-r} \int_r^t v(z_\tau,\tau),d\tau. $$

The training signal comes from the MeanFlow identity, which connects this average velocity to the instantaneous velocity:

$$ v(z_t,t)

\partial_z u \cdot v + \partial_t u. $$

The derivative term is computed with a Jacobian-vector product (JVP), so the method can train the average-velocity model with a regression-style objective rather than a trajectory-consistency constraint.

After training, one-step sampling applies the learned average displacement over the full interval:

$$ x_1 = \epsilon + u_\theta(\epsilon,0,1). $$

Limitation

Additional constraint for the special case. When the interval collapses, MeanFlow needs the average velocity to match the instantaneous velocity:

$$ u_\theta(x_t,t,t)=v(x_t,t). $$

DMD

The key idea of Distribution Matching Distillation (DMD) is to train a one-step generator by directly matching the generated distribution to the data distribution. Instead of enforcing agreement along an ODE trajectory, DMD minimizes a KL divergence between the noised fake distribution and the noised real distribution:

$$ \int_0^T w(t), D_{\mathrm{KL}}!\left(q_t^{\mathrm{fake}} ,|, p_t^{\mathrm{real}}\right),dt. $$

The important observation is that the gradient of this KL objective can be expressed using score functions:

$$ \nabla_\theta \int_0^T w(t), D_{\mathrm{KL}}!\left(q_t^{\mathrm{fake}} ,|, p_t^{\mathrm{real}}\right),dt \approx \int_0^T w'(t), \mathbb{E}_{\hat{x}t} \left[ \epsilon{\mathrm{real}}(\hat{x}_t,t)

\epsilon_{\mathrm{fake}}(\hat{x}_t,t) \right] \frac{\partial \hat{x}_t}{\partial \theta} ,dt. $$

Here, the real score and fake score tell the generator how the fake distribution should move relative to the real distribution. After training, sampling is simply a single generator forward pass:

$$ \hat{x}0 = g\theta(\epsilon), \qquad \epsilon \sim p_\epsilon. $$

Limitation

Not self-contained. DMD requires a pretrained diffusion model to provide the score signal, and the one-step generator also needs a reasonably good initialization.
Heavy overhead before training. The method requires extra data-generation work, such as constructing paired noise-image data for distillation.

Comparing One-step Methods

| Method | Objective Type | ODE Dep. | Self-Contained | |:--|:--|:--:|:--:| | Diffusion Models | Regression | -- | Yes | | Consistency Models | Consistency constraint | Yes | Partial | | CTM | Consistency constraint | Yes | Partial | | MeanFlow | Regression (velocity) | Yes | Yes | | DMD | Regression (KL-divergence) | No | No |

One thing I want to highlight from this table is the objective type. Consistency-based objectives are elegant, but the supervision is indirect: the model is asked to make two paths or two time points agree. Regression-style objectives are often easier to reason about and can be more stable in practice because they provide a more explicit target. This is somewhat orthogonal to the main topic of one-step generation, but it matters when comparing these methods as training recipes.

This is one reason MeanFlow and DMD are interesting in this list. MeanFlow turns flow learning into regression on an average velocity, while DMD turns distribution matching into a score-based regression signal. They are not free of assumptions, but their objectives are closer to direct regression than pure consistency constraints.

The remaining question is then not simply "can we generate in one step?" These methods already show several ways to do that. The more useful question is what kind of training signal makes one-step generation stable, self-contained, and less dependent on an inference-time trajectory or a pretrained diffusion teacher.

Drifting Models

Training-Time Distribution Evolution

Drifting Models take a different perspective on where the generative dynamics happen. In diffusion and flow models, the generated distribution evolves along pseudo-time while sampling. In Drifting Models, the generated distribution evolves during training.

|| |:--:| | Drifting Models shift the distribution evolution from inference-time sampling to training-time updates. |

Method

Drift

Suppose the generator at training step $i$ is $f_i$. For a fixed noise sample $\epsilon$, the generated point is $x_i = f_i(\epsilon)$. After one update, the generator becomes $f_{i+1}$ and the same noise maps to $x_{i+1} = f_{i+1}(\epsilon)$. The difference

$$ \Delta x_i = f_{i+1}(\epsilon) - f_i(\epsilon) $$

is the sample's training-induced movement, which is why the method is called "drifting." Since parameter updates already move generated samples during training, Drifting Models make this movement explicit and govern it with a drift field $V_{p,q}$, where $p$ is the data distribution and $q$ is the current model distribution.

Drift Field

The drift field $V_{p,q}(x)$ determines how a generated sample should move when the current model distribution is $q$ and the target distribution is $p$. The desired equilibrium is clear: when $p=q$, samples should stop drifting. To encode this, the field is designed to be anti-symmetric:

$$ V_{p,q}(x) = -V_{q,p}(x). $$

If $p=q$, anti-symmetry implies $V_{p,q}(x)=0$. The converse is more delicate. A zero drift field does not generally guarantee that $p=q$, so the theory relies on expressive kernels and additional identifiability assumptions. This is one of the important gaps in the method.

Training Objective

The training objective is derived from the fixed-point relation at equilibrium:

$$ f_{\hat{\theta}}(\epsilon)

f_{\hat{\theta}}(\epsilon) + V_{p,q_{\hat{\theta}}}!\left(f_{\hat{\theta}}(\epsilon)\right). $$

Drifting Models turn this relation into a stop-gradient regression target:

$$ L(\theta; \textcolor{#B509AC}{q_\theta})

\mathbb{E}{\epsilon} \left[ \left| \underbrace{f\theta(\epsilon)}_{\text{prediction}}

\underbrace{ \operatorname{sg} \left( f_\theta(\epsilon) + V_{p,\textcolor{#B509AC}{q_\theta}}(f_\theta(\epsilon)) \right) }_{\text{frozen target}} \right|^2 \right]. $$

The important part is the dependence on $\textcolor{#B509AC}{q_\theta}$. The objective changes with the current generated distribution, so training is not just fitting a pre-defined target field; it repeatedly updates the target according to the model's own distribution.

Drifting Field Design

The drift field is built from an attraction-repulsion interaction. For a generated point $x$, positive samples $y^+$ come from the data distribution and negative samples $y^-$ come from the generated distribution. A kernel $k(x,y)$ measures similarity, and the resulting drift has the form of an attraction toward real samples and a repulsion away from generated samples:

Formally, the drifting field is defined as

$$ V_{p,q}(x) \propto \mathbb{E}_{y^+ \sim p,; y^- \sim q} \left[ k(x,y^+) k(x,y^-)(y^+ - y^-) \right]. $$

This construction satisfies the anti-symmetry condition $V_{p,q}=-V_{q,p}$. The form is close in spirit to contrastive learning: similar real samples pull the generated point, while similar fake samples provide repulsion.

Implementation Details

Two practical details matter for applying the drift objective to image generation: feature-space drift and CFG-conditioned sampling.

Encoder-Based Drift

Doing the attraction-repulsion computation directly in raw pixel space is not enough. Pixel distances do not capture semantic similarity well in high-dimensional image domains, so the practical version computes similarity in feature space using a frozen self-supervised encoder $\phi$, such as MoCo, SimCLR, or latent-MAE features. In effect, the kernel becomes

$$ k_\phi(x,y)

k(\phi(x),\phi(y)). $$

The generator can still output pixels or latents, and the encoder is used only during training to construct the drift target. At inference time, the encoder is not evaluated. This is why I would describe Drifting Models as not ODE-dependent, but not completely assumption-free: they remove the diffusion teacher and inference-time trajectory, but introduce a strong dependence on representation and kernel design.

Classifier-Free Guidance

For conditional generation, CFG is introduced by changing how the positive and negative samples are drawn:

$$ \begin{aligned} y^+ &\sim p_{\mathrm{data}}(\cdot \mid c), \ y^- &\sim \tilde{q}(\cdot \mid c)

(1-\gamma)q_\theta(\cdot \mid c) + \gamma p_{\mathrm{data}}(\cdot \mid \phi_{\mathrm{null}}), \end{aligned} $$

$\phi_{\mathrm{null}}$ denotes the null condition. During training, the drift field constructed from these positive and negative samples is used to train the $\gamma$-conditioned generator $f_\theta(\epsilon,c,\gamma)$. During sampling, the drift field is not evaluated. The trained generator directly takes the condition and guidance scale:

$$ x = f_\theta(\epsilon,c,\gamma), \qquad \epsilon \sim p_\epsilon. $$

So CFG is not applied by mixing two predictions at every denoising step, as in diffusion sampling. It is learned during training and exposed as an input to the one-step generator.

Results

Importance of Anti-Symmetry

The anti-symmetric attraction-repulsion structure is not cosmetic. When the balance is broken, FID degrades sharply, which suggests that the field design is central to the method.

|| |:--:| | Breaking the anti-symmetric attraction-repulsion design substantially degrades FID. |

Ablations on Encoders

The feature encoder is another major factor. The ablation shows that representation choice has a large effect on FID, reinforcing that Drifting Models depend strongly on the feature space used to compute the kernel.

|| |:--:| | The feature encoder choice is a major part of the method's empirical behavior. |

Limitation

Theoretical gap. A vanishing drift field, $V \to 0$, does not by itself guarantee $p=q$; the identifiability argument remains heuristic.
Kernel and representation dependency. The field is computed through $k(\phi(x),\phi(y))$, so performance depends heavily on the encoder and kernel design.
Design optimality is not guaranteed. The kernel, drifting field, and architecture are still design choices with room for improvement.
Scalability is still unverified. The presented setting does not yet cover generation beyond $256 \times 256$ or text conditioning, and the contrastive-like objective benefits from large batch sizes.

Positioning

References

Luhman and Luhman, Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed (2021).
Salimans and Ho, Progressive Distillation for Fast Sampling of Diffusion Models (ICLR 2022).
Song et al., Consistency Models (ICML 2023).
Kim et al., Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusion (ICLR 2024).
Geng et al., Mean Flows for One-Step Generative Modeling (NeurIPS 2025).
Yin et al., One-step Diffusion with Distribution Matching Distillation (CVPR 2024).
Deng et al., Generative Modeling via Drifting (arXiv:2602.04770).

Path Signature: Useful Feature for Timeseries

Sun, 01 Feb 2026 00:00:00 GMT

(Motivation) Random Variable: Polynomial Features

Definition (Words / Multi-Indices)

A word (or multi-index) is a finite sequence $I:=i_1 \ldots i_k$ with $i_1,\ldots,i_k \in {1,\ldots,d}$. The length of $I$ is $|I|:=k$. The set ${1,\ldots,d}$ is the alphabet. Denote $\mathcal{W}$ by the set of all words.

Definition (Polynomials for RV)

For a vector $x=(x^1,\ldots,x^d)$ and a multi-index $I=(i_1,\ldots,i_k)$, define $x^{I}=x^{(i_1,\ldots,i_k)}:=x^{i_1}\ldots x^{i_k}$. The collection ${x^I}_{I\in \mathcal{W}}$ is called the polynomials of $x$.

Theorem (Stone-Weierstrass, Carleman)

Under suitable integrability conditions on $\mathbb{P}$, $\mathbb{E}{X \sim \mathbb{P}}[X^I] = \mathbb{E}{X \sim \mathbb{Q}}[Y^I] ~~~\forall I \in \mathcal{W} ~~~~~\Leftrightarrow~~~~~ \mathbb{P} = \mathbb{Q}$

Paths: Iterated Integrals and Signature

Definition (Path-space)

Let $\mathcal{P}([a,b], \mathbb{R}^d)$ denote the space of continuous bounded variation paths $X=(X^1,\ldots,X^d):[a,b] \rightarrow \mathbb{R}^d.$

Definition (Iterated integrals)

For $X \in \mathcal{P}([a,b], \mathbb{R}^d)$ and word $I=i_1 \ldots i_k$, define the iterated integral $S(X)_{a,\cdot}^I \in \mathcal{P}([a,b], \mathbb{R})$ inductively by

$$ S(X){a,t}^{I}=\int_a^t S(X){a,s}^{i_1 \ldots i_{k-1}} dX_s^{i_k}, $$

with base case $k=0$ as $S(X)_{a,t}^{\phi}=1$. In other expression,

$$ S(X){a,t}^{I} = \int{a<t_1<\ldots<t_k<t} dX_{t_1}^{i_1} \ldots dX_{t_k}^{i_k}. $$

Definition (Signature)

The signature $S(X){a,b}$ of $X \in \mathcal{P}([a,b], \mathbb{R}^d)$ is the collection of real numbers indexed by words ${ S(X){a,b}^I}{I\in\mathcal{W}}=(S(X){a,b}^{\phi},S(X){a,b}^1,\ldots,S(X){a,b}^d,S(X){a,b}^{11}, \ldots S(X){a,b}^{dd},\ldots).$

Definition

$$ S(X){a,b}^{(k)} = { S(X){a,b}^I }_{|I|=k} $$

Exercise (Linear path)

Let $x \in \mathbb{R}^d$ and define $X(t) = tx$ for $t\in[0,1]$. Then $S(X){0,1}^{i_1 \ldots i_k} = \frac{x^{i_1}\ldots x{i_k}}{k!}.$ sol)

$$ \begin{align*} S(X){0,1}^{i_1 \ldots i_k} &= \int{0}^{1} S(X){0,t}^{i_1 \ldots i{k-1}}dX_t^{i_k} \ &= \int_0^1 \frac{x^{i_1}\ldots x_{i_{k-1}}}{(k-1)!}t^{k-1}x^{i_k} dt ~~(\because induction) \ &= \frac{x^{i_1}\ldots x_{i_k}}{k!} \end{align*}_\square $$

Exercise (1D path)

For $X \in \mathcal{P}([a,b],\mathbb{R})$, $S(X)_{a,b}^{1 \ldots 1} = \frac{(X_b-X_a)^k}{k!}.$sol)

$$ \begin{align*} S(X){a,b}^{1^{\otimes k}} &= \int_a^b S(X){a,t}^{1^{\otimes {k-1}}} dX_t \ &= \int_a^b \frac{(X_t - X_a)^{k-1}}{(k-1)!} \dot{X}_t dt \ &= [\frac{(X_t - X_a)^{k}}{k!}]a^t = \frac{(X_b - X_a)^{k}}{k!} \end{align*}\square $$

Beyond Defaults: Is Noise Conditioning Necessary for Diffusion Models?

Wed, 05 Nov 2025 00:00:00 GMT

Overview

Traditional diffusion models explicitly specify noise level to neural network to model score function. However, recent research has begun to question whether this conditioning is truly necessary. In this post, I review two papers that challenge this fundamental assumption and propose alternative approaches:

"Is Noise Conditioning Necessary for Denoising Generative Models?"

This paper challenges the convention by demonstrating that denoising networks can perform effectively without explicit noise level conditioning, suggesting that models may inherently learn to estimate noise levels from the input data itself.

"Equilibrium Matching: Generative Modeling with Implicit Energy-based Models"

This work reformulates the generative modeling problem as learning an energy landscape, providing a theoretical foundation for noise-unconditioning approaches.

PinT algorithms for Diffusion Models

Thu, 28 Aug 2025 00:00:00 GMT

Overview

Diffusion models require heavy computation resource due to iterative sampling strategy. Due to their sequential sampling strategy, many accleration algorithms trade sample quality for efficiency. However, there are a group of researches which trade compute for time. In this post, I review two papers accelerating sampling in time, which are motivated from PinT (Parallel in Time) algorithms:

Parallel Sampling of Diffusion Models
Self-Refining Diffusion Samplers: Enabling Paralleization via Parareal Iterations

Test-time scaling in Diffusion Models

Thu, 29 May 2025 00:00:00 GMT

Overview

This post reviews three papers related to test-time scaling which enable optimization with restpect to diverse condtions without fine-tuning diffusion models nor training external models.

Derivative-Free Guidance in Continuous and Discrete Diffusion Models with Soft Value-Based Decoding
Test-Time Alignment of Diffusion Models without Reward Over-Optimization
Inference-Time Scaling for Flow Models via Stochastic Generation and Rollover Budget Forcing

Video Diffusion Models as World Simulators

Fri, 17 Jan 2025 00:00:00 GMT

Overview

World simulators are explorable and interactive systems or models that can mimic real world. Advanced video generation models can function as world simulators, and to achieve it, they should have low latency for input actions, and capable of long sequence generation. Long sequence generation includes capability of long generation itself, preventing error accumulation, and long term context preservation. This post mainly focuses on how related project Oasis: A Universe in a transformer[1] deals with long sequence generation.

Conventional Long Video Generation are inappropriate for World Simulator!

Video Diffusion Models (VDMs)

|| |:--| | Training and sampling of video diffusion models. Darker tokens has higher noise levels. |

Videos are sequential data. However, video diffusion models are trained and inferenced to denoise tokens of same noise levels, interpreting each video clip as a single object. This section reviews approaches to generate long videos using aforementioned VDMs.

Chunked autoregressive methods

Small $k$ (e.g. $k=1$) results in high latency for each action since $f-k$ frames are output for each action. Also it tends to lose contexts.
Large $k$ (e.g. $k=f-1$) results in ineifficient training and inference since models learn only $f-k$ tokens, while models calculate for $f$ tokens.
Chunked autoregressive methods suffers from quality degradation originated from error accumulation.

Hierarchical methods (Multi-stage generation)

It does not fit to interactive generation since the end of the video is already determined.

Conventional approaches are not appropriate for wolrd simulator!

Long Sequence Generation in Oasis

Capability of long sequence generation

Oasis[1] follows Diffusion Forcing[2] to train models for long video generation. Diffusion Forcing inherits advantages of Teacher Forcing and Diffusion Models: flexible time horizon from Teacher Forcing, guidance at sampling from Diffusion Models.

Diffusion Forcing trains models to denoise tokens with independent noise levels, and sampling noise schedules are carefully chosen depending on the purpose. The training offers cheaper training than next-token prediction in video domain, and the complexity added by independent noise level is not excessive since the complexity is only in temporal dimension.

|| |:--:| | Training in Diffusion Forcing |

|| |:--:| | Sampling in Diffusion Forcing |

Preventing error accumulation

The reason of error accumulation

Oasis[1] and Diffusion Forcing[2] hypothesize that the error accumulation stems from the model erroneously treating generated noisy frames as grount truth (GT), despite their inherent inaccuracies. They interpret input noise levels to the models as inversely proportional to the confidence in the corresponding input tokens.

Stable rollout in Diffusion Forcing

Diffusion Forcing suggests to deceive models that generated clean tokens are little noisy, preventing models from believing generated tokens as GT. However, this approach is out of distribution (OOD) inference, and there is no rule of thumb for "little noisy".

Stable rollout (Another option)

To avoid OOD, one may suggest add little noise to generated tokens and tell models that the tokens are noisy. However, this approach may dilute details in generated tokens.

Dynamic Noise Augmentation (DNA)

Oasis[1] suggests Dynamic Noise Augmentation (DNA) to mitigate error accumulation.

For initial denoising steps, conditioning tokens (generated tokens) are moderately noised since models tend to generate low-frequency features during initial steps.
For last denoising steps, noise levels of conditioning tokens gradually decreases.

Long term context preservation

Through above approaches, Oasis can autoregressively generate long videos without much quality degradation. However, models do not have long time horizon memory, leading to inconsistent videos. While there is no innovative breakthrough yet, I believe that video models with long-term memory is an important next step.

References

Etched Decard, Oasis: A Universe in a Transformer (2024)

Boyuan Chen et al., Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion (NeurIPS, 2024)

Controllabilities in Video Diffusion Models

Tue, 14 May 2024 00:00:00 GMT

This presentation file includs videos. You may use pdf viewer that supports video playing such as Adobe Acrobat reader.

Designing Diffusion Models in Real World

Thu, 26 Oct 2023 00:00:00 GMT

Overview

In deep learning, practical implementations are just as important as theoretical supports. Especially when proposing new paradigms, such as GANs, diffusion models, and transformers, etc., engineering skills are essential to bring the paradigms into the real world. Even if the suggested designs on diffusion models in Elucidating the Design Space of Diffusion-Based Generative Models (EDM)[1] are not optimal, the choices of the designs are theoretically or empirically supported. Learning these reasonings may help to bring your theory into the real world.

Revisit Diffusion Models

Reformulate diffusion models

Score-Based Model[2] defines forward SDE and marginal distribution is calculated from forward SDE. However when it comes to training, marginal distribution is more important than the SDE and therefore EDM[1] defines the marginal distribution first.

$$ \begin{align} p_t(\mathrm{x}) = s(t)^{-d}p(\mathrm{x} /s(t);\sigma(t)), \end{align} $$

where $p(\mathrm{x};\sigma) = \leftp_{\text{data}} * \mathcal{N}(\mathrm{0}, \sigma^2\mathrm{I})\right$.

Then, the corresponding probability flow ODE is

$$ \begin{align} d\mathrm{x} = \left[\dot{s}(t)/s(t) - s(t)^2 \dot{\sigma}(t)\sigma(t) \nabla_\mathrm{x} \log p(\mathrm{x}_t/s(t);\sigma(t)) \right] dt. \label{edm:ode} \end{align} $$

Obstacles in diffusion models

Generation by diffusion models can interpreted as solving ODE:

$$ \begin{align} d\mathrm{x} = f(\mathrm{x}_t, s(t), \sigma(t))dt. \end{align} $$

$f(\mathrm{x}t, s(t), \sigma(t))$ is not known and it is parametrized by a network $f\theta(\mathrm{x}_t, s(t), \sigma(t))$. The inaccurate approximation on the target causes degradation. \ → Better training!
The solution at $t=0$ given boundary condition at $t=T$ is \

$$ \begin{align} \mathrm{x}_0 = \mathrm{x}_T + \int_0^T f(\mathrm{x}_t, s(t), \sigma(t))dt. \end{align} $$

The integral is numerically calculated, which causes truncation errors. \ → Reduce truncation errors, focus on important region!

Design space of diffusion models

Components regarding training

Parametrization and network preconditioning: $c_\text{skip}(\sigma)$, $c_\text{out}(\sigma)$, $c_\text{in}(\sigma)$, $c_\text{noise}(\sigma)$.
Loss weighting: $\lambda(t)$.
Noise level distribution for training: $\sigma \sim p_{\text{noise}}$.
Augmentation

Components regarding deterministic sampling

Truncation-error-reducing ODE: $s(t)$, $\sigma(t)$.
Truncation-error-reducing algorithms: Higher-roder integrators
Distributing truncation errors properly: Discretization ${t_i}_0^N$

Components regarding stochastic sampling

Rate of replaced noises: $\beta(t)$
Heuristics: $S_{\text{tmin}}, S_{\text{tmax}}, S_{\text{noise}}, S_{\text{churn}}$.

Improvements to Training

For this section, this post assumes $s(t)=1$.

Parametrization, network preconditioning, loss weighting

$D(\mathrm{x}_t,\sigma)$ is a denoiser which minimizes $\ell_2$-norm with $\mathrm{y}$:

$$ \begin{align} \mathbb{E}{\mathrm{y} \sim p{\text{data}}} \mathbb{E}||D(\mathrm{y} + \mathrm{n}) - \mathrm{y}||_2^2. \label{eq:loss} \end{align} $$

Then, the relation between a score function and the ideal denoiser is

$$ \begin{align} \nabla_{\mathrm{x}} \log p(\mathrm{x};\sigma) = (D(\mathrm{x};\sigma) - \mathrm{x}) / \sigma^2. \end{align} $$

Networks in many baselines predicts either $D(\mathrm{x},\sigma)$ or $\mathrm{n}$. However, Dynamic dual-output diffusion models[3] observes that predicting $D(\mathrm{x},\sigma)$ is easier for high noise level, while predicting $\mathrm{n}$ is easier for low noise level.

|| |:--:| | Loss comparison between predicting the denoised output or the added noise.|

From the observation, EDM[1] designs the network to predict $D(\mathrm{x};\sigma)$ or $\mathrm{n}$, or something in between according to the noise level.

$$ \begin{align} D_\theta(\mathrm{x};\sigma) = c_{\text{skip}}(\sigma)\mathrm{x} + c_{\text{out}}(\sigma) F_\theta(c_{\text{in}}(\sigma)\mathrm{x}; c_{\text{noise}}(\sigma)), \end{align} $$

where $F_\theta$ is a neural network.

Then the loss function ($\ref{eq:loss}$) is

$$ \begin{align} \mathbb{E}{\sigma,\mathrm{y},\mathbf{b}}\left[ \underbrace{\lambda(\sigma)c{\text{out}}(\sigma)^2}{\text{effective weight}} ||\underbrace{F\theta(c_{\text{in}}(\sigma)(\mathrm{y}+\mathbf{n};c_{\text{noise}}(\sigma)))}{\text{network output}} - \underbrace{\frac{1}{c{\text{out}}(\sigma)}(\mathrm{y} - c_{\text{skip}}(\sigma)(\mathrm{y}+\textbf{n}))}_{\text{effective training target}}||_2^2 \right]. \end{align} $$

1.Network inputs should have bounded range

$$ \begin{align} \Rightarrow &\text{Var}{\mathrm{y},\mathbf{n}}\left[c{\text{in}}(\sigma)(\mathrm{y} + \mathbf{n})\right] = 1 \ \Rightarrow &c_{\text{in}}(\sigma) = 1 / \sqrt{\sigma^2 + \sigma_{\text{data}}^2}\ \text{& } &c_{\text{noise}}(\sigma) = \log (\sigma)/4 \end{align} $$

2.Effective training target should have bounded range

$$ \begin{align} &\Rightarrow \text{Var}{\mathrm{y},\mathbf{n}}\left[\frac{1}{c{\text{out}}(\sigma)}(\mathrm{y} - c_{\text{skip}}(\sigma)(\mathrm{y}+\textbf{n}))\right] = 1 \ &\Rightarrow c_{\text{out}}(\sigma)^2 = (1-c_{\text{skip}}(\sigma))^2\sigma_{\text{data}}^2 + c_{\text{skip}}(\sigma)^2 \sigma^2 \end{align} $$

3.Errors of network should not be amplified

$$ \begin{align} \Rightarrow~ &c_{\text{skip}}(\sigma) = \underset{c_{\text{skip}}(\sigma)}{\text{argmin}} ~c_{\text{out}}(\sigma) \ \Rightarrow~ &\begin{cases} c_{\text{skip}}(\sigma) = \sigma_{\text{data}}^2 / (\sigma^2 + \sigma_{\text{data}}^2) \ c_{\text{out}}(\sigma) = \sigma \cdot \sigma_{\text{data}} / \sqrt{\sigma^2 + \sigma_{\text{data}}^2} \end{cases} \end{align} $$

4. Effecitve weight should be uniform

$$ \begin{align} \Rightarrow~ &\lambda(\sigma)c_{\text{out}}(\sigma)^2 = 1 \ \Rightarrow~ & \lambda(\sigma) = (\sigma^2 + \sigma_{\text{data}}^2)/(\sigma \cdot \sigma_{\text{data}})^2 \end{align} $$

Putting 1 ~ 4 together, the expected value of the loss at each noise level is 1. Moreover, the change of effective training target according to $\sigma$ coincides to the observation of Dynamic dual-output diffusion models[3].

Noise level distribution for training

|| |:--| | Observed loss per noise level. The shaded regions represent the standard deviation over 10k random samples. EDM's proposed training sample density is shown by the dashed red curve.|

At low noise levels, seperating the small noise components is difficult and irrelevant, whereas at high noise levels, the correct answer approaches to dataset average; EDM[1] focuses on middle range noise levels for training: $\sigma \sim \mathcal{N}(-1.2, 1.2)$.

Augmentataion

EDM[1] follows the augmentation pipiline from the GAN literature[4].

Each agmentation is enabled with $A_{\text{prob}}$.
Draw $a_i$ from each enabled augmentation and construct transformation matrix.
Pass data through $2\times$ supersampled high-quality Wavelet filters.
Construct a 9-dimensional conditioning input vector for non-leaking augmentation. This vector makes the network to perform auxiliarty tasks.

Improvements to Deterministic Sampling

Higher-order integrators

For $s(t)=1$ and $\sigma(t)=t$, the ODE to solve is

$$ \begin{align} d\mathrm{x}/dt = (\mathrm{x}_t - D(\mathrm{x}_t;t))/t := f(\mathrm{x}_t,t). \end{align} $$

Euler method

Euler method approximates the integral by

$$ \begin{align} \int_{t_{i}}^{t_{i-1}} f(\mathrm{x}t,t) dt = (t{i-1} - t_{i})f(\mathrm{x}{t_i},t_i) + O(|t{i-1}-t_{i}|^2). \end{align} $$

Therefore, the total truncation error is $O(\max \lvert t_{i-1}-t_{i} \rvert)$. Let $\hat{\mathrm{x}}{t{i-1}}$ is a solution obtained by Euler method.

Heun's method

Then, Heun's method approximates the integral by

$$ \begin{align} \int_{t_{i}}^{t_{i-1}} f(\mathrm{x}t,t) dt = (t{i-1} - t_{i})(f(\mathrm{x}{t_i},t_i)+ f(\hat{\mathrm{x}}{t_{i-1}},t_{i-1}))/2+ O(|t_{i-1}-t_{i}|^3). \end{align} $$

Therefore, the total truncation error is $O(\max{\lvert t_{i-1}-t_{i}|^2 \rvert})$. Huen's method decreases truncation error at the cost of one additional evaluation of the network.

Deterministic sampling algorithm for EDM[1]

Discretization

As long as using numerical integrators with limited computational resources, truncation errors are inevitable. In terms of obtaining ODE trajectories accurately, it is important to minimize total truncation erros. Hoever, the interests of diffusion models at generation are only the solutions at low noise levels; it is reasonable to focus on low noise levels. EDM discretizes as

$$ \begin{align} t_{N-i} = \sigma_{i< N} = (\sigma_{\text{max}}^{1/\rho} + \frac{i}{N-1} (\sigma_{\text{min}}^{1/\rho} - \sigma_{\text{max}}^{1/\rho}))^\rho, \sigma_N = 0. \end{align} $$

Increasing $\rho$ results dense discretizations at low noise levels.

|| |:--:| |(a),(b) Local truncation error at different noise levels. (c) FID as a function of $\rho$.|

$\rho=3$ nearly equalizes the truncation error at each step as in (a), (b). On the other hand, $\rho=7$ generates better samples as in (c). Proper value of $\rho$ changes according to the tasks. e.g., equalized truncation error will be better for solving ODE in both directions.

Truncation-error-reducing ODE

Many integrators including Euler and Heun's method have small truncation errors if $f(\mathrm{x}_t,t)$ has small curvature, or is close to linear function. $s(t)$ and $\sigma(t)$ determine the shape of the ODE solution trajectories, which is closely related to linearity of the $f(\cdot)$.

$$ \int_{t_{i}}^{t_{i-1}} f(\mathrm{x}t,t) dt \approx \begin{cases} (t{i-1} - t_{i})f(\mathrm{x}{t_i},t_i) & \text{Euler method} \ (t{i-1} - t_{i}) (f(\mathrm{x}{t_i},t_i)+ f(\hat{\mathrm{x}}{t_{i-1}},t_{i-1}))/2 & \text{Heun's method} \end{cases} $$

|| |:--| |A sketch of ODE curvature in 1D where $p_{\text{data}}$ is two Dirac peaks at $\mathrm{x}= \pm 1$. Axis is chosen to show $\sigma \in [0,25]$ and zoom in $\sigma \in [0,1]$. (c) sketches the curvature when $s(t)=1$ and $\sigma(t)=t$. It has small curvature, while the tangent directs to the datapoints.|

Results of deterministic sampling

Stochastic Sampling

SDE formulation

EDM[1] reformulates forward and backward SDE as a sum of the probability flow ODE and a varying-rate Langevin diffusion SDE:

$$ \begin{align} d\mathrm{x}{\pm} = \underbrace{-\dot{\sigma}(t)\sigma(t) \nabla\mathrm{x} \log p(\mathrm{x};\sigma(t)) dt}{\text{probability flow ODE}} \pm \underbrace{\underbrace{\beta(t)\sigma(t)^2 \nabla\mathrm{x} \log p(\mathrm{x};\sigma(t)) dt}{\text{deterministic noise decay}} + \underbrace{\sqrt{2\beta(t)}\sigma(t) d\mathrm{w}t}{\text{noise injection}}}{\text{Langevin diffusion SDE}} \end{align} $$

Role of stochasticity

In theory, ODE and SDE have the same marginal distributions. However in practice, stochasticity in sampling often enhances the sample quality. The authors attribute the beneficial role of stochasticity to the following steps:

$\mathrm{x}_t$ deviates from the ideal marginal distribution, due to the training and truncation errors.
The Langevin diffusion drives the sample towards the ideal marginal distribution.

Stochastic sampling algorithm in EDM

Stochastic sampling algorithm in EDM[1] is executed in two steps:

Noise injection: integrate noise into samples according to $\gamma_i \geq 0$.
Noise decay with probability flow: solve the ODE from increased noise level to desired level.

Algorithm in real world

|| |:--| |Observe the effect of Langevin diffusion in real world: there is gradual image degradation with the repeated addition and removal of noise. A random image is drawn from $p(\mathrm{x};\sigma)$ and Algorithm 2 is run for a certain number of steps with $\gamma_i=\sqrt{2}-1$.|

Langevin diffusion is supposed to drive the sample towards the true data distribution, however...

For low noise levels, images drift toward oversaturated colors.
For high noise levels, images become abstract when $s_{\text{noise}}=1$.

Authors suspect that non-conservative vector field generated by parametrized denoiser violates the premises of Langevin diffusion since their analytical denoisers have not shown such degradation.

$\Rightarrow$ Fix flaws of $D_\theta(\mathrm{x};\sigma)$ with heuristics!

For low noise levels, images drift toward oversaturated colors.\ $\Rightarrow$ Enable stochasticity within $t_i \in [S_{\text{tmin}}, \underline{S_{\text{tmax}}}]$.
For high noise levels, images become abstract when $S_{\text{noise}}=1$. \ $\Rightarrow$ $D_\theta(\cdot)$ removes too much noise because of regression towards the mean, which often happens when $\ell_2$ trained.\ $\Rightarrow$ Inflate the standard deviation of newly added noise: $S_{\text{noise}}>1$.
New noise never exceeds the noise already in the image. \ $\Rightarrow$ Clamp $\gamma_i$.
Controls the overal stochasticity by $S_{\text{churn}}$.

Results of stochastic sampling

|| |:--| |Evaluation of stochastic samplers with ablations. Red line is deterministic sampler while purple line is optimal stochastic sampler.|

References

Tero Karras et al., Elucidating the Design Space of Diffusion-Based Generative Models (NeurIPS, 2022)

Yang Song et al., Score-Based Generative Modeling through Stochastic Differential Equations (ICLR, 2021)

Yaniv Benny and Lior Wolf, Dynamic Dual-Output Diffusion Models (CVPR, 2022)

Tero Karras et al., Training generative adversarial networks with limited data (NeurIPS, 2020)

Understanding Diffusion Models in Two Perspectives

Sun, 03 Sep 2023 00:00:00 GMT

Overview

DDPM[1] and Score-Based Model[2] introduce diffusion model as a new paradigm of generative models. Since the concepts of both papers are similar, one might regard Score-Based Model[2] as only a continuous version of DDPM[1]. Two papers have slight different views, even their loss functions and implementations are similar.

This post mainly explains how formulations and objectives of two papers are different, and how they are related even with the differences.

Summary of the post

The objective of DDPM[1] is to minimize the surrogate of the negative log-likelihood.
The objective of Score-Based Model[2] is to match marginal distributions of forward SDE and backward SDE/ODE.
Even with the differences, both derivations require the score function, differential of the log of the probability density function. The score functions are parametrized by neural networks and both papaers have similar loss functions.

Maximizing Log-Likelihood

Forward (Diffusion) Process

The forward process is a Markov chain that gradually adds Gaussian noise to the data for $T$ steps with distributions defined as follows:

$$ \begin{align} &\mathrm{x}t \perp\mkern-9.5mu\perp \mathrm{x}{0:t-1}, \ &q(\mathrm{x}0) := \mathrm{P}{data}(\mathrm{x}_0), \ &q(\mathrm{x}t|\mathrm{x}{t-1}) := \mathcal{N}(\mathrm{x}t;\sqrt{1-\beta_t}\mathrm{x}{t-1}, \beta_t \mathrm{I}), \end{align} $$

where ${\beta_t}_{t=1}^T$ are pre-defined constants.

Backward (Denoising) Process

The backward process is a Markov chain that gradually denoises perturbed data and it is parametrized by neural networks. When $\beta_t\ll 1$ the backward distribution can be approximated as

$$ \begin{align} q(\mathrm{x}{t-1}|\mathrm{x}{t}) \approx \mathcal{N}(\mathrm{x}{t-1}; \cfrac{1}{ \sqrt{1-\beta_t}}(\mathrm{x}{t} + \beta_t \nabla \log q (\mathrm{x}_t)), \beta_t \mathrm{I}). \end{align} $$

It is reasonable to parametrize the denoising distribution as Gaussian as long as ${\beta_t}_{t=1}^T$ are infinitesimal. Therefore the bacward process is defined as follows:

$$ \begin{align} &\mathrm{x}t \perp\mkern-9.5mu\perp \mathrm{x}{t+1:T}, \ &p(\mathrm{x}T) := \mathcal{N}(\mathrm{x}T; \mathrm{0}, \mathrm{I}) \ &p(\mathrm{x}{t-1}|\mathrm{x}{t}) = \mathcal{N}(\mathrm{x}{t-1}; \cfrac{1}{ \sqrt{1-\beta_t}}(\mathrm{x}{t} + \beta_t s_\theta(\mathrm{x}_t,t)), \beta_t \mathrm{I}), \end{align} $$

Note that we expect $s_\theta(\mathrm{x}_t,t)$ to learn $\nabla\log q(\mathrm{x}_t)$.

Minimizing Surrogate of Negative Log-Likelihood

The negative log-likelihood of data is

$$ \begin{align} \mathbb{E}{\mathrm{x}0 \sim q} \left[-\log p(\mathrm{x}0)\right] &\leq \mathbb{E}{\mathrm{x}0 \sim q} \mathbb{E}{\mathrm{x}{1:T|0} \sim q} \left[ \log \cfrac{q(\mathrm{x}{1:T}|\mathrm{x}{0})}{p(\mathrm{x}{0:T})} \right]. \end{align} $$

$$ \begin{align*} -\log p(\mathrm{x}0) &= -\log \int p(\mathrm{x}{0:T}) d\mathrm{x}{1:T} \ &= -\log \int q(\mathrm{x}{1:T}|\mathrm{x}{0}) \cfrac{p(\mathrm{x}{0:T})}{q(\mathrm{x}{1:T}|\mathrm{x}{0})} d\mathrm{x}{1:T} \ &\leq -\int q(\mathrm{x}{1:T}|\mathrm{x}{0}) \log \cfrac{p(\mathrm{x}{0:T})}{q(\mathrm{x}{1:T}|\mathrm{x}{0})} d\mathrm{x}{1:T} ~~(\because \text{Jensen})\ &= \mathbb{E}{\mathrm{x}{1:T|0} \sim q} \left[ \log \cfrac{q(\mathrm{x}{1:T}|\mathrm{x}{0})}{p(\mathrm{x}{0:T})} \right]. \end{align*} $$

Using Markov properties,

$$ \begin{align} &q(\mathrm{x}_{1:T} | \mathrm{x}_0) = q(\mathrm{x}T | \mathrm{x}0) \prod{t=2}^T q(\mathrm{x}{t-1} | \mathrm{x}t, \mathrm{x}0), \ &p(\mathrm{x}{T:0}) = p(\mathrm{x}T) \prod{t=T}^{1} p(\mathrm{x}{t-1}|\mathrm{x}_t). \end{align} $$

$$ \begin{align*} q(\mathrm{x}{1:T} | \mathrm{x}0) &= \prod{t=1}^{T} q(\mathrm{x}{t}|\mathrm{x}{0:t-1}) \
&= \prod{t=1}^{T} q(\mathrm{x}{t}|\mathrm{x}{t-1}) \ &= q(\mathrm{x}1|\mathrm{x}0)\prod{t=2}^{T} q(\mathrm{x}{t}|\mathrm{x}{t-1}, \mathrm{x}{0}) \ &= q(\mathrm{x}1|\mathrm{x}0)\prod{t=2}^{T} \frac{q(\mathrm{x}{t},\mathrm{x}{t-1}| \mathrm{x}{0})}{q(\mathrm{x}{t-1}| \mathrm{x}{0})} \ &= q(\mathrm{x}1|\mathrm{x}0)\prod{t=2}^{T} \frac{q(\mathrm{x}{t}|\mathrm{x}{0})q(\mathrm{x}{t-1}| \mathrm{x}{t},\mathrm{x}{0})}{q(\mathrm{x}{t-1}| \mathrm{x}{0})} \ &= q(\mathrm{x}T | \mathrm{x}0) \prod{t=2}^T q(\mathrm{x}{t-1} | \mathrm{x}t, \mathrm{x}0), \ p(\mathrm{x}{T:0}) &= p(\mathrm{x}T) \prod{t=T}^{1} p(\mathrm{x}{t-1}|\mathrm{x}{T:t})\ &= p(\mathrm{x}T) \prod{t=T}^{1} p(\mathrm{x}{t-1}|\mathrm{x}_t). \end{align*} $$

Therefore, the surrogate of negative log-likelihood becomes

$$ \begin{align} &D_{KL}(q(\mathrm{x}_T|\mathrm{x}_0) || p(\mathrm{x}_T)) + \mathbb{E}q\left[-\log p(\mathrm{x}0|\mathrm{x}1)\right] \nonumber \ &+ \sum{t=2}^T D{KL}(q(\mathrm{x}{t-1} | \mathrm{x}_t, \mathrm{x}0) || p(\mathrm{x}{t-1}|\mathrm{x}_t)). \end{align} $$

The surrogate of negative log-likelihood can be explictly expressed using

$$ \begin{align} &p(\mathrm{x}{t-1}|\mathrm{x}{t}) = \mathcal{N}(\mathrm{x}{t-1}; \cfrac{1}{ \sqrt{1-\beta_t}}(\mathrm{x}{t} + \beta_t s_\theta(\mathrm{x}t,t)), \beta_t \mathrm{I}), \ &q(\mathrm{x}{t-1}|\mathrm{x}{t}, \mathrm{x}{0}) = \mathcal{N}(\mathrm{x}{t-1}; \cfrac{1}{ \sqrt{1-\beta_t}}(\mathrm{x}{t} + \beta_t \nabla \log q (\mathrm{x}t|\mathrm{x}{0})), \frac{1-\bar\alpha_{t-1}}{1-\bar\alpha_t}\beta_t \mathrm{I}), \end{align} $$

where $\bar\alpha_t = \prod_{s=1}^t (1-\beta_s)$.

Finally, the objective function becomes

$$ \begin{align} \sum_{t=1}^T \mathbb{E}{\mathrm{x}0}\mathbb{E}{\mathrm{x}{t}|\mathrm{x}{0}} \left[ \lambda_t ||s\theta(\mathrm{x}_t,t) - \nabla \log q(\mathrm{x}_t|\mathrm{x}_0)||_2^2 \right], \end{align} $$

where $\lambda_t$ are some constants.

Matching Marginal Distributions

Forward SDE

For pre-defined function $f:\mathbb{R}^{h\times w \times 3}\times \mathbb{R} \rightarrow \mathbb{R}^{h\times w \times 3}$ and $g:\mathbb{R} \rightarrow \mathbb{R}$, a forward SDE perturbs the data with Gaussian noise by

$$ \begin{align} d\mathrm{x}_t = f(\mathrm{x}_t,t)dt + g(t)d\mathrm{w}_t, ~~\text{and}~~ \mathrm{x}0 \sim \mathrm{P}{data}, \end{align} $$

where $\mathrm{w}_t$ is Brownian process.

If ${\mathrm{x}t}{t=0}^T$ is a solution of the forward SDE, it can be treated as a sample from the joint distribution ${p_t}_{t=0}^T$. However, learning joint distribution is difficult and our interest is only $\mathrm{x}_0$, not ${\mathrm{x}t}{t=0}^T$. Therefore, it suffices to consider weakened objective, learning how marginal distributions evolve as $t$ changes. The evolution of the marginal distributions is goverened by the Fokker-Plank equation:

$$ \begin{align} \partial_t p_t = - \nabla_x (f \cdot p_t ) + \frac{1}{2} \mathrm{tr}(g^T ~\nabla_x^2p_t~ g). \end{align} $$

Backward SDE/ODE

Following backward SDE and ODE are known to have the same marginal distributions:

$$ \begin{align} &d\mathrm{x}_t = \left[ f(\mathrm{x}_t,t)dt - g^2(t) \nabla \log p_t(\mathrm{x}_t) \right]dt + g(t)d\bar{\mathrm{w}}_t , ~~\text{and}~~ \mathrm{x}_T \sim \mathcal{N}(\mathrm{0}, \mathrm{I}), \ &d\mathrm{x}_t = \left[ f(\mathrm{x}_t,t)dt - \frac{1}{2} g^2(t) \nabla \log p_t(\mathrm{x}_t) \right]dt , ~~\text{and}~~ \mathrm{x}_T \sim \mathcal{N}(\mathrm{0}, \mathrm{I}), \end{align} $$

where $\bar{\mathrm{w}}_t$ is the reverse-time Brownian motion.

Since $f(\cdot, \cdot)$ and $g(\cdot)$ are known, the only unknown component in backward SDE/ODE is $\nabla \log p_t (\cdot)$ which is also known as a score function. The score function is parametrized by neural network, $s_\theta(\mathrm{x}_t,t)$.

Learning Score Function

Since we parametrized the score function with the neural network, we can consider a loss function of

$$ \begin{align} \int_{0}^{T} \lambda_t \mathbb{E}_{\mathrm{x}t} \left[ ||s\theta(\mathrm{x}_t,t) - \nabla \log p_t(\mathrm{x}_t)||_2^2 \right] dt, \end{align} $$

where $\lambda_t$ are some constants. Note that $\nabla\log p_t(\mathrm{x}_t)$ is intractable and with some tricks, the loss function changes into tractable form:

$$ \begin{align} \int_{0}^{T} \lambda_t \mathbb{E}{\mathrm{x}0}\mathbb{E}{\mathrm{x}{t}|\mathrm{x}{0}} \left[ ||s\theta(\mathrm{x}t,t) - \nabla \log p{t|0}(\mathrm{x}_t|\mathrm{x}_0)||_2^2 \right] dt. \end{align} $$

References

Jonathan Ho et al., Denoising diffusion probabilistic models (NeurIPS, 2020)

Yang Song et al., Score-Based Generative Modeling through Stochastic Differential Equations (ICLR, 2021)