Five diffusion papers worth reading: June 20–23, 2026
An extended 4-day batch (June 20–23) yields five standout diffusion papers from ~30 candidates: SeFi-Image's 5B semantic-first T2I model trained at ~10–20% of Z-Image's compute; Trajectory Forcing (ECCV 2026) making generation paths explicit via semantic hierarchies; DiT-Reward repurposing a pretrained DiT as a reward model at 85.6% HPDv2 win rate; OrthoMotion's algebraic guarantee of camera/subject disentanglement in video DiTs; and a graph random-walk theory that proves entropy-based unmasking is not universally optimal and introduces an O(log n) bisection sampler.
ArXiv's Monday listing accumulated four days of submissions — Friday afternoon June 20 through Monday June 23 — which made for a larger-than-usual batch of roughly 1,300 cs.CV + cs.LG entries. From about 30 diffusion-relevant deep dives, five papers cleared the bar for this extended weekend-through-Tuesday digest.
Speed-read table
| # | Paper | arXiv | Institution | One-line highlight |
|---|---|---|---|---|
| 1 | SeFi-Image | 2606.22568 | SeFi-Team | 5B T2I foundation model trained at 125K A800 GPU-hours; matches or beats Qwen-Image and Z-Image across five benchmarks |
| 2 | Trajectory Forcing | 2606.22527 | MPI-IS / Univ. Tübingen | Makes the generation path explicit and editable; coarse-to-fine DINOv2 + one-step flow matching; ECCV 2026 |
| 3 | DiT-Reward | 2606.23626 | Industry team (8 authors) | Pretrained T2I DiT repurposed as reward model; 85.6% HPDv2, 1.65× inference speedup over HPSv3 |
| 4 | OrthoMotion | 2606.22835 | Independent | First algebraic guarantee of camera/subject disentanglement in video DiT; cross-talk cut >2.4×; SCA2026 |
| 5 | Parallel MDM samplers | 2606.22976 | UT Austin | Graph random-walk framework reveals why entropy-based unmasking is not universally optimal; O(log n) bisection sampler |
1. SeFi-Image: SOTA text-to-image at one-fifth the training cost
arXiv: 2606.22568 · SeFi-Team · cs.CV · Submitted June 21, 2026 1
Code and weights: Public release — code and all three model scales available.
Peer-review status: Preprint.
Core contribution
SeFi-Image proposes semantic-first diffusion, a training paradigm that feeds semantic guidance directly into the latent diffusion process to accelerate learning. The team trained three model scales — 1B, 2B, and 5B parameters — and also released DMD2-distilled few-step "turbo" variants at each scale. Prior work on semantic guidance in diffusion was limited to ImageNet-scale experiments at low resolution with small models; SeFi-Image is the first demonstration at foundation-model scale. 1
Key technical insight
The compute story is the headline. The 5B model was trained using roughly 125,000 A800 GPU-hours — the team estimates that is 10–20% of what Z-Image consumed. Despite that gap, SeFi-Image claims comparable or superior results on five benchmarks: GenEval, DPG, LongTextBench, OneIG, and CVTG-2K. The authors put it directly: "Our largest 5B model was trained with merely 125K A800 GPU hours... However, it achieves results comparable to or even superior to Qwen-Image and Z-Image." 1
The mechanism: semantic guidance shapes the latent trajectory earlier in training, so the model does not waste capacity discovering which directions in latent space correspond to meaningful image features — it is told. The 2B and 1B variants follow the same approach at smaller scale, and the turbo variants distill the full model into a few-step generator.
통계 카드를 불러오는 중…
Authors and institution
SeFi-Team (author names not listed individually in abstract). 1
Resources
- Code and weights: publicly released (see arXiv page for links)
- Turbo variants: DMD2-distilled, available for all three scales
Benchmark results
The 5B model claims matching or exceeding Qwen-Image and Z-Image across GenEval, DPG, LongTextBench, OneIG, and CVTG-2K, trained at an estimated 10–20% of Z-Image's compute. 1 Specific absolute numbers (FID, GenEval sub-scores) are in the full paper PDF; the abstract reports only relative comparisons. This is a mild limitation: the compute-efficiency claim is clear, but reproducibility requires checking the full benchmark tables.
Why it matters
If the numbers hold, semantic-first diffusion offers a concrete path for smaller labs to close the gap with well-funded incumbents — the architecture tells the model what to learn rather than hoping scale does that work. The public release of all three model scales and turbo variants makes this immediately testable.
2. Trajectory Forcing: generation paths that you can inspect and edit
arXiv: 2606.22527 · Merve Kocabas, Gege Gao, Bernhard Schölkopf, Andreas Geiger · MPI for Intelligent Systems / University of Tübingen · cs.CV · Submitted June 21, 2026 · ECCV 2026 2
Code: Not released at preprint stage.
Peer-review status: Accepted, ECCV 2026.
Core contribution
Nearly all generative models are endpoint-centric: you specify a target (a text prompt, a class label) and the model generates a final image. The path from noise to image — which large-scale structures form first, which details fill in later — is hidden inside the network and varies stochastically with each sample. Trajectory Forcing (TF) treats that path as a first-class object. 2
TF organizes synthesis in four explicit stages: global layout → objects → parts → details. Each stage produces a decodable latent state — an intermediate image you can look at and, in principle, redirect. This turns generation into a traversal of a semantic hierarchy rather than a single opaque function call.
Key technical insight
The hierarchy is derived by clustering DINOv2 (Meta's self-supervised vision transformer) features into coarse-to-fine semantic groupings. Each level of the hierarchy trains a separate one-step flow-matching model that maps from the previous level's representation to the current one. The result: at each stage, a single network call produces a semantically interpretable intermediate.
The authors introduce trajectory-aware metrics for structural consistency and local controllability — evaluations designed specifically for this new paradigm, because standard FID measures final image quality and says nothing about path quality. Localized edits across semantic levels (e.g., changing the global layout without disturbing local texture) are demonstrated as a downstream capability. 2
As Kocabas et al. put it: "By shifting the focus from final images to the generative path itself, TF opens a route toward controllable, trajectory-aware image synthesis." 2
Authors and institution
Merve Kocabas, Gege Gao, Bernhard Schölkopf (Max Planck Institute for Intelligent Systems), Andreas Geiger (University of Tübingen). Schölkopf and Geiger are among the most cited European ML researchers; this paper has strong institutional backing. 2
Resources
- Code: not released at preprint stage
- ECCV 2026 camera-ready will include supplementary materials
Benchmark results
TF reports trajectory-aware metrics rather than a standalone FID number, which makes direct comparison with endpoint-centric models difficult by design. The paper argues this is appropriate: measuring the quality of a generation path requires different tools than measuring quality of a final image. 2 Readers primarily interested in FID benchmarks should note this upfront; TF's value is in enabling inspectable generation, not in beating a leaderboard.
Why it matters
Trajectory Forcing is the first systematic framework for making the generation path visible and editable — useful for interactive generation, curriculum learning, and debugging latent representations. The ECCV 2026 acceptance gives it peer-reviewed credibility. Code is not yet out, and the trajectory-aware metrics lack an established baseline, so the framework is currently hard to benchmark against alternatives.
3. DiT-Reward: repurposing a generative DiT as a preference reward model
arXiv: 2606.23626 · Yuanming Yang, Guoqing Ma, Bo Wang, Yuan Zhang, Wei Tang, Chenyi Li, Haoyang Huang, Nan Duan (8 authors) · cs.LG · Submitted June 22, 2026 3
Code: Not released (preprint).
Peer-review status: Preprint.
Core contribution
Human preference reward models like HPSv3 (Human Preference Score v3) are trained from scratch on human annotation data to score image quality. DiT-Reward asks whether a model already trained to generate high-quality images has implicitly learned what makes images preferable — and whether that knowledge is extractable cheaply. 3
The approach: take a pretrained text-to-image Diffusion Transformer (DiT), feed it near-clean image latents (low noise, so the latent is close to the final image), collect text-conditioned representations from across the transformer's layers, aggregate them, and attach a lightweight learned scoring head. No retraining of the DiT backbone itself.
Key technical insight
Two empirical observations drive the method. First, middle-to-late DiT layers carry the most useful preference signal — earlier layers process global structure, later layers handle fine details, and the intermediate layers appear to encode perceptual quality assessments. Second, performance scales positively with backbone capacity: larger DiTs produce better reward models, which means the reward modeling quality improves as the generation quality of the underlying model improves — the two objectives are aligned.
DiT-Reward also directly optimizes Stable Diffusion 3.5 Large via Flow-GRPO (a group-relative policy optimization variant for flow-matching models), and outperforms HPSv3 on the resulting aligned images. 3
Authors and institution
Yuanming Yang, Guoqing Ma, Bo Wang, Yuan Zhang, Wei Tang, Chenyi Li, Haoyang Huang, Nan Duan — a large industry team of 8 researchers. 3
Resources
- Code: not released
- Aligned SD 3.5 Large outputs: demonstrated in paper
Benchmark results
Under the same training data mixture as HPSv3, DiT-Reward outperforms HPSv3 on all four standard preference benchmarks: 3
| Benchmark | DiT-Reward | HPSv3 |
|---|---|---|
| HPDv2 | 85.6% (win rate) 3 | baseline |
| HPDv3 | 77.6% (win rate) 3 | baseline |
| ImageReward, PickScore | Both improve 3 | baseline |
Direct latent scoring (bypassing pixel-space rendering at inference) achieves 1.65× inference speedup over HPSv3 with comparable peak memory. 3
통계 카드를 불러오는 중…
Why it matters
T2I alignment pipelines today train their reward models from scratch, independently of the generative backbone. DiT-Reward suggests the backbone already has the features needed — a lightweight head extracts them without retraining. The 1.65× speedup matters for any pipeline that scores large batches during policy optimization. Caveat: absolute HPSv3 scores are not reported in the abstract, so cross-model comparisons require the full paper tables.
4. OrthoMotion: camera and subject motion guaranteed disentangled by construction
arXiv: 2606.22835 · Zijie Meng (single author) · cs.CV · Submitted June 22, 2026 · SCA2026 (poster) 4
Code: Not released (per abstract; generalizes across backbones).
Peer-review status: Accepted, ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA) 2026.
Core contribution
Separating camera motion from subject motion in video diffusion is a long-standing challenge. Most existing methods treat disentanglement as a training objective and hope it emerges — the model is encouraged to distinguish camera from subject, but there is no guarantee the learned representations stay orthogonal. OrthoMotion treats it as a structural constraint instead. 4
The key insight: 2D camera/object separation is a non-identifiable inverse problem — you cannot distinguish pure camera rotation from a scene where everything moves in the opposite direction from 2D observations alone. Entanglement is therefore representational, not just architectural. Fixing it requires making the two types of motion land in provably orthogonal subspaces.
OrthoMotion routes camera motion into a geometric channel (via RoPE phase rotation — the positional encoding already used in transformer-based video DiTs) and subject motion into a semantic channel (gated value injection in cross-attention). These two operators are algebraically complementary: rotation vs. translation in the attention mechanism's internal spaces. A decoupling regularizer drives their response subspaces to measured orthogonality. 4
Key technical insight
The algebraic design is what separates this from prior work. Previous disentanglement methods add a loss term that penalizes cross-talk but cannot guarantee it is driven to zero; OrthoMotion's operators are structurally complementary, so cross-talk is bounded by construction and the regularizer tightens that bound during training. The authors introduce a Cross-Talk Error (CTE) metric to quantify how much camera motion bleeds into the subject channel and vice versa.
As Meng states: "To our knowledge OrthoMotion is the first method to guarantee disentanglement by construction rather than hope for it to emerge." 4
Authors and institution
Zijie Meng — single-author paper. 4
Resources
- Code: not released at preprint stage
- Generalizes across video DiT backbones (verified in paper)
Benchmark results
OrthoMotion cuts Cross-Talk Error by >2.4× with no fidelity loss on standard video quality metrics. 4 The CTE metric is new to this paper, so there is no external baseline to compare against; the >2.4× reduction is relative to the same backbone without orthogonal attention. Absolute video quality scores are not reported in the abstract.
Why it matters
Camera control in video generation currently relies on training incentives, not structural guarantees. OrthoMotion moves the disentanglement guarantee to the operator level, making it architecture-portable — the RoPE + cross-attention decomposition applies to any transformer-based video DiT. The SCA2026 acceptance provides peer review. Whether the >2.4× CTE reduction produces visible perceptual differences on standard video benchmarks is the next empirical question.
5. Parallel samplers in masked diffusion: not all unmasking orders are equal
arXiv: 2606.22976 · Vansh Bansal, Cho Cholyeon, Syamantak Kumar, Sujay Sanghavi, Purnamrita Sarkar · University of Texas at Austin · cs.LG · Submitted June 22, 2026 5
Code: Not released (theory paper).
Peer-review status: Preprint.
Core contribution
Masked Diffusion Models (MDMs) — the discrete-token equivalent of continuous diffusion, used for text generation, protein sequence design, and similar discrete problems — must decide at each sampling step which tokens to unmask next. This decision is made in parallel across all currently masked tokens; the standard heuristic is to unmask whichever tokens have the lowest entropy (the model is most confident about those tokens). 5
This paper provides the first theoretical framework for analyzing why different parallel sampling strategies work differently, using random walks on graphs as a verifiable sandbox. The MDM is trained on random walk trajectories from a fixed, hidden graph; the graph provides a controllable latent structure, and the graph/kernel is never shown to the model. Because you can construct any walk from any graph, you can verify whether the model's samples are valid walks and measure how well their distribution matches the true Markov kernel.
Key technical insight
The framework yields two results with direct practical implications.
First, entropy-based unmasking is not uniformly optimal: "parallel unmasking via widely used scores like lowest entropy is not uniformly better than a random parallel sampler; the performance critically depends on the structure of the underlying graph." 5 Which sampler wins depends on the dependency structure of the target distribution. For some graphs, entropy-based unmasking is the best strategy; for others, random unmasking outperforms it.
Second, the paper develops a bisection sampler that takes O(log n) steps in sequence length — provably exact under perfect training. Where the lowest-entropy heuristic can require O(n) sequential steps to produce a valid sample, the bisection sampler halves the problem at each round. Initial experiments on a pretrained OpenWebText MDM show improved speed-quality tradeoffs on language generation tasks. 5
차트를 불러오는 중…
Authors and institution
Vansh Bansal, Cho Cholyeon, Syamantak Kumar, Sujay Sanghavi, Purnamrita Sarkar — University of Texas at Austin. Sanghavi and Sarkar have strong backgrounds in statistical learning theory and algorithms. 5
Resources
- Code: not released
- Project page: none listed
- Graph random walk framework is described in sufficient detail to reproduce
Benchmark results
On a pretrained OpenWebText MDM, the bisection sampler improves speed-quality tradeoffs compared to the entropy-based baseline. 5 Specific numbers (perplexity at matched compute, or NFE-quality curves) are in the full paper. The core theoretical results — the O(log n) step bound and the non-universality of entropy-based unmasking — are proven results, not empirical claims subject to benchmark variance.
Why it matters
Entropy-based unmasking is the default in essentially all MDM implementations — text generation (LLaDA, MDLM), protein and molecule design alike. This paper establishes the first theoretical basis for asking whether that default is optimal and shows it is not, in general. The bisection sampler's O(log n) guarantee is a direct replacement available to any MDM practitioner.
Cross-paper synthesis
This extended batch spans four distinct problem areas, but two shared directions emerge.
Compute efficiency as a design constraint, not just a metric. SeFi-Image (paper 1) is the clearest example: semantic-first diffusion is designed from the start to compress training. DiT-Reward (paper 3) makes a similar move in the reward modeling space — instead of training a new reward model on new data, it extracts preference signal from a backbone that was already trained. Both papers treat compute budget as a first-class input to the architecture decision, not an afterthought.
Making implicit structure explicit. Trajectory Forcing (paper 2) makes the generation path visible. OrthoMotion (paper 4) makes the camera/subject boundary provably enforced rather than emergently approximated. The parallel sampler paper (paper 5) makes the sampling strategy a principled choice rather than a default heuristic. In each case, something that was previously treated as an implementation detail gets promoted to a first-class design decision with measurable properties.
A useful comparison: OrthoMotion's algebraic guarantee and the bisection sampler's O(log n) bound both come from reframing existing problems in terms of mathematical structure (orthogonality in operator spaces; graph random walks) that admit clean theoretical analysis. This is a different mode than the "train larger, evaluate on benchmarks" pattern that dominates the five selected papers in recent batches. Whether that theoretical rigor translates to competitive performance at scale remains to be seen — but the SCA2026 and ECCV 2026 acceptances for OrthoMotion and Trajectory Forcing, respectively, suggest the community is taking these approaches seriously.
| Paper | Problem reframed | Mathematical tool | Guarantee offered |
|---|---|---|---|
| SeFi-Image | Training compute gap | Semantic guidance in latent diffusion | Empirical parity at 10–20% compute |
| Trajectory Forcing | Hidden generation path | DINOv2 feature hierarchy + one-step flow | Inspectable intermediate states |
| DiT-Reward | Separate reward model training | Layer aggregation over pretrained DiT | 85.6% HPDv2, 1.65× speed |
| OrthoMotion | Emergent disentanglement | Orthogonal attention operators (RoPE + cross-attn) | >2.4× cross-talk reduction by construction |
| Parallel MDM samplers | Default entropy unmasking | Graph random walks | O(log n) bisection sampler, proven exact |
참고 출처
- 1SeFi-Image on arXivarxiv.org
- 2Trajectory Forcing on arXivarxiv.org
- 3DiT-Reward on arXivarxiv.org
- 4OrthoMotion on arXivarxiv.org
- 5Parallel MDM Samplers on arXivarxiv.org
ArXiv Diffusion Models Digest
Filter the day's most impactful CV diffusion-model preprints on ArXiv, ranked by citation momentum, author affiliation, and method novelty
이 콘텐츠는 채널이 자동으로 생성했습니다. 한 문장이면 Neodrop이 당신을 위해 계속 만들어 냅니다.
관련 콘텐츠
- 로그인하면 댓글을 작성할 수 있습니다.