📚 Journal club on discrete diffusion models 🎥 Replays available on YouTube! Contact: [email protected] Hosted by @ssahoo_, @jdeschena, @zhihanyang_
Joined August 2025
- Tweets278
- Following0
- Followers2.4K
- Likes261
RT @JCJesseLai: 🎉 Excited to extend our book, The Principles of Diffusion Models, into a tutorial at @NeurIPSConf 2026!
This time, it’s al…
This quoted post is unavailable.
Discrete Diffusion Reading Group retweeted
🎊 Excited to give a tutorial on diffusion LLMs at @NeurIPSConf 2026 with @StefanoErmon @JCJesseLai @mittu1204
📆 Mon, Dec 7, 9 AM AEDT
📍 TikTok Ent Centre (Sydney, Australia)
Topics covered:
> A beginner friendly introduction to diffusion-LLMs
> Designing frontier diffusion-LLMs
> Using Continuous diffusion for LLMs
Discrete Diffusion Reading Group retweeted
Missed the presentation yesterday? We just uploaded the recording on YouTube: youtube.com/watch?v=J8rcc00l…
I especially tried to give some intuition for the sampler, which can look surprising on a first read!
This Monday, Oct 19, Justin Deschenaux (@jdeschena) will present his paper:
Simplex Diffusion Models
Discrete Diffusion Reading Group retweeted
Join our reading group next Monday!
Topic: Simplex Diffusion Models
Presenter: Justin Deschenaux
This Monday, Oct 19, Justin Deschenaux (@jdeschena) will present his paper:
Simplex Diffusion Models
Discrete Diffusion Reading Group retweeted
Really excited to see Uno-adapters incorporated into @nvidia -Nemo
Thanks @huiying_lii @krasul @Chonghan_Liu
Learn more about Uno: xitter.cf/ssahoo_/status/2095741…
We added Uno training to NeMo AutoModel.
The neat idea: let the model draft for itself, then verify with the original AR model — no separate draft model needed.
Our Qwen3-0.6B run gets ~2 committed tokens per forward on GSM8K and 1.84 on HumanEval.
PR: github.com/NVIDIA-NeMo/Autom…
cc @IFM_AI @ssahoo_
Thanks @krasul and @Chonghan_Liu for getting this in 🙏
Discrete Diffusion Reading Group retweeted
Join our reading group next Monday!
Topic: Gumbel Straight Flow: Distilling Autoregressive Models into One-Step Flow Maps
Presenter: Yeongmin Kim
Discrete Diffusion Reading Group retweeted
Excited to share my work during summer internship at NVIDIA! We build the first large-scale continuous dLLM, reaching comparable and even better performance compared with masked dLLM trained under aligned budgets, and discover interesting properties its continuous dynamics!
💫 Large Language Continuous Diffusion Models
Presenting Sigma, the first large-scale (3B/8B) continuous diffusion LM (dLM) trained with likelihood.
📈 Comparable with discrete dLMs and AR on math and coding
🎛️ Enables meaningful study of inference-time interventions
Jointly led with Wei Guo (@WeiGuo01), Jean-Marie Lemercier (@jm_lemercier), Simon Welker (@_cobalamin)
Team: @YongganFu @mmkamani7 Sajad Norouzi @julberner @tomasgeffner @karsten_kreis @YongxinChen1 @MoleiTaoMath @jwthickstun @PavloMolchanov Ante Jukic @ArashVahdat @MardaniMorteza
Discrete Diffusion Reading Group retweeted
💫 Large Language Continuous Diffusion Models
Presenting Sigma, the first large-scale (3B/8B) continuous diffusion LM (dLM) trained with likelihood.
📈 Comparable with discrete dLMs and AR on math and coding
🎛️ Enables meaningful study of inference-time interventions
Jointly led with Wei Guo (@WeiGuo01), Jean-Marie Lemercier (@jm_lemercier), Simon Welker (@_cobalamin)
Team: @YongganFu @mmkamani7 Sajad Norouzi @julberner @tomasgeffner @karsten_kreis @YongxinChen1 @MoleiTaoMath @jwthickstun @PavloMolchanov Ante Jukic @ArashVahdat @MardaniMorteza
Discrete Diffusion Reading Group retweeted
I’ll be speaking at the EDPM workshop about our recent paper:
“Unlocking Lossless Speedups in LLMs via Discrete Diffusion.”
📅 October 7, 10:50–11:30 AM
In NYC until October 7. Reach out if you’d like to meet up and talk shop!
Register for the Zoom Link: simonsfoundation.zoom.us/web…
Diffusion LLMs have two limitations relative to AR models:
(1) Lower quality, and
(2) Slower inference at large batch sizes.
We address this "Uno"
> Retains the AR architecture of LLMs
> Each layer has two sets of weights: AR weights and Diffusion weights
> Diffusion weights enable parallel sampling from the AR distribution losslessly
Results:
💥Faster than all speculative decoding methods: DFlash and EAGLE-3
🔥 Beats ALL diffusion LLMs: Mercury 2, Diffusion Gemma, Llada
Links to the paper, models, code below
Discrete Diffusion Reading Group retweeted
⭐️Continuous Diffusion Scales Competitively with Discrete Diffusion for Language
Has been accepted at NeurIPS 2026!
See you at the conference!
📢Excited to share our new paper:
Continuous Diffusion Scales Competitively with Discrete Diffusion for Language
We introduce RePlaid 🌊, a continuous diffusion language model (DLM) with
🏅Discrete likelihood bound
🏅Scaling laws competitive with SOTA discrete DLMs
How? Dive in👇[🧵1/12]
Paper: arxiv.org/abs/2605.18530
Work done with my amazing collaborators: @WeiGuo01 @ShuibaiZ69721 @ssahoo_ @YongxinChen1 @ArashVahdat @MardaniMorteza @jwthickstun
Discrete Diffusion Reading Group retweeted
Join our reading group next Monday!
Topic: DFlash: Block Diffusion for Flash Speculative Decoding
Presenter: Jian Chen (@jianchen1799)
📢Sep 21 (Mon): DFlash: Block Diffusion for Flash Speculative Decoding
🤔 Speculative decoding is meant to fix the sequential bottleneck in autoregressive LLMs — but the drafter is usually autoregressive too. The cure inherits the disease:
1️⃣ Drafting stays sequential, which caps the practical speedup
2️⃣ Diffusion LLMs can draft in parallel, but on their own they still underperform autoregressive models
🔑 The insight: speculative decoding is exactly the setting where a diffusion model shines. The target model verifies every draft, so the drafter's parallelism is pure upside — and the output stays lossless.
🔧 DFlash uses a lightweight block diffusion model as the drafter, with two pieces:
(1) Draft tokens are produced in a single forward pass — drafting stops being sequential
(2) The drafter is conditioned on context features extracted from the target model, which lifts draft quality and acceptance rates
📈 Over 6× lossless acceleration across a range of models and tasks — and up to 2.5× higher speedup than EAGLE-3, the state-of-the-art speculative decoding method.
This Monday, the lead author (@jianchen1799) will present DFlash.
Discrete Diffusion Reading Group retweeted
Today’s LLMs still write like typewriters: one token at a time. This sequential process creates a hard inference bottleneck.
We're introducing Uno, a diffusion-augmented LLM that delivers autoregressive quality at diffusion speed. It’s a lossless speedup method that accelerates generation without degrading response quality.
With Uno, K2-Horizon-7B outperforms state-of-the-art diffusion methods in both quality and throughput, delivering up to a 2.2× speedup with no loss in quality.
Paper: arxiv.org/abs/2609.04010
Model available at: huggingface.co/IFM/K2-Horizo…
Discrete Diffusion Reading Group retweeted
You should use diffusion adapters to speed up post-training.
It’s almost impossible to get speculative decoding to speed up RL, but with diffusion adapters, you can easily get into 30-40% speedup
Happy to chat more
Diffusion LLMs have two limitations relative to AR models:
(1) Lower quality, and
(2) Slower inference at large batch sizes.
We address this "Uno"
> Retains the AR architecture of LLMs
> Each layer has two sets of weights: AR weights and Diffusion weights
> Diffusion weights enable parallel sampling from the AR distribution losslessly
Results:
💥Faster than all speculative decoding methods: DFlash and EAGLE-3
🔥 Beats ALL diffusion LLMs: Mercury 2, Diffusion Gemma, Llada
Links to the paper, models, code below
Discrete Diffusion Reading Group retweeted
I’ll share new experimental results in this talk, including:
⚡️ Faster RL training
📊 Comparisons with a retrained DFlash model
📅 Sep 18, Friday (10am PT)
This week, @ssahoo_ presents diffusion-augmented LLMs:
🔥 AR-level Quality
⚡️ Diffusion-level speed
✨ A drop-in replacement for speculative decoding:
>> No separate draft model to train, plus
>> Faster RL post-training.
The method, Uno, beats all diffusion models by a wide margin in system throughput, agentic evals, and long-context reasoning.
📅 Sep 18, Friday
🕒 10am PT / 1PM ET
Discrete Diffusion Reading Group retweeted
Uno shows a simple way to speed up existing LLMs without changing their output distribution: keep the original model in charge, and use diffusion only to draft multiple tokens in parallel.
Keep the autoregressive model for quality and add diffusion for parallel drafting
On Qwen3-8B, Uno delivered 2.5X higher per-request throughput and 1.6X higher system throughput at the largest tested batch size.
Normal autoregressive LLMs generate 1 token at a time, making long responses, agents, and RL rollouts slow.
Uno adds lightweight diffusion adapters that propose several future tokens at once, while the original model verifies them.
That avoids a separate draft model and preserves the base model’s sampling behavior.
It also sped up end-to-end RL training by up to 40% in the reported runs.
Discrete Diffusion Reading Group retweeted
Thanks AK, for sharing our work.
Project page: s-sahoo.com/uno/
Github: github.com/ifm-ai/uno
HF: huggingface.co/collections/s…
Discrete Diffusion Reading Group retweeted
The 2B MDLM, Duo, Eso-LMs models are up on HF 🤗
(finally)
📢 New Paper: Scaling Beyond Masked Diffusion Language Models
Takeaways👇
🚫 Masked Diffusion ≠ endgame
🔥 Uniform-state beats AR on GSM8K
📉 Perplexity is a bad metric for comparing diffusion families
📰 arxiv.org/abs/2602.15014
🖥️ github.com/s-sahoo/scaling-d…
🌐 s-sahoo.com/scaling-dllms/
collaborators: @jm_lemercier @zhihanyang_ @jdeschena @Jingyu227 @jwthickstun Ante Jukic
(1/5)
Discrete Diffusion Reading Group retweeted
It's currently trending as the #1 paper of the day on HF!
Thanks for sharing our work, @rohanpaul_ai
Folks, pls upvote if you liked our work 🤗
huggingface.co/papers/2609.0…
cc:@_akhaliq
Uno shows a simple way to speed up existing LLMs without changing their output distribution: keep the original model in charge, and use diffusion only to draft multiple tokens in parallel.
Keep the autoregressive model for quality and add diffusion for parallel drafting
On Qwen3-8B, Uno delivered 2.5X higher per-request throughput and 1.6X higher system throughput at the largest tested batch size.
Normal autoregressive LLMs generate 1 token at a time, making long responses, agents, and RL rollouts slow.
Uno adds lightweight diffusion adapters that propose several future tokens at once, while the original model verifies them.
That avoids a separate draft model and preserves the base model’s sampling behavior.
It also sped up end-to-end RL training by up to 40% in the reported runs.
Discrete Diffusion Reading Group retweeted
It's currently trending as the #1 paper of the day on HF!
Folks, pls upvote if you liked our work 🤗
huggingface.co/papers/2609.0…
cc:@_akhaliq
