Jonathan Geuter retweeted
⚡NEW: Unlocking lossless speedups in LLMs via discrete diffusion. A new diffusion-augmented model, Uno, combines autoregressive and diffusion weights in one network to generate multiple tokens per forward pass. More in today’s Deeper Learning blog post: bit.ly/4AmooR0
Jonathan Geuter retweeted
Great blog post by @JonathanGeuter and @KempnerInst on our recent Uno paper.
Autoregressive LLMs + Diffusion adapters >>> Training Diffusion LLMs from scratch or Speculative Decoding with small drafters
Stay tuned for a follow-up to the Uno paper with comparisons against DFlash2 and DSpark.
kempnerinstitute.harvard.edu…
Diffusion LLMs have two limitations relative to AR models:
(1) Lower quality, and
(2) Slower inference at large batch sizes.
We address this "Uno"
> Retains the AR architecture of LLMs
> Each layer has two sets of weights: AR weights and Diffusion weights
> Diffusion weights enable parallel sampling from the AR distribution losslessly
Results:
💥Faster than all speculative decoding methods: DFlash and EAGLE-3
🔥 Beats ALL diffusion LLMs: Mercury 2, Diffusion Gemma, Llada
Links to the paper, models, code below
Jonathan Geuter retweeted
Today, we introduce Priming Models, a new paradigm for training one-step generative models from scratch.
No diffusion teacher. No flow map. No CFG. No JVP.
We believe this work will challenge how we think about building one-step models.
Joint work with @sitanch, @du_yilun, @parkwbin03, Bohyung Han.
🔥 1.19 FID one-step on ImageNet 256×256 while using substantially less compute, e.g., 12X less than Drifting-B, 6X less than Drifting-L
🎨 Scales to text-to-image generation (GenEval 0.82)
🤖 World modeling on RT-1 / Bridge-V2
Priming Model is built on one core message: [Reconstruction pretraining --> Distributional post-training].
Jonathan Geuter retweeted
It's currently trending as the #1 paper of the day on HF!
Thanks for sharing our work, @rohanpaul_ai
Folks, pls upvote if you liked our work 🤗
huggingface.co/papers/2609.0…
cc:@_akhaliq
Uno shows a simple way to speed up existing LLMs without changing their output distribution: keep the original model in charge, and use diffusion only to draft multiple tokens in parallel.
Keep the autoregressive model for quality and add diffusion for parallel drafting
On Qwen3-8B, Uno delivered 2.5X higher per-request throughput and 1.6X higher system throughput at the largest tested batch size.
Normal autoregressive LLMs generate 1 token at a time, making long responses, agents, and RL rollouts slow.
Uno adds lightweight diffusion adapters that propose several future tokens at once, while the original model verifies them.
That avoids a separate draft model and preserves the base model’s sampling behavior.
It also sped up end-to-end RL training by up to 40% in the reported runs.
Jonathan Geuter retweeted
As per Artificial Analysis' updated Evals, our K2-Horizon-400B-Moe model still beats Thinking Machine's Inkling model.
Pretty respectable for a lab that was set up a little over a year ago, with a decent chunk of the K2-series effort being led by interns!!!
Couldn't be prouder to be part of the @IFM_AI family ♥️
Jonathan Geuter retweeted
Introducing K2 Horizon: a connected fleet of six foundation models ranging from 0.9 billion to 375 billion parameters.
- Frontier performance: Across coding and agentic tasks, K2 Horizon delivers top-tier performance in every size class—with the 0.9B, 3.7B and 7B models setting new state of the art at their respective scales.
- Radical openness: K2 Horizon represents the largest fully open-source model launch in AI history. The fully open code, training data and recipes are a significant step forward in transparency.
Launch page: ifm.ai/k2/
Tech blog: ifm.ai/blog/k2
Hugging Face: huggingface.co/collections/I…
Jonathan Geuter retweeted
Paper + models drop later tonight. Stay tuned.
Sneak peek at the Diffusion LLM that will be released soon
I’m at #ICML2026 in Korea presenting two papers (details in the comments) and joining a panel discussion at the SPIGM workshop.
Drop by my posters or DM me if you’d like to chat more.
Jonathan Geuter retweeted
Next week, we will show that discrete diffusion replaces speculative decoding approaches like Dflash and Eagle-3.
If you are an early career researcher interested in discrete diffusion, focus on continuous methods instead.
New blog post: continuous diffusion for language is back!
This research direction receded into the background for a while, but as of this year, it is once again a hot topic. I wrote down a historical perspective and some thoughts on the recent revival.
sander.ai/2026/08/24/continu…
Jonathan Geuter retweeted
Day 2 of #ICML2026 is packed with #KempnerInstitute presentations on diffusion models, LLM reasoning, world models for robotics, and much more. Don't miss Wednesday's sessions! #AI
Jonathan Geuter retweeted
Our Data-Centric ML group is at ICLR 🇧🇷this week. I couldn't make it this year 😰, but @SaraKangaslahti, @JonathanGeuter, @rach_it_ are there. Find them, say hi. Quick rundown 👇
Jonathan Geuter retweeted
🚨🚨🚨 Now you can stop training your masked diffusion models ''for the worst''.
We propose 🐆PUMA🐆--Progressive UnMAsking, a simple modification of the forward masking process that speeds up the masked diffusion training.
Jonathan Geuter retweeted
📄 New preprint alert:
We study 🪃Boomerang Distillation🪃, a surprising phenomenon that allows generating a family of pre-trained LLMs of intermediate sizes from a single teacher–student pair — 𝐧𝐨 𝐞𝐱𝐭𝐫𝐚 𝐭𝐫𝐚𝐢𝐧𝐢𝐧𝐠 𝐫𝐞𝐪𝐮𝐢𝐫𝐞𝐝!
🧵👇
Jonathan Geuter retweeted
'Universal Neural Optimal Transport'
@JonathanGeuter · Gregor Kornhardt · Ingimar Tomasson · Vaios Laschos
doi.org/10.48550/arXiv.2212.…
(5/20)
Jonathan Geuter retweeted
If you're at #AISTATS2025, check out the presentation by Jonathan Geuter, in collaboration with @Clement_Bonet_ , @Korba_Anna and @elmelis.
'DDEQs: Distributional Deep Equilibrium Models through Wasserstein Gradient Flows'
openreview.net/forum?id=rFfN…
#AI #statistics #ML


















