
Founding research scientist @mostik_ai Ph.D. student at @constructor_uni
Bremen, Germany
Joined July 2023
- Tweets57
- Following273
- Followers224
- Likes88
Pinned Tweet
Continuous diffusion for text is taking off.
But most methods still diffuse in spaces that were never learned for denoising and this limits their performance.
In LDLM, we train the encoder, diffusion model, and decoder jointly, so the denoising objective shapes the representation directly.
❌ Not: learn a space → run diffusion on it.
✅ Instead: learn a diffusable space.
On OpenWebText & LM1B, LDLM achieves a better quality–speed trade-off than MDLM, Duo, CANDI, and FLM.
Up to 13× faster sampling.
🧵 Recipe and insights below ↓
Four years ago, when we started working on latent diffusion for text, there were only a handful of papers. This year, the paper count has nearly tripled.
It's been great to see the field grow and to have contributed to 7 of these projects.
New survey on Continuous Diffusion and Flow Language Models is out! 🌊💬
One of my first papers, back in early 2025, was on continuous simplex flows. After working on discrete diffusion, I wanted to try controllable sequence generation with flow matching, which naturally operates in continuous spaces. But at the time, I found them much harder to train well than discrete diffusion. 🥲
2026 has changed this picture completely! 🚀 It’s been super exciting to watch continuous LMs take off, but with the rapid advancements, it’s also becoming harder to keep up with all the new work across pre-training, few-step generation, guidance, and scaling 🥹
So together with @chandavidlee, @FEijkelboom, and @jw9730, we put together a survey that consolidates the continuous LM literature and its key insights into 37 pages. It covers everything from the early work before the so-called “winter” of continuous LMs to the wave of new papers in 2026 🌟
More in thread! 🧵
Our papers:
TEncDM (AAAI 2025, oral): arxiv.org/abs/2402.19097
Smoothie (ICML 2026): arxiv.org/abs/2505.18853
Cosmos (NeurIPS 2025): arxiv.org/abs/2506.21170
Why Gaussian Fails (COLM 2026): arxiv.org/abs/2604.02028
LDLM (NeurIPS 2026): arxiv.org/abs/2605.07933
Rep-MMD (BeNTo Workshop, NeurIPS 2026): arxiv.org/abs/2610.06648
CoFlow (BeNTo Workshop, NeurIPS 2026): openreview.net/forum?id=vqMo…
1
5
128
Latent diffusion for text usually freezes the encoder before diffusion training, so the diffusion model gets no say in its own latent space. We let its gradient in.
LDLM, NeurIPS 2026. Code and checkpoints below.
At NeurIPS in Paris? Let's grab coffee and talk about latent diffusion and latent communication between models.
Code + checkpoints: github.com/MeshchaninovViach…
Paper: arxiv.org/abs/2605.07933
Earlier post: nitter.cf/ViacheslavMesh/status/…
Continuous diffusion for text is taking off.
But most methods still diffuse in spaces that were never learned for denoising and this limits their performance.
In LDLM, we train the encoder, diffusion model, and decoder jointly, so the denoising objective shapes the representation directly.
❌ Not: learn a space → run diffusion on it.
✅ Instead: learn a diffusable space.
On OpenWebText & LM1B, LDLM achieves a better quality–speed trade-off than MDLM, Duo, CANDI, and FLM.
Up to 13× faster sampling.
🧵 Recipe and insights below ↓
Viacheslav Meshchaninov retweeted
Can flow language models scale to billions of parameters and handle real generative tasks?
Introducing 🌊 Lafa (Large flow language model), a flow LM scaled up to 8B that matches AR quality, with 🚀 2.5× AR throughput at 7B.
Co-led with @ArnavChavan6 @UdbhavBamba.
Joint work with @wognsfjq96 @hongseu33 @AdtRaghunathan @jw9730
Viacheslav Meshchaninov retweeted
Announcing FlashMap: a new speculative decoding algorithm based on flow maps! >6x faster inference on Qwen3-8B 🧵
Project page: jshen.net/flashmap
arXiv: arxiv.org/abs/2610.24365
w/ @Jay41969851 @_ryantlee @jonathanmay @cosmo_shirley
Viacheslav Meshchaninov retweeted
PS: Please do check out the amazing analyses of CDLMs that are already out there:
- Why Gaussian Diffusion Fails on Discrete Data (@a__shabalin @Viacheslav91112): arxiv.org/abs/2604.02028
- CANDI (@PatrickPyn35903 @thjashin @ruqi_zhang): arxiv.org/abs/2510.22510
- FMLM* (@wognsfjq96 @wonjung_ki13109 @chandavidlee @FEijkelboom @jw9730): arxiv.org/abs/2607.00714
Viacheslav Meshchaninov retweeted
Introducing a simple method to post-train both discrete and continuous diffusion LMs.
It's based on Maximum Mean Discrepancy (MMD) that matches token-level feature distributions. Our method improves generation and speed on OWT/GSM8K and scales to 16B models. 🧵
Really excited to finally share what we’ve been building at @mostik_ai.
I’ve spent years working on latent diffusion, so I’m especially happy to now be part of pushing latent communication forward and seeing these ideas turn into something real.
meet @mostik_ai!
what happens when you put 12 PhDs in one room for four months? first place on the ARC-AGI leaderboard, which I can't say much about while the competition is still running. and this, which I can.
everyone's arguing about whether open models will catch up to frontier models. we think it's the wrong question. here's the one we pose: why does a frontier model have to generate your answer at all, when the only thing you need from it is the reasoning?
we do this by enabling models to communicate in latent space. through our protocol, hidden states pass straight from a frontier model into a small one running on your infrastructure -- no text between them, and neither model is fine-tuned. two models from different families, sharing reasoning, both left untouched.
how do we know it works? we tested it on a setup where a 753B model reads the problem, and a 4B edge-class model writes the answer. with this approach, we get results 80% as accurate as the frontier model, but at 20x faster performance.
we're committed to preventing frontier model lock-in and are already partnering with inference providers to accelerate open-weight adoption. we've done this between 15 of us, in four months, 12 PhDs and a Fields medalist, backed by @generalcatalyst
WIRED has the first external account of the company and the work: wired.com/story/russian-star…
full writeup, the setup, and all the numbers: mostik.ai/read-more
Viacheslav Meshchaninov retweeted
AI has had exactly two scaling axes that worked so far, and the second one is starting to look finite too
the first one was pretraining: with scaling parameters and data, we got world knowledge (i.e. ChatGPT had read enough to know things), but it started saturating a while ago
the second one was RL, and people had been doing RL the whole time before that: RLHF is RL but it never scaled far because it was trying to control the exact output, which tokens come out, how the text reads, but you can only push that so far before you’re just polishing
RLVR dropped that constraint: giving the model a task, then checking whether the final answer is right, and ignoring everything in between -- so the model does whatever it wants in the middle and only the endpoint gets graded, and that’s much closer to actual RL and it’s what bought us planning and reasoning
(arguably, tool use sits around 2.5 on this list -- while useful, it's not a different kind of thing)
so one axis gave knowledge, the other gave reasoning, and both of them are one model working alone
the next axis is how many models you can get working on the same problem, which is a different kind of axis than the previous two
we know that multi-agent RL has always been the harder problem: I spent years in that literature and the gap between single-agent and multi-agent is definitely not incremental -- it’s a whole different class of difficulty! which is also why the derivatives are steep at the start, nobody has picked the easy wins yet...
and the thing that gates this multi-agent coordination is communication: models can only coordinate as well as they can exchange information, and right now they do that by writing sentences to each other
imagine what could we possibly achieve if we properly open that third axis development by letting models to exchange information in their native "language" without loosing any computational data that they produce during inference
on Thursday, @taalas_inc agreed to join AMD
they etch a model's weights directly into the transistors, so there's no external memory to read during a forward pass and the memory wall that caps every GPU inference system doesn't apply: their demo chip runs Llama 3.1 8B at ~17k tokens/sec, where an H100 does about 150
the objection people raised back in February was that a chip like this only runs the model it was built for
but the weights go into mask ROM permanently while the KV cache and LoRA adapters live in SRAM where they can still change -- so there's already a frozen half and a moving half
right now the frozen half is an entire model, and when the model changes they respin the metal layers, roughly two months of turnaround, and they've priced three of those revisions into a four-year lifespan
nobody's tried making the frozen half smaller than that: you'd need some way to connect the parts that stay put with the ones that change, which is its own hard problem
but if that existed you'd only re-etch whatever actually moved, and a chip could last past one model generation
and all of this is seven months after Nvidia paid $20B for Groq
Viacheslav Meshchaninov retweeted
12 PhDs and a Fields Medalist in the mountains arguing about latent communication, and it happened to be my birthday 🎂
taking suggestions on how to top that 😂
Viacheslav Meshchaninov retweeted
DiffusionGemma brings high intelligence and lightning fast ⚡️ inference to local developers (>1100 tok/s on a single H100)!
I'm excited to see what people will do with this model - and what improvements people can build on top (better samplers maybe??).
So unbelievably proud of the hard work the team put in to get this out!🪐🪐🪐
Viacheslav Meshchaninov retweeted
Nice work! CANDI (Oct ‘25) gives a complementary explanation for why Gaussian diffusion struggles on text — via token identifiability and temporal dissonance — plus a retroactive account of why self-conditioning and related heuristics help (App A.3). Our analysis is mainly one-hot, with theory for embedding tables in App A.4; extending it to encoder representations as you do is a nice step we didn’t cover.
Viacheslav Meshchaninov retweeted
Why Gaussian diffusion models fail on text data and how to prevent it?
☝️ We find that discrete-like latent spaces are fundamentally bad for continuous diffusions.
☝️ We explain what happens inside, and why self-conditioning and other heuristics improve generation.
🧵👇 1/7
Viacheslav Meshchaninov retweeted
🥝🍑 SMOOTHIE was accepted to ICML 2026!
A new text diffusion method that generalises simplex diffusion by gradually smoothing the distances to all vocab embeddings.
This makes denoising more natural for text data, resulting in improved generation.
📄 arxiv.org/abs/2505.18853
Viacheslav Meshchaninov retweeted
When it comes to one-step discrete generation, almost everyone has been relying on flow maps. We took a different route: a normalizing flow. 🌊
By learning the coupling between discrete sequences and noise directly, we avoid a high-dimensional flow. Check it out! 👇
New paper 🎉: Coupling Models for One-Step Discrete Generation
What is the right route to one-step discrete generation?
Recently, the default answer has been: make discrete data continuous, learn a flow map, and then decode back to discrete samples.
This is a natural route, but it is not the only one.
We propose a different perspective: instead of forcing discrete generation through a continuous flow map, we learn a coupling between discrete sequences and Gaussian noise. Then we train a one-step generator that directly predicts the sequence from the noise using cross-entropy.
At inference time, generation is simple:
sample Gaussian noise → decode once → obtain a discrete sample.
No autoregressive loop.
No iterative denoising trajectory.
No learning continuous flow model.
The key idea is that, for one-step discrete generation, the data–noise coupling should not be treated as a fixed background choice. It is the central object to learn.
Empirically, Coupling Models improve strong one-step baselines across text generation, biological sequence generation, and binary image generation.
Paper: arxiv.org/pdf/2605.07193
Code: github.com/pengzhangzhi/Coup…
Work done with @bose_joey @Anru_Zhang @AlexanderTong7
Viacheslav Meshchaninov retweeted
🔥 New paper: Language Modeling with Hyperspherical Flows
Recent flow language models (FLMs) all use Gaussian noise. Makes sense for images, but not necessarily for text 🫠 We propose to add noise by rotating embeddings on 𝕊^{d−1} instead 🌐
w/ @caglarml
(1/9)
This direction has been moving really fast recently, with many great works on continuous diffusion for text.
Our main takeaway: the choice of continuous space matters a lot, and adapting it jointly with the denoising objective seems crucial.
Would be very curious to hear thoughts from @zhuci19, @chenxiao_yang_, @PatrickPyn35903, @thjashin, @UnderGroundJeg, @YizheZhangNLP, @dario_sha, @linluqiu, @HuLillian39250, @jacobandreas, @AndrewC_ML, @zhihanyang_, @dvruette, @yjelid, @ValentinDeBort1, @ArnaudDoucet1, @chandavidlee, @sedielem, @Sam_Acqua, @LucaAmb, @gabriberton, @mmbronstein, @karsten_kreis, @MardaniMorteza, @msalbergo


























