building fun stuff @modal
San Francisco, CA
Joined January 2018
- Tweets89
- Following722
- Followers1.3K
- Likes523
Nan Jiang retweeted
Yes, comparing with their published numbers is actually unfair to DeepSeek, so we just did an apples-to-apples comparison on the same machines, which is more persuasive, professional, and rigorous.
Nan Jiang retweeted
I had tons of fun working on pretraining (& automating the pretraining) of Flint! It's a really good local model and a small preview of what we are cooking!
Introducing Flint, @Recursive_SI’s first model: a fast local coding model built with our automated research system, the Eureka Machine.
• Best-in-class fast local coding
• Up to 2x faster than comparable models: 125+ tokens/sec on a MacBook Pro, 150+ on RTX Spark
• Built from scratch in under 4 months
Private, unmetered intelligence on your own machine. Open weights available now on @huggingface, run it locally on your favorite framework or try it for free on @opencode or @pidotdev.
Nan Jiang retweeted
Everybody buying compute for RL needs to stop immediately and think about this result.
Big homogenous clusters are a sledgehammer for what is essentially an inference-shaped nail.
Fantastic work from @nanjiangwill.
Training large models with RL typically runs on a fixed homogeneous cluster of the latest accelerators for numerical stability and faster throughput. This throws away inference optimization work done for other vendors and generations, limits the capacity you can utilize, and is unnecessarily expensive.
So we trained a model on agentic coding tasks using fully asynchronous RL and mixed-precision, mixed-accelerator samplers. With off-policy correction, training stayed stable through convergence, on par with homogeneous samplers, at a lower cost.
Nan Jiang retweeted
RL is so much more elastic if the FLOPs for rollouts can come from any hardware.
If only there were a product that's good for spinning up lots of different kinds of hardware elastically...
Training large models with RL typically runs on a fixed homogeneous cluster of the latest accelerators for numerical stability and faster throughput. This throws away inference optimization work done for other vendors and generations, limits the capacity you can utilize, and is unnecessarily expensive.
So we trained a model on agentic coding tasks using fully asynchronous RL and mixed-precision, mixed-accelerator samplers. With off-policy correction, training stayed stable through convergence, on par with homogeneous samplers, at a lower cost.
Nan Jiang retweeted
A cool application of score centering: running RL on heterogeneous hardware
It uses the Miles implementation of score centering, which supports top-p sampling.
Applying score centering to top-p sampling is a generalization of the same "mask replay" correction that Cognition and DeepSeek use for top-p training (under zero TIM, these corrections are equivalent). Another benefit is that under top-p sampling, it is feasible to run exact score centering (which is infeasible with full sampling).
Training large models with RL typically runs on a fixed homogeneous cluster of the latest accelerators for numerical stability and faster throughput. This throws away inference optimization work done for other vendors and generations, limits the capacity you can utilize, and is unnecessarily expensive.
So we trained a model on agentic coding tasks using fully asynchronous RL and mixed-precision, mixed-accelerator samplers. With off-policy correction, training stayed stable through convergence, on par with homogeneous samplers, at a lower cost.
Spent the past three weeks on this idea.
A sampler on a different precision or accelerator is just off-policy, like stale samples in fully async RL, and off-policy correction doesn't care why the sampler differs from the trainer. Meanwhile, inference keeps diversifying across precisions(MXFP8, MXFP4, NVFP4) and accelerators outside RL. So why not use all of that compute for RL rollout?
The sampler and the trainer are two systems built for different purposes. Hope this opens up new possibilities for RL algorithms and systems, and leads to cleaner system designs. Enjoy the blog, and stay tuned for the next one on our infra :)
Training large models with RL typically runs on a fixed homogeneous cluster of the latest accelerators for numerical stability and faster throughput. This throws away inference optimization work done for other vendors and generations, limits the capacity you can utilize, and is unnecessarily expensive.
So we trained a model on agentic coding tasks using fully asynchronous RL and mixed-precision, mixed-accelerator samplers. With off-policy correction, training stayed stable through convergence, on par with homogeneous samplers, at a lower cost.
thanks to @zz30gs @yueming02 @JiajunLi111649 @whatthelukh @mrtnm @dhruvgautam @xiuyu_l @hsu_byron @zijie_y @zianglih @zhuzilinallen
for all the discussions that helped shape this!
having fun with the goat
Happy to contribute a fast and correct RL sampling mask from @periodiclabs, developed jointly with @nanjiangwill @agarwl_ and now running in our production RL pipeline!
Nan Jiang retweeted
Lean Verified Transformers (srush.github.io/lean-transfo…)
In which we prove a bunch of Transformer invariants from scratch in Lean, and speculate about how hard it would be to do that for the rest of the world's code.
Nan Jiang retweeted
We built high-throughput materials labs in Menlo Park to create a loop between experiments and models. The labs generate fresh data, the models learn from it, and then help us decide what to try next.
Using only 1,300 H200s, plus months of our experimental data, we mid-trained and RL’d an open-source model to surpass GPT-6 Astra on our analysis benchmark. We call it Neon.
This is real footage from our lab. We’re focusing first on hard problems in materials science, including superconductors, magnets, and semiconductor materials.
Read our blog posts below.
Nan Jiang retweeted
The full technical report for Miles is now available.
Miles is built for production-level post-training, with stability, efficiency, and flexibility at its core. The report covers its system design and how it works in practice.
Today we're launching Miles v0.1, an open-source RL framework for LLMs and multimodal models.
RL training is easy to start and hard to debug. Miles helps you ensure your run is correct, use hardware efficiently, and keep RL running at scale.
Over the past 9 months, 72 contributors have landed 1,326 commits, 85 GPU E2E CI tests, battle-testing Miles on frontier open models like Kimi K3, DeepSeek V4, Qwen 3.8, GLM 5.2, Inkling, MiniMax H3, etc.
Miles powers frontier-model development and production RL workloads at @humansand, @periodiclabs, @modal, @DecagonAI, @Eigent_AI, @nebiusai, @IBM and more, on both @NVIDIAAI and @AIatAMD hardware.
Here is what we built, and why teams picked Miles🧵
Nan Jiang retweeted
We share our early research on building Software World - a "GitHub" run by agents. We deploy agents for Python packages in a dependency chain, and each agent is tasked with collaborating with others and optimizing the package it owns.
huge
Today we're launching Miles v0.1, an open-source RL framework for LLMs and multimodal models.
RL training is easy to start and hard to debug. Miles helps you ensure your run is correct, use hardware efficiently, and keep RL running at scale.
Over the past 9 months, 72 contributors have landed 1,326 commits, 85 GPU E2E CI tests, battle-testing Miles on frontier open models like Kimi K3, DeepSeek V4, Qwen 3.8, GLM 5.2, Inkling, MiniMax H3, etc.
Miles powers frontier-model development and production RL workloads at @humansand, @periodiclabs, @modal, @DecagonAI, @Eigent_AI, @nebiusai, @IBM and more, on both @NVIDIAAI and @AIatAMD hardware.
Here is what we built, and why teams picked Miles🧵
Nan Jiang retweeted
🧵I post-trained Qwen3-4B-Instruct to play MegaGem, @JaneStreetGroup's new auction game.
It ranks #1 on 3-player MegaGem, above models like GPT-5.5, Claude Opus 4.8, and Gemini 3 Flash, but statistically unresolved with Gemini 3.1 Pro.
However, self-play RL wasn’t what got it there.
goated as always
Ever since RL became a core part of LLM training, train–rollout numerical mismatch has been one of the recurring problems in RL infra.
There have been several great open-source efforts toward alignment, but production-scale support still often comes with tradeoffs in model scale, performance, or feature coverage.
Today, we're open-sourcing the deterministic train–rollout alignment stack we use for GLM-5.2-scale training — covering FP8 weights + FP8 KV rollout, DeepEP, DeepGEMM, and sparse attention.
github.com/THUDM/slime/pull/…
1/3
Nan Jiang retweeted
While testing Kimi-K3 on CEO-Bench, we found that inference provider impacts results significantly.
While @modal's endpoint led to #1 on leaderboard, another inference provider produced loopy model reasonings and early bankruptcy on all runs. Always evaluate endpoint you use!
There's a new #1 on CEO-Bench! @Kimi_Moonshot Kimi-K3 beats Fable 5 & GPT-5.6 Sol
K3 maintains better late-game customer retentions by testing price elasticity. It changed price $139➡️$129➡️$119; saw retention 56%➡️95.5%➡️96%; and decided on $129 to balance retention and revenue
Nan Jiang retweeted
I was always slow on Forward v.s. Reverse KL. Then I realized a mnemonic trick.
FOrward -> FOMO
Fear of missing out on some probability mass:
min_q KL(p || q) encourages mode coverage of p.
Reverse -> Retreat
Retreat to a high-density mode:
min_q KL(q || p) is mode seeking.
