Not easy, definitely tough, but incredibly fruitful. Tackling large-scale infrastructure bottlenecks, especially eliminating infrastructure bubbles to improve experiment throughput, was a major challenge. Excited for whatโs next with our new Wrench framework! ๐ง
We're open-sourcing RL360, the RL infrastructure behind K2 Horizon's six domain experts: 8,192 concurrent sandboxes, 150 steps in 7 days, ~1.3M rollouts.
Long-horizon agentic RL runs thousands of agents in real sandboxes on tasks that take minutes. Some of those sandboxes crash mid-task, and a crash arrives as the same low score as a wrong answer.
Without infrastructure that can keep up, GPUs are underutilized and the model gets penalized for system crashes rather than its own mistakes.
RL360 is built for this. We share how it works, what broke at scale, and what we learned in the thread.
Code: github.com/ifm-ai/RL360
Run: wandb.ai/mbzuai-llm/rl360-puโฆ
2
53
Ming Shan retweeted
๐๏ธ Mon Oct 5, 8โ10pm PT
AMA with the team behind K2 Horizon, hosted by r/LocalLLaMA (@LocalLLaMAsub).
Ask us anything about models, deployment, training, what's open so far and what's next.
๐ฌ Questions are open now. We start answering when we go live.
Drop your questions here ๐ย redd.it/1wv8zww
Journal and conference submissions are getting out of hand. Definitely need stricter screening and more rigorous peer review.
TMLR has faced a deluge of submissions, necessitating stricter desk rejection policies due to limited reviewer capacity
Co-EiC Nihar Shah reached out to authors of 10 papers slated for desk reject. Could they answer questions about their *own* submission?
medium.com/@TmlrOrg/asking-aโฆ
59
Ming Shan retweeted
Introducing K2 Horizon: a connected fleet of six foundation models ranging from 0.9 billion to 375 billion parameters.
- Frontier performance: Across coding and agentic tasks, K2 Horizon delivers top-tier performance in every size classโwith the 0.9B, 3.7B and 7B models setting new state of the art at their respective scales.
- Radical openness: K2 Horizon represents the largest fully open-source model launch in AI history. The fully open code, training data and recipes are a significant step forward in transparency.
Launch page: ifm.ai/k2/
Tech blog: ifm.ai/blog/k2
Hugging Face: huggingface.co/collections/Iโฆ
Ming Shan retweeted
๐ ๐ถ๐
-๐ฅ๐ (pooling prompts from different domain into a single joint RL run), is the most common technique for multi-task RL post-training of LLMs. Train on math, code, and instruction-following prompts together in one GRPO run, and in principle you get one model that's good at all three.
But in practice it runs into a well known problem: the ๐๐ฒ๐ฒ-๐๐ฎ๐ ๐ฒ๐ณ๐ณ๐ฒ๐ฐ๐. Push one domain up and another quietly drops when trained jointly, even though each domain improves fine on its own. The recent MOPD paper states that cross-domain training signals interfere with each other, and the joint model ends up below every one of the per-domain specialists it was supposed to combine. (arxiv.org/abs/2606.30406).
There's concrete evidence of this too. The Modular Gradient Surgery paper shows that even a heavily skewed ๐ต๐ฌ% ๐บ๐ฎ๐๐ต / ๐ญ๐ฌ% ๐ฐ๐ต๐ฎ๐ mixture still lags behind a math-only model trained the same number of epochs. Skewing the data mix in favor of a domain doesn't rescue it (arxiv.org/abs/2602.02301).
I read through a bunch of recent papers that investigate into this issue and try to point towards the root causes. Below I summarize the main findings:
๐ญ. ๐๐ฟ๐ฎ๐ฑ๐ถ๐ฒ๐ป๐ ๐บ๐ฎ๐ด๐ป๐ถ๐๐๐ฑ๐ฒ ๐ถ๐บ๐ฏ๐ฎ๐น๐ฎ๐ป๐ฐ๐ฒ, ๐ฑ๐ฒ๐ฐ๐ผ๐๐ฝ๐น๐ฒ๐ฑ ๐ณ๐ฟ๐ผ๐บ ๐ฎ๐ฐ๐๐๐ฎ๐น ๐น๐ฒ๐ฎ๐ฟ๐ป๐ถ๐ป๐ด
The Imbalanced Gradients paper (Meta) shows certain tasks (e.g. code) produce squared gradients up to ๐ญ๐ฑ๐
larger than others (e.g. math) in joint training, but large-gradient tasks often show similar or lower learning gains than small-gradient ones. The shared update gets biased toward whichever task shouts loudest, not whichever task needs it. (Wu et al., Imbalanced Gradients in RL Post-Training of Multi-Task LLMs, arxiv.org/abs/2510.19178)
๐ฎ. ๐๐ฅ๐ฃ๐ข'๐ ๐ด๐ฟ๐ผ๐๐ฝ ๐ป๐ผ๐ฟ๐บ๐ฎ๐น๐ถ๐๐ฎ๐๐ถ๐ผ๐ป ๐๐๐ฎ๐ฟ๐๐ฒ๐ ๐ต๐ฎ๐ฟ๐ฑ๐ฒ๐ฟ ๐ฑ๐ผ๐บ๐ฎ๐ถ๐ป๐
The Multi-Task GRPO paper and the DISCO paper both show that prompts where every rollout in a group gets an identical reward produce zero advantage and zero gradient, so domains with more "live" (non-saturated) prompts dominate the pooled update by construction. Naively upweighting the weaker domain doesn't fix this without explicit ratio-preserving sampling. (Multi-Task GRPO, arxiv.org/abs/2602.05547; DISCO, arxiv.org/abs/2505.15074)
๐ฏ. ๐๐ฟ๐ฎ๐ฑ๐ถ๐ฒ๐ป๐ ๐ฑ๐ถ๐ฟ๐ฒ๐ฐ๐๐ถ๐ผ๐ป ๐ฐ๐ผ๐ป๐ณ๐น๐ถ๐ฐ๐, ๐ป๐ผ๐ ๐ท๐๐๐ ๐บ๐ฎ๐ด๐ป๐ถ๐๐๐ฑ๐ฒ
The Modular Gradient Surgery paper computes math and chat gradients separately on the same shared weights and finds negative cosine similarity between them: on many weights, the two updates point in opposite directions. Averaging them into one Mix-RL step causes partial cancellation instead of accumulation, so the net update is a diluted compromise that serves neither domain well. This is the direct mechanism behind the underperformance above. (Cai et al., Modular Gradient Surgery, arxiv.org/abs/2602.02301)
๐ฐ. ๐ฅ๐ฒ๐๐ฎ๐ฟ๐ฑ-๐๐ฐ๐ฎ๐น๐ฒ ๐ถ๐ป๐ฐ๐ผ๐ป๐๐ถ๐๐๐ฒ๐ป๐ฐ๐ ๐น๐ฒ๐๐ ๐ผ๐ป๐ฒ ๐๐ถ๐ด๐ป๐ฎ๐น ๐ผ๐๐ฒ๐ฟ๐ฟ๐ถ๐ฑ๐ฒ ๐ฎ๐ป๐ผ๐๐ต๐ฒ๐ฟ
The "Two is better than one" paper shows that when combined reward components differ in numerical scale, the higher-variance component dominates the per-token gradient by roughly an order of magnitude while the other gets effectively zeroed out. It demonstrates this within a single domain's combined reward channels, but it's the same scale-domination mechanism that shows up across domains with differently calibrated rewards. (arxiv.org/abs/2605.22620)
๐ฑ. ๐ฆ๐ต๐ฎ๐ฟ๐ฒ๐ฑ ๐ฝ๐ฎ๐ฟ๐ฎ๐บ๐ฒ๐๐ฒ๐ฟ๐ ๐บ๐ฎ๐ธ๐ฒ ๐๐ฟ๐๐ฒ ๐ถ๐ป๐ฑ๐ฒ๐ฝ๐ฒ๐ป๐ฑ๐ฒ๐ป๐ฐ๐ฒ ๐ถ๐บ๐ฝ๐ผ๐๐๐ถ๐ฏ๐น๐ฒ
The Local Perturbation Theory paper (Yang, Ding & Xiong) shows that full-model gradients across domains can look nearly orthogonal while a later domain still damages an earlier one, through a second-order effect concentrated in a low-dimensional shared parameter subspace. This is shown for sequential/cascade multi-domain RL rather than simultaneous joint training, but it points at the same underlying constraint: one set of weights, multiple domains pulling on it. (arxiv.org/abs/2606.02398)
Today, MBZUAI released K2-V2, a fully transparent and open-source 70B model built entirely from scratch. I am happy to be part of this incredible team that is committed to developing transparent and reproducible models.
Today, we are releasing a new version of K2 (K2-V2), a 360-open LLM built from scratch as a superior base for reasoning adaptation, while still excelling at core LLM capabilities like conversation, knowledge retrieval, and long-context understanding.
K2 fills a major gap: highly capable models with no transparency. Instead of releasing only weights, weโre sharing the full training story โ dataset recipes, mid-training checkpoints, logs, code, and evaluation tools. Thatโs 360-open.
Whatโs inside:
โข 70B dense transformer engineered as a reasoning-enhanced base model
โข Native 512K context (extendable via RoPE scaling)
โข Mid-training reasoning phase
โข Strong tool-use scaffolding
What weโre open-sourcing:
โข 250M+ reasoning traces (math, planning, multi-step logic)
โข Full pre- & mid-training data compositions
โข All mid-training checkpoints
โข Training logs, code, Eval360
Performance:
โข GPQA-Diamond: 55.1% mid-training โ 69.3% after SFT (strongest fully open 70B model)
โข KK-8 Logic Puzzles: 83% โ competitive with DeepSeek-R1 & OpenAI o3-mini-high
โข ArenaHard V2: 62.1% โ close to Qwen3 235B
โข Outperforms Qwen2.5-72B and approaches Qwen3-235B despite being smaller and fully transparent.
๐ The Model:
bit.ly/3KIYwuo
๐Technical Report:
bit.ly/49V8h2U
๐Blog:
bit.ly/49V7gb6
3
269
Now there is roughly 90GB worth of photos and footage to edit (total about 2 hours)... What have I gotten myself into...
Find yourself a @mingshanhee who captures the world so you can truly live in the moment!
1
5
364
So ICWSM Main Conference Day 1 has kicked off! Catch me at "Session 1: Multimodal Meme Analysis" from 10:30โฏAM to 11:00โฏAM, where Iโll be presenting two papers.
2) Contrastive Instruction Fine-Tuning Large Multimodal Model for Hateful Meme Classification (ojs.aaai.org/index.php/ICWSMโฆโฆ)
- Post-train LMMs for hateful meme classification tasks
- Applies Contrastive Learning for better classification boundaries
1) Demystifying Hateful Content: Leveraging Large Multimodal Models for Hateful Meme Detection with Explainable Decisions (ojs.aaai.org/index.php/ICWSMโฆ)
- Reduce reliance on model's implicit knowledge
- Provide possible explanations behind models' decisions
Flying to attend @icwsm 2025 in Copenhagen, Denmark now! Let's talk about memes, social problems, and MAYBE have a few Danish pastries ๐ฉ๐ฐโจ #ICWSM2025
SUTD AI Day concludes with @hwaran_lee presenting on "Evaluation with socio-cultural awareness and multilingual LLMs" ๐!
Next at SUTD AI Day, we have @pliang279 delivering keynote on "New Advances in Multimodal Reasoning"!
SUTD AI Day, held at @sutdsg, has officially begun!
Starting with @preslav_nakov presenting on "Towards Truly Open, Language-Specific, Safe, Factual, and Specialized Large Language Models."
Tomorrow (technically today), @sutdsg will be hosting SUTD AI Day where @preslav_nakov @pliang279 and @hwaran_lee will deliver three amazing keynotes!
See sutdaiday.github.io/sutdaidaโฆ for more information!
Now, Dr. Faeze Brahman @faeze_brh is delivering "Open Language Model Adaptation and Reliable Evaluation"! An exciting time for open-source and reproducible models! #SSNLP2025
Prof. @VioletNPeng delivering a talk on "Controllable and Creative Natural Language Generation", lowering the barrier for the layman! #SSNLP2025 โ at Singapore





















