Ming Shan (@mingshanhee) profile picture

Ph.D under Prof @sroylee at SUTD (@sutdsg) | Trustworthy and Explainable Vision-Language Approaches | Hateful Memes & Multimodal Misinformation

Singapore
Joined June 2014
Photo by Ming Shan (@mingshanhee)Photo by Ming Shan (@mingshanhee)Photo by Ming Shan (@mingshanhee)Photo by Ming Shan (@mingshanhee)Photo by Ming Shan (@mingshanhee)Photo by Ming Shan (@mingshanhee)Photo by Ming Shan (@mingshanhee)Photo by Ming Shan (@mingshanhee)Photo by Ming Shan (@mingshanhee)Photo by Ming Shan (@mingshanhee)
Not easy, definitely tough, but incredibly fruitful. Tackling large-scale infrastructure bottlenecks, especially eliminating infrastructure bubbles to improve experiment throughput, was a major challenge. Excited for whatโ€™s next with our new Wrench framework! ๐Ÿ”ง
We're open-sourcing RL360, the RL infrastructure behind K2 Horizon's six domain experts: 8,192 concurrent sandboxes, 150 steps in 7 days, ~1.3M rollouts. Long-horizon agentic RL runs thousands of agents in real sandboxes on tasks that take minutes. Some of those sandboxes crash mid-task, and a crash arrives as the same low score as a wrong answer. Without infrastructure that can keep up, GPUs are underutilized and the model gets penalized for system crashes rather than its own mistakes. RL360 is built for this. We share how it works, what broke at scale, and what we learned in the thread. Code: github.com/ifm-ai/RL360 Run: wandb.ai/mbzuai-llm/rl360-puโ€ฆ
2
53
๐Ÿ—“๏ธ Mon Oct 5, 8โ€“10pm PT AMA with the team behind K2 Horizon, hosted by r/LocalLLaMA (@LocalLLaMAsub). Ask us anything about models, deployment, training, what's open so far and what's next. ๐Ÿ’ฌ Questions are open now. We start answering when we go live. Drop your questions here ๐Ÿ‘‰ย redd.it/1wv8zww
4
6
22
1,670
Journal and conference submissions are getting out of hand. Definitely need stricter screening and more rigorous peer review.
TMLR has faced a deluge of submissions, necessitating stricter desk rejection policies due to limited reviewer capacity Co-EiC Nihar Shah reached out to authors of 10 papers slated for desk reject. Could they answer questions about their *own* submission? medium.com/@TmlrOrg/asking-aโ€ฆ
59
Introducing K2 Horizon: a connected fleet of six foundation models ranging from 0.9 billion to 375 billion parameters. - Frontier performance: Across coding and agentic tasks, K2 Horizon delivers top-tier performance in every size classโ€”with the 0.9B, 3.7B and 7B models setting new state of the art at their respective scales. - Radical openness: K2 Horizon represents the largest fully open-source model launch in AI history. The fully open code, training data and recipes are a significant step forward in transparency. Launch page: ifm.ai/k2/ Tech blog: ifm.ai/blog/k2 Hugging Face: huggingface.co/collections/Iโ€ฆ
182
363
291
1,999
1,279,752
Meet K2 Horizon.
47
119
116
429
311,539
Stay tuned.
19
7
68
6,755
๐— ๐—ถ๐˜…-๐—ฅ๐—Ÿ (pooling prompts from different domain into a single joint RL run), is the most common technique for multi-task RL post-training of LLMs. Train on math, code, and instruction-following prompts together in one GRPO run, and in principle you get one model that's good at all three. But in practice it runs into a well known problem: the ๐˜€๐—ฒ๐—ฒ-๐˜€๐—ฎ๐˜„ ๐—ฒ๐—ณ๐—ณ๐—ฒ๐—ฐ๐˜. Push one domain up and another quietly drops when trained jointly, even though each domain improves fine on its own. The recent MOPD paper states that cross-domain training signals interfere with each other, and the joint model ends up below every one of the per-domain specialists it was supposed to combine. (arxiv.org/abs/2606.30406). There's concrete evidence of this too. The Modular Gradient Surgery paper shows that even a heavily skewed ๐Ÿต๐Ÿฌ% ๐—บ๐—ฎ๐˜๐—ต / ๐Ÿญ๐Ÿฌ% ๐—ฐ๐—ต๐—ฎ๐˜ mixture still lags behind a math-only model trained the same number of epochs. Skewing the data mix in favor of a domain doesn't rescue it (arxiv.org/abs/2602.02301). I read through a bunch of recent papers that investigate into this issue and try to point towards the root causes. Below I summarize the main findings: ๐Ÿญ. ๐—š๐—ฟ๐—ฎ๐—ฑ๐—ถ๐—ฒ๐—ป๐˜ ๐—บ๐—ฎ๐—ด๐—ป๐—ถ๐˜๐˜‚๐—ฑ๐—ฒ ๐—ถ๐—บ๐—ฏ๐—ฎ๐—น๐—ฎ๐—ป๐—ฐ๐—ฒ, ๐—ฑ๐—ฒ๐—ฐ๐—ผ๐˜‚๐—ฝ๐—น๐—ฒ๐—ฑ ๐—ณ๐—ฟ๐—ผ๐—บ ๐—ฎ๐—ฐ๐˜๐˜‚๐—ฎ๐—น ๐—น๐—ฒ๐—ฎ๐—ฟ๐—ป๐—ถ๐—ป๐—ด The Imbalanced Gradients paper (Meta) shows certain tasks (e.g. code) produce squared gradients up to ๐Ÿญ๐Ÿฑ๐˜… larger than others (e.g. math) in joint training, but large-gradient tasks often show similar or lower learning gains than small-gradient ones. The shared update gets biased toward whichever task shouts loudest, not whichever task needs it. (Wu et al., Imbalanced Gradients in RL Post-Training of Multi-Task LLMs, arxiv.org/abs/2510.19178) ๐Ÿฎ. ๐—š๐—ฅ๐—ฃ๐—ข'๐˜€ ๐—ด๐—ฟ๐—ผ๐˜‚๐—ฝ ๐—ป๐—ผ๐—ฟ๐—บ๐—ฎ๐—น๐—ถ๐˜‡๐—ฎ๐˜๐—ถ๐—ผ๐—ป ๐˜€๐˜๐—ฎ๐—ฟ๐˜ƒ๐—ฒ๐˜€ ๐—ต๐—ฎ๐—ฟ๐—ฑ๐—ฒ๐—ฟ ๐—ฑ๐—ผ๐—บ๐—ฎ๐—ถ๐—ป๐˜€ The Multi-Task GRPO paper and the DISCO paper both show that prompts where every rollout in a group gets an identical reward produce zero advantage and zero gradient, so domains with more "live" (non-saturated) prompts dominate the pooled update by construction. Naively upweighting the weaker domain doesn't fix this without explicit ratio-preserving sampling. (Multi-Task GRPO, arxiv.org/abs/2602.05547; DISCO, arxiv.org/abs/2505.15074) ๐Ÿฏ. ๐—š๐—ฟ๐—ฎ๐—ฑ๐—ถ๐—ฒ๐—ป๐˜ ๐—ฑ๐—ถ๐—ฟ๐—ฒ๐—ฐ๐˜๐—ถ๐—ผ๐—ป ๐—ฐ๐—ผ๐—ป๐—ณ๐—น๐—ถ๐—ฐ๐˜, ๐—ป๐—ผ๐˜ ๐—ท๐˜‚๐˜€๐˜ ๐—บ๐—ฎ๐—ด๐—ป๐—ถ๐˜๐˜‚๐—ฑ๐—ฒ The Modular Gradient Surgery paper computes math and chat gradients separately on the same shared weights and finds negative cosine similarity between them: on many weights, the two updates point in opposite directions. Averaging them into one Mix-RL step causes partial cancellation instead of accumulation, so the net update is a diluted compromise that serves neither domain well. This is the direct mechanism behind the underperformance above. (Cai et al., Modular Gradient Surgery, arxiv.org/abs/2602.02301) ๐Ÿฐ. ๐—ฅ๐—ฒ๐˜„๐—ฎ๐—ฟ๐—ฑ-๐˜€๐—ฐ๐—ฎ๐—น๐—ฒ ๐—ถ๐—ป๐—ฐ๐—ผ๐—ป๐˜€๐—ถ๐˜€๐˜๐—ฒ๐—ป๐—ฐ๐˜† ๐—น๐—ฒ๐˜๐˜€ ๐—ผ๐—ป๐—ฒ ๐˜€๐—ถ๐—ด๐—ป๐—ฎ๐—น ๐—ผ๐˜ƒ๐—ฒ๐—ฟ๐—ฟ๐—ถ๐—ฑ๐—ฒ ๐—ฎ๐—ป๐—ผ๐˜๐—ต๐—ฒ๐—ฟ The "Two is better than one" paper shows that when combined reward components differ in numerical scale, the higher-variance component dominates the per-token gradient by roughly an order of magnitude while the other gets effectively zeroed out. It demonstrates this within a single domain's combined reward channels, but it's the same scale-domination mechanism that shows up across domains with differently calibrated rewards. (arxiv.org/abs/2605.22620) ๐Ÿฑ. ๐—ฆ๐—ต๐—ฎ๐—ฟ๐—ฒ๐—ฑ ๐—ฝ๐—ฎ๐—ฟ๐—ฎ๐—บ๐—ฒ๐˜๐—ฒ๐—ฟ๐˜€ ๐—บ๐—ฎ๐—ธ๐—ฒ ๐˜๐—ฟ๐˜‚๐—ฒ ๐—ถ๐—ป๐—ฑ๐—ฒ๐—ฝ๐—ฒ๐—ป๐—ฑ๐—ฒ๐—ป๐—ฐ๐—ฒ ๐—ถ๐—บ๐—ฝ๐—ผ๐˜€๐˜€๐—ถ๐—ฏ๐—น๐—ฒ The Local Perturbation Theory paper (Yang, Ding & Xiong) shows that full-model gradients across domains can look nearly orthogonal while a later domain still damages an earlier one, through a second-order effect concentrated in a low-dimensional shared parameter subspace. This is shown for sequential/cascade multi-domain RL rather than simultaneous joint training, but it points at the same underlying constraint: one set of weights, multiple domains pulling on it. (arxiv.org/abs/2606.02398)
4
8
46
4,641
Today, MBZUAI released K2-V2, a fully transparent and open-source 70B model built entirely from scratch. I am happy to be part of this incredible team that is committed to developing transparent and reproducible models.
Today, we are releasing a new version of K2 (K2-V2), a 360-open LLM built from scratch as a superior base for reasoning adaptation, while still excelling at core LLM capabilities like conversation, knowledge retrieval, and long-context understanding. K2 fills a major gap: highly capable models with no transparency. Instead of releasing only weights, weโ€™re sharing the full training story โ€” dataset recipes, mid-training checkpoints, logs, code, and evaluation tools. Thatโ€™s 360-open. Whatโ€™s inside: โ€ข 70B dense transformer engineered as a reasoning-enhanced base model โ€ข Native 512K context (extendable via RoPE scaling) โ€ข Mid-training reasoning phase โ€ข Strong tool-use scaffolding What weโ€™re open-sourcing: โ€ข 250M+ reasoning traces (math, planning, multi-step logic) โ€ข Full pre- & mid-training data compositions โ€ข All mid-training checkpoints โ€ข Training logs, code, Eval360 Performance: โ€ข GPQA-Diamond: 55.1% mid-training โ†’ 69.3% after SFT (strongest fully open 70B model) โ€ข KK-8 Logic Puzzles: 83% โ€” competitive with DeepSeek-R1 & OpenAI o3-mini-high โ€ข ArenaHard V2: 62.1% โ€” close to Qwen3 235B โ€ข Outperforms Qwen2.5-72B and approaches Qwen3-235B despite being smaller and fully transparent. ๐Ÿ”— The Model: bit.ly/3KIYwuo ๐Ÿ”—Technical Report: bit.ly/49V8h2U ๐Ÿ”—Blog: bit.ly/49V7gb6
3
269
Now there is roughly 90GB worth of photos and footage to edit (total about 2 hours)... What have I gotten myself into...
Find yourself a @mingshanhee who captures the world so you can truly live in the moment!
1
5
364
So ICWSM Main Conference Day 1 has kicked off! Catch me at "Session 1: Multimodal Meme Analysis" from 10:30โ€ฏAM to 11:00โ€ฏAM, where Iโ€™ll be presenting two papers.
2
2
10
720
2) Contrastive Instruction Fine-Tuning Large Multimodal Model for Hateful Meme Classification (ojs.aaai.org/index.php/ICWSMโ€ฆโ€ฆ) - Post-train LMMs for hateful meme classification tasks - Applies Contrastive Learning for better classification boundaries
1
1
107
1) Demystifying Hateful Content: Leveraging Large Multimodal Models for Hateful Meme Detection with Explainable Decisions (ojs.aaai.org/index.php/ICWSMโ€ฆ) - Reduce reliance on model's implicit knowledge - Provide possible explanations behind models' decisions
1
1
83
Flying to attend @icwsm 2025 in Copenhagen, Denmark now! Let's talk about memes, social problems, and MAYBE have a few Danish pastries ๐Ÿ‡ฉ๐Ÿ‡ฐโœจ #ICWSM2025
1
1
11
1,183
And nightmare happens... @overleaf went down... ๐Ÿฅฒ
2
2
1
16
3,147
SUTD AI Day concludes with @hwaran_lee presenting on "Evaluation with socio-cultural awareness and multilingual LLMs" ๐ŸŽ‰!
3
3
375
Next at SUTD AI Day, we have @pliang279 delivering keynote on "New Advances in Multimodal Reasoning"!
2
1
126
SUTD AI Day, held at @sutdsg, has officially begun! Starting with @preslav_nakov presenting on "Towards Truly Open, Language-Specific, Safe, Factual, and Specialized Large Language Models."
3
6
326
Tomorrow (technically today), @sutdsg will be hosting SUTD AI Day where @preslav_nakov @pliang279 and @hwaran_lee will deliver three amazing keynotes! See sutdaiday.github.io/sutdaidaโ€ฆ for more information!
3
7
457
Now, Dr. Faeze Brahman @faeze_brh is delivering "Open Language Model Adaptation and Reliable Evaluation"! An exciting time for open-source and reproducible models! #SSNLP2025
1
2
2
113
Prof. @VioletNPeng delivering a talk on "Controllable and Creative Natural Language Generation", lowering the barrier for the layman! #SSNLP2025 โ€“ at Singapore
3
6
514