Samuel Horváth (@sam_hrvth) profile picture

Assistant professor of Machine Learning @mbzuai, Ph.D. @KAUST_News, former intern @metaAI, @samsungresearch, and @amazon. Co-organizer of @flow_seminar.

Abu Dhabi, UAE
Joined April 2022
I'm speaking at ICOMP 2026 and ASCOMP in Casablanca, Morocco, this week about our work at @Pluralis: pooling compute from many contributors to train foundation models far beyond the capacity of any single device. I'll cover some of our recent architectural and algorithmic innovations that make this practical. We see this as a path to sustainable open-source AI: open, collective development and innovation while weights stay private and no participant ever holds them in full. If you're attending, I'd be interested to have a chat.
4
1
19
485
Samuel Horváth retweeted
I am looking for the amazing postdocs to advance the foundations of optimization, machine learning, and AI. Requirement: - Outstanding training and track record in mathematics and computer science. - Stellar research track record in optimization, ML and AI, a closely related field. Supplementary info: richtarik.org/i_apply.html
Lead your independent research. Create global impact.​ Designed for exceptional early-career researchers, KAUST Global Fellowship offers the opportunity to pursue independent research in areas aligned with global priorities, including Health, Water, Energy, Food Security, AI, and Marine Science.​ Register your interest: kgfp.kaust.edu.sa/
1
8
1
34
5,548
Samuel Horváth retweeted
🚨 OPEN WEIGHTS RELEASED The FlexRank weights are officially out! Curious to see where this takes us — whether in applications, experiments, or new research directions. 👀 👇
1
2
5
234
Samuel Horváth retweeted
At @icmlconf, the most common question about FlexRank was: “Can it really create smaller versions of an LLM?” In practice: “Does this scale if I invest in making it production-ready?” So we ran more experiments to answer that. Our findings 🧵 1/3
4
2
4
283
Samuel Horváth retweeted
Super excited to kick off the AdaptFM @ #ICML2026 workshop on Resource-Adaptive Inference. We are in Ballroom 202 with a great line-up of speakers and contributed papers. Program: adaptfm.gitlab.io/program/ Papers: adaptfm.gitlab.io/accepted-p… Accepted Papers: adaptfm.gitlab.io/accepted-p…
2
8
438
Samuel Horváth retweeted
We had an amazing time yesterday at the Protocol Learning Workshop at #ICML2026 in Seoul! It was a full day of learning from researchers working across distributed optimisation, decentralised training, and large-scale LLM training. Here’s a brief recap 🧵 1/n
1
6
2
29
5,021
Samuel Horváth retweeted
Just in time for ICML week, we are sharing our take on a key question for recursive self-improving AI. How can AI keep improving if the evaluator stays fixed? Our answer is the Red Queen Gödel Machine (RQGM), developed as a collaboration between the @CaMLSys, @nvidia, @flwrlabs, @mbzuai and @Inria, and led by @Alex__Iacob (@Cambridge_Uni). It is still early, but the potential is high. One headline result is that, in a few narrow cases, we show an agent using largely Nemotron 3 Ultra can reach similar performance to GPT 5.5. Self-improvement is one of the more plausible bets for the next major breakthrough in AI. But there is a core issue limiting a lot of this work. The evaluator. Current approaches often assume something like a benchmark or reward model that sits outside the self-improvement loop. That works for a while. But evaluators saturate, they get gamed, and as the system improves, they can stop being useful. Our Red Queen Gödel Machine lets the agent and evaluator improve together. The agent gets better at doing the task. The evaluator gets better at judging it. On coding, it already beats the prior best search method for self-improving AI while using 1.35–1.72× fewer tokens, and the same idea carries over to paper writing and proof grading. Paper: arxiv.org/abs/2606.26294 Full RQGM team: @Alex__Iacob, @itsmaddox_j, @williamfshen, @DBBurkhardt, @meghdadkurmanji, Nurbek Tastan, @lorenzosani97, @NickVenanzi, @AmbroiseOdonnat, Jamie Cao, Bill Marino, @xinchiqiu, and @niclane7.
2
8
1
36
1,940
Samuel Horváth retweeted
✨ [SPOTLIGHT] FlexRank: Nested Low-Rank Knowledge Decomposition for Adaptive Model Deployment How to make any pretrained model ready for elastic deployment: turn it into a nested, low-rank structure and resize it for different inference budgets. Led by @RickZack96, with @sam_hrvth & @stevelaskaridis 📌Poster: Wed Jul 8 · 2:30–4:15 PM KST · Hall A #2208 🧵Full thread: nitter.cf/RickZack96/status/2063…
Are curious about how to resize any pretrained model on demand? Then you'll love our paper "FlexRank: Nested Low-Rank Knowledge Decomposition for Adaptive Model Deployment​", accepted at ICML 2026 (Spotlight) Joint work with @sam_hrvth, @stevelaskaridis, @mciccone_AI 🧵1/7
3
4
169
Samuel Horváth retweeted
Excited for #ICML2026 in Seoul 🇰🇷 next week! We’ll be presenting two papers on adaptive inference: - FlexRank: arxiv.org/abs/2602.02680 (Spotlight) - MoSE: arxiv.org/abs/2602.06154 We’re also organising the AdaptFM workshop (July 11): adaptfm.gitlab.io/ See you there!
4
12
417
Samuel Horváth retweeted
We’re excited to share our #ICML2026 paper on optimizer robustness in Async Pipeline Parallel LLM pretraining Main message: with the right PP schedule, optimizer choice, and Error Feedback correction, Async PP can match sync training quality even at 10B MoE scale [1/10]
3
15
4
45
5,096
Our second Protocol Learning workshop will be held in Seoul on 10th of July, next to ICML convention centre. There will be many interesting talks on optimization and large-scale distributed training. Registration link below. We're also accepting lightning talks and poster presentations in relevant topics.
We are hosting a workshop during ICML on Protocol Learning to examine the key open challenges in collaborative training across distributed networks. Talks by @Ana_koloskova @sam_hrvth @hmdolatabadi & more speakers to be announced Register here: pluralis.ai/events/icml-prot…
1
3
566
Samuel Horváth retweeted
🔥 A sneak peek at the incredible #ICOMP2026 lineup! Here is a subset of our confirmed plenary speakers so far: 🎙️ Michael Jordan 🎙️ Anatoli Juditsky 🎙️ @SebastienBubeck (online) 🎙️ @Ana_koloskova 🎙️ @konstmish 🎙️ @orvieto_antonio More stellar speakers to be announced soon!
Replying to @ed_gorbunov
We're excited to welcome the following confirmed plenary speakers: Sebastian Bubeck (@SebastienBubeck, online), Anatoli Juditsky, Michael Jordan, Anastasia Koloskova (@Ana_koloskova), Konstantin Mishchenko (@konstmish), and Antonio Orvieto (@orvieto_antonio).
2
1
9
2,080
Samuel Horváth retweeted
Are curious about how to resize any pretrained model on demand? Then you'll love our paper "FlexRank: Nested Low-Rank Knowledge Decomposition for Adaptive Model Deployment​", accepted at ICML 2026 (Spotlight) Joint work with @sam_hrvth, @stevelaskaridis, @mciccone_AI 🧵1/7
6
4
5
24
9,713
Samuel Horváth retweeted
🚨 Postdoc opening in my group at Johns Hopkins on min-max optimization, ML/AI & large-scale RL. Apply by June 15 for full consideration: apply.interfolio.com/184958 I’ll be at #SIAMOP26 in Edinburgh next week! Please reach out, happy to chat! @HopkinsDSAI, @JohnsHopkinsAMS
🤖 Made with AI
6
3
18
5,277
Samuel Horváth retweeted
📣 We're extending the AdaptFM @ ICML'26 paper submission deadline to May 8! This will align better with the ICML acceptance announcements and NeurIPS deadlines. 👉 Submit your work at: openreview.net/group?id=ICML… 🌐 CfP: adaptfm.gitlab.io/
2
3
571
Samuel Horváth retweeted
🚀 We're launching the Efficient Qwen Competition @ AdaptFM #ICML2026! How fast can you make Qwen3.5-4B run on a single A10G without breaking it? $6,000 in prizes. Open to all. Submissions open May 4. 👉 adaptfm.gitlab.io/call-for-c… #LLM #EfficientML #ModelOptimization
2
2
2
475
Samuel Horváth retweeted
📢 Announcing the AdaptFM Workshop @icmlconf As foundation models grow in scale and ubiquity, the ability to adapt inference dynamically to the task and available resources becomes critical. Submission deadline: May 1, 2026 (AoE) 📍Seoul, South Korea 🌐adaptfm.gitlab.io
2
3
1
5
1,366
Samuel Horváth retweeted
I am pleased to announce that together with 3 of my (then present and now former) KAUST PhD students - Samuel Horváth @sam_hrvth (now Assistant Professor at MBZUAI, Abu Dhabi), - Dmitry Kovalev @dakovalev1 (now Research Scientist at Yandex, Moscow), - Konstantin Mishchenko @konstmish (now Research Scientist at Meta, Paris) and Sebastian U Stich @SebastianUStich (now Faculty at CISPA, Saarbrücken; who was visiting us at KAUST at the time) we have won the  2023 Charles Broyden Prize for the paper Stochastic distributed learning with gradient quantization and double-variance reduction the paper: tandfonline.com/doi/full/10.… previous prizes: tandfonline.com/journals/gom… ***** The Charles Broyden Prize is an annual international award honoring the best paper published in the journal Optimization Methods and Software (OMS) during the preceding year. Established in 2009, the prize commemorates the life and work of British mathematician Charles George Broyden (1933–2011), a pioneer in numerical optimization known for his namesake methods and his role in the development of the BFGS algorithm. ***** I consider this prize to be shared with the authors of the original DIANA paper Konstantin Mishchenko @konstmish, Eduard Gorbunov @ed_gorbunov, Martin Takáč @TakacMartin and P.R. Distributed learning with compressed gradient differences, arxiv.org/abs/1901.09269, which was the inspiration for this follow up work that polished, generalized and improved it, and the authors of the previous SEGA work, Filip Hanzely, Konstantin Mishchenko @konstmish and P.R.. SEGA: Variance reduction via gradient sketching, Advances in Neural Information Processing Systems 31:2082-2093, 2018, which can be seen as a single-node variant of DIANA, as well as the authors of the JacSketch paper, which was the inspiration for SEGA: Robert M. Gower @gowerrobert, P.R. and Francis Bach @BachFrancis, Stochastic quasi-gradient methods: Variance reduction via Jacobian sketching, Mathematical Programming 188:135–192, 2021 (arxiv.org/abs/1805.02632) One can keep going like this, since every new discovery builds on prior work, but I'll stop here. So, once again, congrats to the authors of the award-winning paper, as well as to the authors of all these prior works!
4
10
1
151
11,947
Samuel Horváth retweeted
Read about our faculty recognition awards here: mbzuai.ac.ae/news/mbzuai-cel… Read about the MBZUAI Teaching Award here: mbzuai.ac.ae/news/hisham-cho… Find all our faculty here: mbzuai.ac.ae/study/faculty-d…
1
4
781
Samuel Horváth retweeted
🚀 AI4OPT Challenge! Build an agentic AI that reads, experiments, and writes a mini-paper. Tweak the pipeline. Beat the baselines. Win an award. Check: ai4opt.fmin.xyz #AgenticAI #ML #Optimization #LLM #MBZUAI #ICOMP
3
5
818