Research Intern at Google (Intrinsic) for Model Optimization. PhD in Deep Learning @ the University of Basel. Past: Flexion Robotics, Yahoo! Research.
Zurich
Joined May 2022
- Tweets214
- Following464
- Followers266
- Likes918
Pinned Tweet
Cool! We derived an AdamW batch-size scaling rule in Lemma 3.13 of arxiv.org/abs/2411.15958
B → B × k
𝗟𝗥 → 𝗟𝗥 × sqrt(𝗸)
𝗪𝗗 → 𝗪𝗗 × sqrt(𝗸)
βᵢ → 1 − sqrt(k) × (1 − βᵢ)
This was subsequently validated empirically in an ICLR 2026 paper:
openreview.net/pdf?id=elB9k4…
I'm posting a series of prediction questions from Stanford CS312 (Deep Learning Alchemy) exams to test deep learning intuition.
First up: optimization. If we increase an LM's batch size 8× (same token budget), can rescaling learning rate or weight decay compensate? Try below ↓
This quiz is part of our optimizer hyperparameter assignment and unit designed by @wen_kaiyue
Setup: We train an 8-layer, ~35M-parameter Transformer on 614.4M tokens using AdamW. At batch size 64, the best reference run has peak LR 0.003, weight decay (WD) 0.2, and final validation loss 2.9178.
Now increase the batch size from 64 to 512, which gives 8× fewer optimizer steps. Everything else stays fixed except the changes below.
Which recipe gives the lowest final validation loss?
A. Keep LR and WD unchanged
B. LR ×√8
C. LR ×8
D. WD ×8
E. LR ×√8 and WD ×√8
What would you predict, and why?
Enea Monzio Compagnoni retweeted
Excited to share about what I’ve been working on over the past year: quantifying the capacity of a neuron😅
This led to a mathematical framework we call HOPE, which lets us rigorously deconstruct what deep networks might have learned. Paper: arxiv.org/abs/2607.21366
Btw, this was joint work with the formidable @ruuustem_10 and @orvieto_antonio, who have lately become Scaling Law Overlords:
arxiv.org/abs/2603.15958
arxiv.org/abs/2603.21191
And @tianlinliu0121, whose expertise was invaluable in successfully validating our theory!
Enea Monzio Compagnoni retweeted
I'm posting a series of prediction questions from Stanford CS312 (Deep Learning Alchemy) exams to test deep learning intuition.
First up: optimization. If we increase an LM's batch size 8× (same token budget), can rescaling learning rate or weight decay compensate? Try below ↓
This quiz is part of our optimizer hyperparameter assignment and unit designed by @wen_kaiyue
Setup: We train an 8-layer, ~35M-parameter Transformer on 614.4M tokens using AdamW. At batch size 64, the best reference run has peak LR 0.003, weight decay (WD) 0.2, and final validation loss 2.9178.
Now increase the batch size from 64 to 512, which gives 8× fewer optimizer steps. Everything else stays fixed except the changes below.
Which recipe gives the lowest final validation loss?
A. Keep LR and WD unchanged
B. LR ×√8
C. LR ×8
D. WD ×8
E. LR ×√8 and WD ×√8
What would you predict, and why?
Enea Monzio Compagnoni retweeted
Giving a talk today at @MistralAI optimization seminar. Many thanks to @konstmish for organizing the series of talks and inviting me. I will talk about our recent works on hyperparameter scaling laws for LLMs. Looking forward to fruitful discussion with the team
Enea Monzio Compagnoni retweeted
I'm hiring a Student Researcher to work on scaling laws at Google DeepMind! Project is for 19 weeks, starting winter/spring '27 (ideally: January '27), in-person in downtown SF. If you're interested, fill out this form: forms.gle/9MKo9ciBmp5wsyiU7
Enea Monzio Compagnoni retweeted
Happy to share that I co-authored two papers accepted to NeurIPS: one on local SGD: arxiv.org/abs/2607.14731 and one on Adam-SGD gap: arxiv.org/abs/2606.14259. Thanks a lot to all collaborators!
Enea Monzio Compagnoni retweeted
We're excited to be partnering with @odysseyml to bring foundation world models into physical applications.
Here's a first look at the world action model we've been training.
Read more about our collaboration on their blog.
Enea Monzio Compagnoni retweeted
We have been named the Hottest Robotics Startup at the TOP 100 Swiss Startup Awards.
Building general-purpose intelligence that can adapt to any robot requires exceptional people working across every layer.
Congratulations to the entire Flexion team for teaching robots how to think, move and adapt.
This is just the beginning.
#GeneralRobotIntelligence #SwissStartups
🔥 Meet the winners of this year’s Hottest Startup Special Awards!
Tonight at the Top100 Swiss Startup Award 2026, for the first time, three standout companies out of this year’s ranking are being recognized for their achievements in some of Switzerland’s most exciting innovation fields:
🌱 Hottest Cleantech Startup: Voltiris (ranked #4)
🧬 Hottest Biotech Startup: Aukera Therapeutics (ranked #25)
🤖 Hottest Robotics Startup: @FlexionAI (ranked #6)
From advancing sustainable technologies to pushing the boundaries of biotechnology and robotics, these three startups are making their mark in their respective industries.
Congratulations to all three winners! 🚀
A big thank you to our presenting partner, @UBS, and to the great event partners: Amt für Wirtschaft (@KantonZuerich), Awardness, @grstiftung_ch, @Handelszeitung, @ifj_ch, PME, @startuptickerCH, STARTUP.CH, @Swisscom Ventures, @venturekick, and Walder Wyss.
#TOP100SSU #SwissStartups #Robotics #Cleantech #Biotech
Enea Monzio Compagnoni retweeted
Former Anthropic employee Jacob Coxon says the AI industry is "gambling with our lives." He tells Anderson what he finds "most scary is if AI is used to make itself more intelligent" and warns these companies are "compelled to race toward building a deadly technology."
Enea Monzio Compagnoni retweeted
Great collaboration with @NianticSpatial
Trained a humanoid entirely in a 3D scan of the office.
Zero real-world fine-tuning. It just walked in and worked.
RL needs hundreds of thousands of attempts, and real robots can't afford to crash. A misjudged gap or a glass door collision breaks hardware and costs hours resetting. So you train in a sim.
But sim policies usually train on randomized, untextured geometry; depth is easy to fake. The robot learns structure, not the real world: no materials, no lighting, no idea what anything actually is. RGB cameras carry all of that but training RGB policies in generic fake worlds won’t generalize to the real world.
@NianticSpatial Scaniverse reconstructs your scan of the real deployment site. One 360° camera walkthrough → photorealistic 3D Gaussian splat at metric scale → collision mesh pulled from the same reconstruction, so vision and physics match exactly. Drops straight into NVIDIA Isaac Sim/Lab, no manual conversion.
@FlexionAI simulation-first approach then seamlessly enables the training of RGB-only nav policies inside that reconstruction. With added domain randomization + large image encoders for robustness, this deploys straight to hardware.
No real-world fine-tuning. Deployment: months of on-site adaptation → days.
Tune into the NVIDIA livestream on 12 August to hear how these companies are closing the sim2real gap: addevent.com/event/gvb8rkdrc…
@NVIDIARobotics
~~
♻️ Join the weekly robotics newsletter, and never miss any news → ziegler.substack.com
Enea Monzio Compagnoni retweeted
Historically, RL policies for robots have been trained in synthetic, untextured environments, limiting perceptive policies to depth images where the sim-to-real gap is manageable. RGB has always had more potential, but leveraging it to train policies in simulation remained an open problem.
Partnering with @NianticSpatial and @NVIDIARobotics, we built a pipeline that addresses exactly that. We can now scan a real deployment site with off-the-shelf hardware, reconstruct it into a photorealistic Gaussian splat, and run massively parallel RL training. The policies trained in our Gym environment then transfer zero-shot to the real robot and environments they were trained for. This enables faster deployment of more capable and robust policies for the end user.
The new resulting capabilities are a big step towards solving sim-to-real and also apply well beyond navigation. Read the full technical breakdown on our blog; link in the comments.
#HumanoidRobots #Flexion #NianticSpatial #NVIDIA
Enea Monzio Compagnoni retweeted
Our Workshop on Weight-Space Symmetries @icmlconf @weightsymmetry is starting now! You can reach Room 403 by going up from Hall A.
Program: weightsymmetry.com/schedule
Enea Monzio Compagnoni retweeted
(1/6) A Warp Drive for Neural PDE Solvers???
Tomorrow in Seoul, Till and Alex are presenting their floral #ICML2026 Spotlight. Don’t miss it if you care about neural surrogates for physics, foundation models, physics world models, …
Enea Monzio Compagnoni retweeted
The oral presentation is happening today!
📍Grand Ballroom 101-105
⏰10:15 am KST
Happy to chat during the poster session as well:
📍Hall A, poster #3708
⏰2:30pm KST
The morning could not start any better:) I’m happy and grateful that our work on non-Euclidean Edge of Stability with @CrichaelMawshaw @deepcohen and @gowerrobert was selected for an oral presentation at @icmlconf in Seoul! Big thanks to the committee!
That's a wrap on our #ICML2026 poster!
Thanks to everyone who stopped by. The questions and pushback were the best part.
"SDEs have been lying to you" turned out to be a solid conversation starter😄
Missed it & curious? Read the paper:
arxiv.org/abs/2506.00181
Grazie Seoul!🇰🇷
Slightly provocative poster at #ICML2026 this week: the SDEs we trust to analyze SGD can predict convergence right where it's actually unstable - so we fixed them.
📍Wed Jul 8, 2:30 PM, Hall A #3600. Come by and tell me what you think!🙂
(The pic is me as a K-caricature, lol.)
Slightly provocative poster at #ICML2026 this week: the SDEs we trust to analyze SGD can predict convergence right where it's actually unstable - so we fixed them.
📍Wed Jul 8, 2:30 PM, Hall A #3600. Come by and tell me what you think!🙂
(The pic is me as a K-caricature, lol.)
Enea Monzio Compagnoni retweeted
We are at ICML 2026! 🇰🇷
📅 Date: Friday, July 10, 2026
📍 Room 403, COEX, Seoul
🗓️ Workshop Schedule: icml.cc/virtual/2026/worksho…
🌐 More Details: weightsymmetry.com/
