Riccardo Zaccone (@RickZack96) header
Riccardo Zaccone (@RickZack96) profile picture

PhD Candidate in Computer Eng. (distr. learning, efficiency) @PoliTONews | Alumno IEEE-HKN @HKNPoliTo

Turin, Italy
Joined August 2022
Photo by Riccardo Zaccone (@RickZack96)Photo by Riccardo Zaccone (@RickZack96)Photo by Riccardo Zaccone (@RickZack96)Photo by Riccardo Zaccone (@RickZack96)Photo by Riccardo Zaccone (@RickZack96)Photo by Riccardo Zaccone (@RickZack96)Photo by Riccardo Zaccone (@RickZack96)Photo by Riccardo Zaccone (@RickZack96)Photo by Riccardo Zaccone (@RickZack96)Photo by Riccardo Zaccone (@RickZack96)
Are curious about how to resize any pretrained model on demand? Then you'll love our paper "FlexRank: Nested Low-Rank Knowledge Decomposition for Adaptive Model Deployment​", accepted at ICML 2026 (Spotlight) Joint work with @sam_hrvth, @stevelaskaridis, @mciccone_AI 🧵1/7
6
4
5
24
9,713
I'd add: (make your agent) use design principles. They're no less useful than when used by humans, and a machine can have complete access to pros/cons of everything we understood about soft. eng. Less code alone only works for naturally small projects
Reposting this because it's becoming more useful every day Less code means cheaper agents, shorter context, less context rot, and fewer bugs
2
104
🚨 OPEN WEIGHTS RELEASED The FlexRank weights are officially out! Curious to see where this takes us — whether in applications, experiments, or new research directions. 👀 👇
1
2
5
234
At @icmlconf, the most common question about FlexRank was: “Can it really create smaller versions of an LLM?” In practice: “Does this scale if I invest in making it production-ready?” So we ran more experiments to answer that. Our findings 🧵 1/3
4
2
4
283
My take: - More compute on the same data is hitting diminishing returns — broader data is the next lever. - Distillation works remarkably well — now we need better data for downstream tasks. Keep an eye out — we’ll be releasing the weights soon!
42
You may notice a small gap at full budget (parameter count = 1.0). Is something wrong with optimization? Not quite. Distilling from the base model (grey dashed line) works surprisingly well: up to 80% of the parameters, models achieve comparable or even lower loss.
60
The results are encouraging: simply training longer gives a large boost in downstream performance. No new data. No tricks. Just more training.
48
In our paper, we prioritized broad applicability over convergence in a single setting. But what if we just scale? We extended Llama 3.2-1B from 5B → 60B tokens, with the same budget as LlamaFlex, still on FineWeb.
56
Riccardo Zaccone retweeted
Second Protocol Learning Workshop at ICML is a wrap. A packed day of talks and posters covering large scale distributed training and open source AI. A movement is growing.
1
5
1
51
3,747
Riccardo Zaccone retweeted
✨ [SPOTLIGHT] FlexRank: Nested Low-Rank Knowledge Decomposition for Adaptive Model Deployment How to make any pretrained model ready for elastic deployment: turn it into a nested, low-rank structure and resize it for different inference budgets. Led by @RickZack96, with @sam_hrvth & @stevelaskaridis 📌Poster: Wed Jul 8 · 2:30–4:15 PM KST · Hall A #2208 🧵Full thread: nitter.cf/RickZack96/status/2063…
Are curious about how to resize any pretrained model on demand? Then you'll love our paper "FlexRank: Nested Low-Rank Knowledge Decomposition for Adaptive Model Deployment​", accepted at ICML 2026 (Spotlight) Joint work with @sam_hrvth, @stevelaskaridis, @mciccone_AI 🧵1/7
3
4
169
Thrilled for our poster presentation for FlexRank @icmlconf Save the date! 날짜를 기억하다 🗓️ Wed, Jul 8, 2026  •  🕝 2:30–4:15 PM KST  •  📍 Hall A #2208 Full post: nitter.cf/RickZack96/status/2063…
Are curious about how to resize any pretrained model on demand? Then you'll love our paper "FlexRank: Nested Low-Rank Knowledge Decomposition for Adaptive Model Deployment​", accepted at ICML 2026 (Spotlight) Joint work with @sam_hrvth, @stevelaskaridis, @mciccone_AI 🧵1/7
1
3
88
Riccardo Zaccone retweeted
Excited for #ICML2026 in Seoul 🇰🇷 next week! We’ll be presenting two papers on adaptive inference: - FlexRank: arxiv.org/abs/2602.02680 (Spotlight) - MoSE: arxiv.org/abs/2602.06154 We’re also organising the AdaptFM workshop (July 11): adaptfm.gitlab.io/ See you there!
4
12
417
Results, finally! FlexRank exhibits much more graceful degradation as the param budgets reduces. Results on commonsense tasks of lm-eval-harness (LLMs) and on ImageNet-1K (ViTs). More results and ablations in the paper 📑 📄 Project Page: rickzack.github.io/FlexRank/
129
How does layerwise rank reduction translate into real inference savings? Standard SVD adds parameter/FLOP overhead, requiring aggressive rank reduction to yield benefits. We propose a novel factorization that guarantees savings at any budget, without aggressive rank reduction.
1
1
119
🧐Noticed the nestedness constrain above? ‼️This is a key passage: we prove for a 1-NN that violating nestedness degrades the Pareto Front. This serves as insight that the same constraint should be imposed globally on DNNs.
110
We now have layers which can be cut by reducing their rank: how to spend any global budget across layers? While the problem is combinatorial, we can find an approximate Pareto Front in polynomial time. A subset of these submodels is then optimized with KD.
1
1
135
The first step is initializing an elastic rank space without retraining the model from scratch.​ This is achieved by decomposing each pretrained layer via a DataSVD, i.e. an SVD aligned with data directions. ⚠️IMPORTANT: we show that this is good only as initialization
128
Pretrained models are fixed monoliths, but in practice runtime budgets vary beyond the coarsely grained released variants. Ideally, we'd like a family of shared-weight, pareto-optimal models, i.e. models lying on the optimal performance/cost frontier.
1
159
Do you feel FL research is stuck with methods that do not work well in realistic scenarios? 🤔 🫵We got you! Introducing 🚀Generalized Heavy-Ball Momentum (GHBM)🚀, accepted at TMLR: the FL algorithm with both SOTA theoretical guarantees and much better empirical results. 🧵1/9
1
7
4
21
4,100
Tagging relevant people in Optimization, FL and Distributed Training who might be interested in this work @KairouzPeter @gingsmith @konstmish @aaron_defazio @MatharyCharles @Ar_Douillard @niclane7 @sam_hrvth @_arohan_ @samsja19
1
4
207
📢 NEWS: if you wondered if it is possible to just use classical momentum (e.g. FedAvgM) and achieve the results of GHBM, our follow-up work at OPT@NeurIPS2025 provides the answer: arxiv.org/pdf/2511.20168
1
124