
PhD Candidate in Computer Eng. (distr. learning, efficiency) @PoliTONews | Alumno IEEE-HKN @HKNPoliTo
Turin, Italy
Joined August 2022
- Tweets52
- Following81
- Followers81
- Likes89
Pinned Tweet
Are curious about how to resize any pretrained model on demand?
Then you'll love our paper "FlexRank: Nested Low-Rank Knowledge Decomposition for Adaptive Model Deployment", accepted at ICML 2026 (Spotlight)
Joint work with @sam_hrvth, @stevelaskaridis, @mciccone_AI
🧵1/7
I'd add: (make your agent) use design principles. They're no less useful than when used by humans, and a machine can have complete access to pros/cons of everything we understood about soft. eng.
Less code alone only works for naturally small projects
Reposting this because it's becoming more useful every day
Less code means cheaper agents, shorter context, less context rot, and fewer bugs
2
104
At @icmlconf, the most common question about FlexRank was: “Can it really create smaller versions of an LLM?”
In practice: “Does this scale if I invest in making it production-ready?”
So we ran more experiments to answer that.
Our findings 🧵 1/3
Replying to @RickZack96 @icmlconf
You may notice a small gap at full budget (parameter count = 1.0).
Is something wrong with optimization?
Not quite. Distilling from the base model (grey dashed line) works surprisingly well: up to 80% of the parameters, models achieve comparable or even lower loss.
60
Replying to @RickZack96 @icmlconf
The results are encouraging: simply training longer gives a large boost in downstream performance.
No new data. No tricks. Just more training.
48
In our paper, we prioritized broad applicability over convergence in a single setting.
But what if we just scale?
We extended Llama 3.2-1B from 5B → 60B tokens, with the same budget as LlamaFlex, still on FineWeb.
56
Riccardo Zaccone retweeted
Second Protocol Learning Workshop at ICML is a wrap.
A packed day of talks and posters covering large scale distributed training and open source AI.
A movement is growing.
Riccardo Zaccone retweeted
✨ [SPOTLIGHT] FlexRank: Nested Low-Rank Knowledge Decomposition for Adaptive Model Deployment
How to make any pretrained model ready for elastic deployment: turn it into a nested, low-rank structure and resize it for different inference budgets.
Led by @RickZack96, with @sam_hrvth &
@stevelaskaridis
📌Poster: Wed Jul 8 · 2:30–4:15 PM KST · Hall A #2208
🧵Full thread: nitter.cf/RickZack96/status/2063…
Are curious about how to resize any pretrained model on demand?
Then you'll love our paper "FlexRank: Nested Low-Rank Knowledge Decomposition for Adaptive Model Deployment", accepted at ICML 2026 (Spotlight)
Joint work with @sam_hrvth, @stevelaskaridis, @mciccone_AI
🧵1/7
Thrilled for our poster presentation for FlexRank @icmlconf
Save the date!
날짜를 기억하다
🗓️ Wed, Jul 8, 2026 • 🕝 2:30–4:15 PM KST • 📍 Hall A #2208
Full post: nitter.cf/RickZack96/status/2063…
Are curious about how to resize any pretrained model on demand?
Then you'll love our paper "FlexRank: Nested Low-Rank Knowledge Decomposition for Adaptive Model Deployment", accepted at ICML 2026 (Spotlight)
Joint work with @sam_hrvth, @stevelaskaridis, @mciccone_AI
🧵1/7
Riccardo Zaccone retweeted
Excited for #ICML2026 in Seoul 🇰🇷 next week!
We’ll be presenting two papers on adaptive inference:
- FlexRank: arxiv.org/abs/2602.02680 (Spotlight)
- MoSE: arxiv.org/abs/2602.06154
We’re also organising the AdaptFM workshop (July 11):
adaptfm.gitlab.io/
See you there!
Results, finally!
FlexRank exhibits much more graceful degradation as the param budgets reduces.
Results on commonsense tasks of lm-eval-harness (LLMs) and on ImageNet-1K (ViTs).
More results and ablations in the paper 📑
📄 Project Page: rickzack.github.io/FlexRank/
129
How does layerwise rank reduction translate into real inference savings?
Standard SVD adds parameter/FLOP overhead, requiring aggressive rank reduction to yield benefits.
We propose a novel factorization that guarantees savings at any budget, without aggressive rank reduction.
🧐Noticed the nestedness constrain above?
‼️This is a key passage: we prove for a 1-NN that violating nestedness degrades the Pareto Front.
This serves as insight that the same constraint should be imposed globally on DNNs.
110
We now have layers which can be cut by reducing their rank: how to spend any global budget across layers?
While the problem is combinatorial, we can find an approximate Pareto Front in polynomial time.
A subset of these submodels is then optimized with KD.
The first step is initializing an elastic rank space without retraining the model from scratch.
This is achieved by decomposing each pretrained layer via a DataSVD, i.e. an SVD aligned with data directions.
⚠️IMPORTANT: we show that this is good only as initialization
128
Pretrained models are fixed monoliths, but in practice runtime budgets vary beyond the coarsely grained released variants.
Ideally, we'd like a family of shared-weight, pareto-optimal models, i.e. models lying on the optimal performance/cost frontier.
1
159
Do you feel FL research is stuck with methods that do not work well in realistic scenarios? 🤔
🫵We got you!
Introducing 🚀Generalized Heavy-Ball Momentum (GHBM)🚀, accepted at TMLR:
the FL algorithm with both SOTA theoretical guarantees and much better empirical results.
🧵1/9
Tagging relevant people in Optimization, FL and Distributed Training who might be interested in this work @KairouzPeter @gingsmith @konstmish @aaron_defazio @MatharyCharles @Ar_Douillard @niclane7 @sam_hrvth @_arohan_ @samsja19
1
4
207
📢 NEWS: if you wondered if it is possible to just use classical momentum (e.g. FedAvgM) and achieve the results of GHBM, our follow-up work at OPT@NeurIPS2025 provides the answer:
arxiv.org/pdf/2511.20168
1
124



















