
ML efficiency PhD @ELLISInst_Tue & @MPI_IS before @uni_tue & @unibarcelona
Joined November 2024
- Tweets23
- Following861
- Followers94
- Likes149
🚨 Quantization robustness isn’t just post-hoc engineering; it’s a training-time property.
Our new paper studies the role of training dynamics in quantization robustness.
More in 🧵👇
7️⃣ 🧰 Practical takeaways for model builders:
Maintain higher learning rates longer when possible
Prefer higher weight decay (λ) if performance is equal
Use weight averaging to counteract brittleness
Study quantization during pretraining, not only after
1
6
580
8️⃣ 🌍 Big picture:
Quantization degradation is not inevitable as datasets grow. Similar to recent results (@AshkboosSaleh), it can be mitigated through smarter optimization.
Arxiv: arxiv.org/abs/2510.06213
Team: @n_ajroldi @jonasgeiping
Good: afternoon 👋

