Ali Naeimi (@Ali_NT99) header
Ali Naeimi (@Ali_NT99) profile picture

PhD student at TUM | ProteinLMs | Perfetto enjoyer | Distributed training/inference Optimization

Manchester, UK
Joined July 2023
Photo by Ali Naeimi (@Ali_NT99)
Ali Naeimi retweeted
i'll let you in on something: inference optimization has moved up the abstraction ladder. it's not about handwriting the fastest kernels anymore. it's about how well you can define your optimization context, I/O, constraints, and testing environment for your AI agents.
28
15
3
642
32,458
TLDR: we integrate many computational and statistical improvements for muon-style updates here: github.com/microsoft/dion . Comp: Gram Newton Schulz, Row Selection, Megabatching, Cudagraph Capture, Symmetric Kernels Stat: Joint learning rate+fraction, row norm, fused rounding
Newton-Schulz is what makes Muon work, but it's also a massive pain compared to Adam. *Dion3* makes it ~6x faster and more parallelizable too. Combines our work on row-filtered gradients, Gram-NS alg, and GPU kernels in one package. Try pip install dion! arxiv.org/abs/2608.11612
1
25
152
12,038
Ali Naeimi retweeted
Newton-Schulz is what makes Muon work, but it's also a massive pain compared to Adam. *Dion3* makes it ~6x faster and more parallelizable too. Combines our work on row-filtered gradients, Gram-NS alg, and GPU kernels in one package. Try pip install dion! arxiv.org/abs/2608.11612
11
40
5
363
34,750
If you're on Grace Hopper/Blackwell and not fully utilizing the C2C BW for targetted offloading, you're ngmi...
2
161