🚀 Today we’re releasing FlashOptim: better implementations of Adam, SGD, etc, that compute the same updates but save tons of memory. You can use it right now via `pip install flashoptim`. 🚀 arxiv.org/abs/2602.23349 A bunch of cool ideas make this possible: [1/n]
31
227
26
1,548
224,037
@vikhyatk they also have decoupled weight decay built in.

Mar 4, 2026 · 6:12 AM UTC

1,166