BRAInLab (@Lab4BRAIn) profile picture

Basic Research of AI: Optimization in ML, Federated Learning, GenAI, LLM, Maths for AI, AI for Science Head: DrSc Aleksandr Beznosikov [email protected]

Joined September 2025
Photo by BRAInLab (@Lab4BRAIn)Photo by BRAInLab (@Lab4BRAIn)Photo by BRAInLab (@Lab4BRAIn)Photo by BRAInLab (@Lab4BRAIn)Photo by BRAInLab (@Lab4BRAIn)Photo by BRAInLab (@Lab4BRAIn)Photo by BRAInLab (@Lab4BRAIn)Photo by BRAInLab (@Lab4BRAIn)Photo by BRAInLab (@Lab4BRAIn)Photo by BRAInLab (@Lab4BRAIn)
Training neural networks for tabular regression? Our latest work may be of interest And it's not only about which of the 18! regression objectives is best, but touches on the role of noise and global early stopping in tabular DL in general arxiv.org/abs/2610.13909 Thread (1/N)
2
5
4
58
7,228
1/10 Muon and Shampoo make optimizer updates matrix-aware. What about weight decay? Our Spectral Weight Decay encourages low-rank weights during training, improving compression and resistance to label noise. We'll present it at NeurIPS OPT 2026. arxiv.org/abs/2610.11730
1
2
1
278
1) Muon works because it updates a matrix as a matrix. LoRA splits that matrix into A and B, then most optimizers update the factors independently. The induced step on BA can point somewhere quite different. We built LoRA-TSD to put Muon's geometry back where it belongs.
1
2
2
3
714
馃У Accepted at UAI 2026, @UncertaintyInAI . Distributed training is bottlenecked by communication, so everyone compresses gradients: RandK, QSGD, PermK. But almost every compressor is memory-less. It redraws coordinates from scratch every step, as if history did not exist.
2
1
3
133
1) Happy to share our paper is out at @RealAAAI 2025! Stochastic optimization assumes i.i.d. noise. In RL and distributed training it is not: the noise is correlated across steps. That Markovian case had stayed Euclidean. We bring geometry, acceleration, and optimality. 馃У
1
1
4
151
Learning-rate warm-up sits in nearly every training recipe, from ResNets to LLMs, and almost nobody can say why it has to be there. Turns out it isn't a heuristic at all. Warm-up followed by decay falls straight out of the convergence proof, and the right length picks itself. 馃У
2
1
4
175