Hong-Yi Wang retweeted
What determines the compressibility of a neural network? In our most recent work, we showed that (sufficiently smooth) deep MLPs can be compressed to a size that depends only on the input data dimension and, surprisingly, is independent of the model's width.
arxiv.org/abs/2608.21752
1/7 🧵 Can we make neural networks and datasets radically smaller without losing anything?
Our new ICLR 2026 paper says yes — provably. d variables can be losslessly compressed to O(polylog d), and this is optimal.
📄 Paper: openreview.net/forum?id=vxkz…
6/7 Application 2 — Improving neural scaling laws.
Compress dataset (or width) from d to d′=⌈16d⌉ before training. The scaling exponent roughly doubles in our experiments:
– Dataset: α=0.68→1.27
– Width: α=0.31→0.61
1
31
7/7 Asymptotically, power-law scaling can be boosted all the way to stretched-exponential. Anywhere a permutation symmetry hides, compression follows. We also demo it on transformers.
💻 Code: github.com/WHY-David/moment_…
w/ Di Luo, Tomaso Poggio, Isaac Chuang, @LiuZiyin10 Ziyin
2
48
Hong-Yi Wang retweeted
Excited to share our theory on the universal compressibility of neural nets (led by @HYWangphy)! This leads to a new dynamical lottery ticket hypothesis and implications for improving neural scaling laws. arxiv.org/abs/2510.00504
Check out our ICLR poster!
Hong-Yi Wang retweeted
A new formulation of non-Hermitian band theory is applicable to any number of spatial dimensions, a development useful for the study of physical effects exclusive to open systems.
Read go.aps.org/3TSHPxe
#openaccedd #PRXjustpublished #PRXcondmatt






