Comp Neuro, ML, Dynamical Systems 🧠🤖PhD student @PiN_Harvard @KempnerInst | prev @mcgillu @Mila_Quebec @EPFL | 💻: https://nitter.cf/t.co/BG5c7FkJ3D
Joined November 2020
- Tweets131
- Following451
- Followers447
- Likes599
Pinned Tweet
How much representational diversity exists around a trained network?
A lot! Moving a ViT’s weights by 1.3% while preserving its predictions, its representations end up less similar to the original than a random-init ViT’s ‼️
New paper: Hessian Null Space Continuation (HNC) 💫🧵 (1/12)
Ann_Huang retweeted
New preprint out (first from my new group!) 🚨 "Predictive Grid Cell Coding Does Not Require Explicit Planning" biorxiv.org/content/10.64898… 1/
🧠 In RNNs trained on a memory task, HNC drives the networks to learn drastically different representations and dynamics.
The canonical trained network stores memory as fixed points. These fixed points split, multiply, or disappear during HNC, while the networks maintain 100% task accuracy (5/12)
Project page: ann-huang-0.github.io/Hessia…
Code: github.com/Ann-Huang-0/hessi…
Paper: arxiv.org/abs/2609.38081
w/ my awesome team!🌟🌟 @neurostrow, @HansonZLu_, @wredman4 @Leokoz8 @KanakaRajanPhD (12/12)
May be of interest to you!
@phillip_isola @minyoung_huh @thisismyhat @EkdeepL @janleike @engeltatiana @aran_nayebi @dyamins @ermgrant @ScienceLukas @SussilloDavid
Mode connectivity shows that different trained networks can be connected by low-loss regions in weight space. But do these regions contain networks with different internal computations?
💡HNC reveals many diverse mechanisms within the connected low-loss region around a single network (2/12)
HNC is a scalable method for traversing connected regions of weight space while preserving network function.
We define a function-matching loss and use its Hessian to find flat directions, where the loss barely changes, then step along them to find alternative networks. (3/12)
HNC is a scalable method for traversing connected regions of weight space while preserving network function. We define a function-matching loss and use its Hessian to find flat directions, where the loss barely changes, then step along them to find alternative networks. (3/12)
