Doing cybernetics (without being allowed to call it that). Assistant Professor @BrownUniversity. Part-time @unconvAI. Prev: @IBMResearch @MIT, @RutgersU
Providence, RI
Joined February 2009
- Tweets310
- Following592
- Followers1.4K
- Likes1.4K
See here for a correct way to use embodied fly models 😜 amazing work, congrats to the team!
1/ Animals explore without rewards, tasks, or instructions. What computational objective drives that autonomy?
We trained virtual flies with intrinsic-motivation algorithms to see which best reproduce real fly behavior and brain activity.
Work with @rdkeller and @aran_nayebi !🧵
Leo Kozachkov retweeted
Super excited to share that I'm co-founding Read/Write Labs for #NeuroAI applications to human brain health and human-aligned AI. We could not ask for a more wonderful team: prn.to/4yK9vqo
Leo Kozachkov retweeted
Can a neural network learn ~2^100 LDPC (Low-Density Parity-Check) codewords from just 10-20 examples?
👉 Grokking in LLMs and memorization-to-generalization transition in diffusion models are related phenomena — the model learns true process of data generation from a small number of training samples.
👉 It is hard to study this phenomenon analytically on natural datasets.
👉 LDPC codeword generation is a precise mathematical process in which the codewords are constructed as solutions to parity check equations.
👉 We train a Dense Associative Memory with a tiny fraction of codewords from the full vocabulary, without giving the network the generation algorithm itself. The model manages to learn the entire vocabulary on its own. It turns out, you only need log(n) samples to learn the entire space of 2^n codewords.
👉 The network only needs approximately P=10-20 training examples to correctly learn the full vocabulary containing ~2^100 codewords.
Our new paper with Andy Lucas: arxiv.org/abs/2610.27384.
Let us know what you think!
The valley of neural network solutions is vast and dense. With our new method, Hessian Null Space Continuation (HNC), you can wander through this valley and encounter endless solutions most beautiful.
Some of these solutions are representationally aligned with other models, brains, whathaveyou. Others are not aligned at all.
Contra the Platonic Representation Hypothesis, this suggests that representational alignment is not a necessary consequence of successful task optimization.
This doesn't mean representational alignment is an illusion or "fake". It just means that alignment is not inevitable, and may depend on choices like the optimizer rather than the task-optimization alone.
On a more optimistic note, HNC also lets us seek out solutions with desirable properties (e.g., sparsity, safety) without sacrificing performance.
Huge congrats to @AnnHuang42 for leading this work and doing an amazing job!!
My approach these days is that I will only read a paper if it is personally recommended to me by someone I know and trust.
I def miss some interesting papers, but that's OK :)
Leo Kozachkov retweeted
Good lord. They released several major advances at 3:34 AM on a Saturday morning. The results where I'm familiar enough with the field to read them-- 1, 2, 5, 8-- seem like major big deals. But I have to go to synagogue right now and can't give them the attention they deserve. How frustrating!
yes, nonsofic groups exist: this statement is one of many new beautiful results proved by Astra, our next major model.
We're releasing 10 such Astra proofs, complete with lean certificates and CoT walkthroughs for each of them. The results are wide-ranging, from von Neumann algebras (disproof of Connes' Rigidity Conjecture) to better bounds for high dimensional sphere packing, for circuit complexity, for monochromatic triangles in multicolored graphs, and more.
More thoughts here: openai.com/index/ten-advance…
Leo Kozachkov retweeted
Come join us in Toronto. @UofT is hiring an Associate Professor/Professor and endowed Hinton Chair in Artificial Intelligence.
jobs.utoronto.ca/job/Toronto…
I'm a simple man. @Gradientdinner writes, I read. Congrats Mikail!!
Leo Kozachkov retweeted
1/4 @aran_nayebi and I have created a new mathematical Theory of Contravariance in NeuroAI. There are two high-impact concepts: weak-strong equivalence (WSE) and Zippering (ZIP) that shape how to think about NeuroAI going forward.
Leo Kozachkov retweeted
What do control theory and neuroscience have in common? Shared origins in cybernetics — and, lately, “poor cousin” status next to ML. To reconnect them, we’ve been organizing workshops at this interface, most recently at the Flatiron Institute. Thanks to all who joined!
Leo Kozachkov retweeted
Fixed point iterations for parallelizing nonlinear dynamics is all the rage:
- Newton for RNNs
- Picard for diffusion models
- Jacobi for parallel decode of LLMs
But how do these techniques relate, and when should you use them?
We show you how in our new paper 🧵
Leo Kozachkov retweeted
Ok -- I've started a substack -- The Principle Investigator! My goal is communicating advanced takes on ideas in NeuroAI+. Here's the first substantive post: danyamins.substack.com/p/kol… Calculating the aesthetics of DNN models of the brain
Leo Kozachkov retweeted
This past week several people asked me about how Loop Transformers are related to Energy Transformers.
Energy Transformers are:
👉 looped transformers
👉 energy-based models (EBMs)
👉 Dense Associative Memories — generalized Hopfield nets with superior scaling laws for information storage
That combo is powerful:
👉 looping = iterative refinement = reasoning in the latent space (not in token space)
👉 energy-based = stability of token dynamics
👉 Associative Memory = strong retrieval capabilities
Put together: models that settle into good solutions, not just predict next tokens.
I’ve been especially excited about this class of models for a while. They feel like a promising direction for more stable, interpretable, and memory-rich AI systems.
This week at #ICLR2026 we are presenting NRGPT arxiv.org/abs/2512.16762, which is a:
👉 a looped transformer
👉 a stable Dense Associative Memory
👉 works great on ListOps and real text
Original Energy Transformer paper: arxiv.org/abs/2302.07253 from NeurIPS 2023
Leo Kozachkov retweeted
I think we finally made really significant progress on the biggest unsolved "developmental AI" problem: learning from human-scale data. Key idea: zero-shot world models that support concept extraction via approximate causal inference. amazing collab w/ @khai_loong_aw @mcxfrank
Today's best AI needs orders of magnitude more data than a human child to achieve visual competence.
We introduce the Zero-shot World Model (ZWM), an approach that substantially narrows this gap. Even when trained on the first-person experience of a single child, BabyZWM matches state-of-the-art models on diverse visual-cognitive tasks – with no task-specific training, i.e., zero-shot. 🧵
Dynamical systems theory, energy-based models, and transformers: what could be better? Check out our new #ICLR2026 paper and a great writeup from @DimaKrotov!
GPT is a standard paradigm for language models:
👉 tokens → neural net → shifted tokens
Energy-based models (Hopfield style) is a different paradigm:
👉 tokens evolve on an energy landscape
What if you could have both? In our new #ICLR2026 paper, we build a hybrid:
• Tokens evolve over time (like an energy-based model)
• Dynamics runs on a learned energy landscape
• Then converges to a steady state, which defines the next token
So instead of a single forward pass:
👉 prediction = a dynamical process
Our model, called NRGPT:
• Combines GPT-style inference with EBM dynamics
• Enables iterative refinement of token representations
• Opens the door to more stable architectures
And it’s not just theory:
✅ Strong results on ListOps
✅ Works on real text
Paper: arxiv.org/abs/2512.16762.
What would you build with dynamical tokens?
Leo Kozachkov retweeted
Postdoc opening in APMA @BrownUniversity — specifically looking for candidates bridging APMA + brain science or CS. Review starts April 1! Great fit for anyone interested in collaborating with faculty at the @CarneyInstitute Nancy G. Zimmerman Center for Comput. Brain Science. 🧵
Leo Kozachkov retweeted
If you're at #Cosyne2026, stop by @rdkeller's poster tonight (Poster 1-034) and ask him questions! :)
1/ I'm excited to share recent results from my first collaboration with the amazing @aran_nayebi and @Leokoz8!
We show how autonomous behavior and whole-brain dynamics emerge in embodied agents with intrinsic motivation driven by world models.