Łukasz Stafiniak

Want to understand LLMs? Start with how neural networks store memories. This idea deeply shapes how I think about them, so I'm glad to see @heyyalexwang explaining it this clearly, and to see it getting the attention it deserves. Using this occasion to share some of the recent research I’ve been part of on using this perspective to better understand LLMs: 1. LLMs are extremely good at storing knowledge, but how do they learn to do so? We study this question in arxiv.org/abs/2503.21676, showing that attention circuits need to be created before models can store memories. These attention circuits emerge during training; we study how this depends on data in arxiv.org/abs/2505.17863. 2. The memory setup in the blog, and the standard one more generally, is to have only one possible value, e.g. a next token. In practice, LLMs have to model different possible future tokens; we show in arxiv.org/abs/2608.15448 that such “ambiguity” requires larger networks to store! 3. What happens when a network with a lot of knowledge learns new memories: it catastrophically forgets. This is a nice toy setup to systematically study it and there are some surprising other phenomena that one can study. In arxiv.org/abs/2610.08718 we study spurious forgetting: in some cases there is forgetting in the sense that memories cannot be recalled, but there are still present if probing the network in the right way. 4. One can see post-training as a slight modifications of the memory of the network. We use this perspective in arxiv.org/abs/2610.03529 to give some intuition about how different distillation strategies affect model behavior; this is imo a cool setup to understand some distillation behaviors. Bonus. From the arguments in the quoted blog, one can heuristically derive scaling laws. I gave a tutorial on that (which reviews some of the work linked in the blog) nicolaszucchet.github.io/Tut…; there is even a colab where you can train your own memories and derive their scaling laws! There are still many open questions related to memories, knowledge and LLMs, and I hope to see a lot more of cool research to do in this direction!