Somesh Misra / ERP.ai

My ranking of foundational work in LLMs: 1. Mixture of Experts 2. Grouped Query Attention 3. Multi head Latent Attention 4. Selective State Space Models 5. Diffusion Language Models 6. TestTime Training Layers 7. Titans Neural Memory 8. Mixture of Depths

My ranking of foundational work in LLMs: 1. Mixture of Experts 2. Grouped Query Attention 3. Multi head Latent Attention 4. Selective State Space Models 5. Diffusion Language Models 6. TestTime Training Layers 7. Titans Neural Memory 8. Mixture of Depths

I think it's becoming rather clear that ALL of mathematics will become possible to formalize rather soon. There are deep implications: 1. Mathematics becomes super interactive. Formalizations make it easy to browse, recombine, and build mathematical proofs from components. 2. Each new idea and paper can be checked much faster. The deep part is meaning and explanation. Validation is almost automatic. 3. Optimization and identification of new proofs can be done partially via agents, and humans can contribute much faster with great new ideas. Proofs from the Book are possible at scale. 4. Scientists and people at large can finally use mathematical artifacts as black boxes with trust and confidence. We can commoditize mathematical theorems. I think the mathematical enterprise also becomes very gamified. We can clearly see each unproven statement as a concrete goal, identify existing tools and directions, and focus quickly on what's missing. And everyone can contribute, with agents or through understanding. Soon, new languages and new theories will also develop, but the structure will keep growing steadily, with no mistakes and on fundamentally solid foundations. Hilbert's dream is becoming a reality. One important caveat: mathematics is by no means trivial now! It's just becoming extremely well-structured and laser-focused. I think the jump in capacity, quality, and scale will be massive. I also hope that our understanding of the big picture will remain solid, because there will be a lot to grasp.

interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works already-clean.goodfire.com/

For what its worth, for Gap-3-SAT, we can shave off a log log n factor in the exponent (based on what Astra tells me)..So Gap-3-SAT can be solved in 2^{n/\sqrt{n}} time. Its the first time there is an algorithmic separation between 3SAT and Gap-3-SAT.

I am back with a few thoughts. I stopped reading all the trauma logs of mathematicians who recently got impacted by AI. Instead I switched to building a corpus of knowledge for why and how AI is interacting with mathematics. I teach this term a new course here at Adam Mickiewicz University. It's title is "Computer assisted solutions to mathematical problems". I proposed it because I realized in the last 6 months something important. A. It makes no sense to grieve the "good old ways". We should study to the bone how mathematics and computers interact. There are three levels: numeric, symbolic and formal. B. The only way to prepare yourself for the rise of the technology is to understand it well. You cannot understand the progress of AI in mathematics if you don't know how the models are trained on the symbolic representation of mathematical proofs, how the training goes, why specific transformer architectures, reinforcement learning and basically a huge amount of incredible insights gives in practice some new value in mathematics. C. It is incredibly fun to train your own models (yes I use generative models to help me build training pipelines too, otherwise I would not have enough lifetimes to do so). D. The AIxMaths space is so incredibly complex - you have these weird, scalable token generating algorithms, recently in agentic form with memory, interactions etc. This is a field of study - we need to understand how it works. On the other hand, mathematics as a field of science is incredibly complex, people do very different kinds of axiomatic theories with their intuitions, tactics and strategies. This was never studied too seriously as a whole. We need to study it deeply. So I am spending this term gearing up my students and myself for whatever is coming. We don't have any assumptions. I don't claim that models will become better at maths than all of human mathematicians. I don't think it's a good question. We just want to stay in the loop with the developments and see whether we can contribute to the change and see if this can be useful and productive. I am looking forward to your feedback. The current plan contains the following topics: - basic distinction between numerical computation, symbolic calculations and formal proofs - basic understanding of the IEEE 754 standard and things around numerical computations - how to build your own computer algebra system in Python and why it's so tricky - why and how proof-checking software is important; we start with the basic principles of such designes, I plan to talk first about the foundations going back to Automath and the foundational work of de Bruijn. - practical examples spanning real use cases of computer algebra, numerical integration and proofs in Peano arithmetic and how to read languages like Lean - in the end I want to present some little post-trained models like Qwen which I taught how to do Peano arithmetic better and how they prove statements about this theory; there will be AlphaGeometry too and maybe a bit more about Mizar, Metamath and Lean. Just to give students some scope and perspective. I plan to continue working on this course into a full book. I think I won't stop until I understand deeply why language models are currently so good at maths. It's something of utmost importance. It's important to see where this might stop and what lies beyond. I personally think these kind of questions will be the most important problems to answer by mathematicians themselves. We shouldn't stop until we find good answers to all of this. Reading AI slop is perhaphs a bit useful, but deep understanding of why it sometimes makes sense is even more challenging.