Johan Puisais 🇺🇦 🇪🇺 🇫🇷

Yann LeCun was right & nobody in this industry wants to say it out loud. 109 million parameters beat 72 billion. He has spent years saying scale is not the thing that makes a model understand the world. Meta bet the other way. He left in November and started his own lab in Paris. Last Thursday he published the proof. The theory everyone has been running on since 2024 is that if you train models big enough, on text, on images, on sound, they all end up with the same picture of reality inside them. Size does it. Nobody ever proved that part. Randall Balestriero and LeCun proved it. And the proof says size was never doing the work. What does the work is which two things you show a model together. A photo and the caption of that photo are two views of one thing. Models trained on pairs like that get there fast. Two paragraphs pulled from the same document are not two views of one thing. Models trained on those get there slowly and expensively. Next word prediction is the second kind. That is what every chatbot in the world runs on. In their tests the first method reached the same understanding after 21.6 million examples. Next word prediction needed 900 million. And a 109 million parameter text model ended up closer to an image model's view of the world than a 72 billion parameter language model was. 660 times smaller.