CEO @fastcodeai | Building @billyfscd | Dad
Bangalore, Karnataka, India
Joined March 2012
- Tweets962
- Following425
- Followers5.8K
- Likes2.2K
Billy explained in 46 seconds what took us six months to build.
Slightly humbling. nitter.cf/billyfscd/status/21176…
For my whole PhD, the case against neural nets was simple: nobody can read the weights.
Last night, one model read another’s weights and edited 0.016% of them. No gradients, no training. Math went from 7.9% to 48.1%.
The black box finally has a reader.
It isn’t us.
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math.
It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training.
We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵
Blog: rikiyat.github.io/blog/weigh…
Update: Hugging Face reproduced it within a day, using an open-source research agent. Model and demo are public.
So the reader is real.
Still not us. nitter.cf/NielsRogge/status/2117…
We @huggingface reproduced this paper!!
It's built with ML Intern, an open-source harness to do AI research, train models, and more
Model: huggingface.co/ML-Intern-lab…
Demo: huggingface.co/spaces/ML-Int…
Humans are 1,000,000x more sample efficient than LLMs.
The new fix: a model 7,000x bigger than Chinchilla.
I trained in Vapnik’s school. Capacity was the enemy.
He spent decades telling us to count the parameters.
We finally listened. We counted up. nitter.cf/industriaalist/status/…
Humans are ~1,000,000x more sample efficient than LLMs, and there are no good explanations.
We’ve found that a big part of the answer is that LLMs are *massively* underparameterized by many orders of magnitude. The brain has 100T synapses learning from ~200M tokens of language. Chinchilla prescribes a 1T-parameter model on 20T tokens. That’s 7 OOMs apart in parameters per token.
After a lot of painstaking work, we present new data-constrained scaling laws and learning algorithms to scale to (extrapolated) quadrillion-parameter transformers.
w/ @akshayvegesna, @EthanBaron78553, @asatheesh6561
Full paper led by @EthanBaron78553, with @ShmuelBerman, @andrewgwils
My PhD school had one rule: never trust a model you can’t bound.
Then I built an AI that lives in my company’s Slack.
It reads every channel I’m in. Sits in my meetings. Drafts my emails.
What lets me sleep isn’t how smart it is.
It’s that it can’t send anything without me.
I spent years proving bounds on functions.
Turns out the bound that matters is a button.
The most important line here is “No ML background is required.”
For 8 years I’ve hired software engineers and trained them into ML engineers.
Teaching them ML took months.
I’ve never figured out how to teach someone to engineer.
Applications are now open for the Anthropic Residency, a paid, fixed-term research program for academics, technical founders, and staff-level engineers who want to make AI safe and beneficial for everyone. No ML background is required.
Apply by November 22: alignment.anthropic.com/2026…
The caveats are in their own blog, and they’re good: it needed 50 forward passes to get there, and on Gemma 3 270M it barely moved anything. Editing seems to unlock what’s already in the weights, not add what isn’t.
Someone under my LeCun post asked: did NYU change what I count as evidence, or just who I trust?
I said I didn’t know. I’ve thought about it since.
Evidence.
In Germany, evidence was a bound.
At NYU, it was the test set. Even when every bound I knew had nothing to say.
My first paper with Yann was a hedge: a ConvNet trained jointly with a graphical model. Half his world, half mine.
arxiv.org/abs/1406.2984
My first paper with Yann was a hedge: a ConvNet trained jointly with a graphical model. Half his world, half mine.
arxiv.org/abs/1406.2984
I learned machine learning from Vapnik’s academic grandson.
Vapnik → Schölkopf → Hein → me.
So I was raised on his side of this bet: by 2005, no one in their right mind would use neural nets.
The witness was Yann LeCun.
Years later, I went to NYU and worked with the witness.
Best part is the fine print on the other bet: if Vapnik worked out the theory himself, he’d still win.
Only a theorist hedges a bet on himself.
Part 1 of this story: nitter.cf/Arjunjain/status/21172…
I was trained to think Yann LeCun was wrong.
My PhD was in Vapnik’s school. Kernels. Bounds. Neural nets were alchemy.
Then I went to NYU, and he changed my mind.
Recently, he did the one thing my old school always asked for.
He brought a proof.
LLMs accumulate a lot of declarative knowledge.
That's how they can answer a large number of questions.
That's also why they require so many parameters and hence so much memory.
But one can replace the accumulation of knowledge by *understanding*
That what JEPA world models do.
That's one reason they can be useful with considerably less memory and training data than LLMs.
This paper merely shows that the underlying similarity structure of the data, whether text or image (e.g. co-occurence of words in text or objects in images) uniquely determines the shape of the representation point cloud, when the encoder is trained trained with a common anti-collapse criterion like SIGReg or VCReg (not by reconstruction).
The paper: Proving and Improving the Platonic Representation Hypothesis with JEPAs, Balestriero & LeCun
arxiv.org/abs/2610.39302
I learned nothing this week.
@karpathy had Claude read 43M Hacker News comments in a weekend.
This week my agents read every NeurIPS paper. 9,130 of them. Then 19,273 robotics papers. Then 20 years of SIGGRAPH.
I read none.
I tell my students: if you didn’t suffer, you didn’t learn.
My agents suffered. I didn’t.
In 2010, the Max Planck Institute for Informatics had more SIGGRAPH Asia papers than all of China. 4 to 3.
This year, Max Planck still has 4. China has 126.
Since my agents had nothing to do, I got them to go back and study 20 years of SIGGRAPH.
Some context first. In 2010, I had a paper at SIGGRAPH Asia. I was a PhD student at the Max Planck Institute for Informatics in Saarbrücken, Germany. That year, my institute tied for the most papers in the world there.
America led 23 papers. China led 3. India led 0.
And since then:
In 2006, America led 69% of SIGGRAPH papers. China led 5%.
In 2026, China leads 37%. America leads 22%.
China passed America in 2024.
India had 15 papers in the last 20 years with a first or last author in India. Only 8 had both. All 8 came in 2019 or later. Rajiv Soundararajan's lab at IISc wrote half of them. 👏🫡
The good news: SIGGRAPH is still small. It took 309 papers this year. NeurIPS took 9,130.
And it still rewards a new idea, not a big lab. India doesn't need 126 papers. It needs ten more labs like Rajiv's.