
Applied AI @gentrajectory. Prev researcher @nasa, @caltech, @uchicago
San Francisco, CA
Joined February 2023
- Tweets58
- Following792
- Followers904
- Likes2.3K
Videogame decompilation is making bullish on RL sims for robotics. Like ultra high fidelity physics, contact dynamics, assets
4
198
Joshua retweeted
Today, the promises of transhumanism and posthumanism are multiplying. These are ideological currents according to which, sooner or later, technology will eliminate all disease and pain, as well as aging and even death itself. Precisely because scientific progress is a supreme achievement of human intelligence, it must not be confused with a privilege reserved for millionaires seeking immortality, while entire peoples suffer from poverty and oppression. In the face of this mirage, which promotes the illusion of human beings perfected by cybernetics and biotechnology, Christian anthropology reminds us that a limitation is not a manufacturing defect, but rather the very condition of human grandeur. #ApostolicJourney #Peru #MagnificaHumanitas vatican.va/content/leo-xiv/e…
Cerebellum containing ~80% of all neurons in the brain is kinda a blackpill for robot learning, no?
2
3
299
Joshua retweeted
RLVR isn't just for math and code. Engineers spent decades building physics-based verifiers for electromagnetic systems. @gentrajectory is using these and Tinker to train models that design power transformers to meet specs at low cost in the real world.
AI is now improving its own model architectures and chips. The power infrastructure it runs on deserves the same attention.
We used RL to teach a Kimi base model to design medium-power transformers. It met 93% of unseen specs, helping to compress multi-week engineering efforts into a few minutes of inference.
1/5
Welcome to the singularity
We’re sharing a solution to the Navier-Stokes Millennium Prize Problem, one of the deepest problems at the frontier of mathematics.
The proof was produced by a group of agents, using an OpenAI next-generation model significantly more capable than GPT-6 Astra.
The problem concerns whether the description of smooth three-dimensional fluid motion modeled by the Navier-Stokes equations can break down. It has remained unresolved for roughly 90 years.
6
646
Joshua retweeted
Kimi K2.6 btw, not K3
Relatively weak models can go very far with niche RL
AI is now improving its own model architectures and chips. The power infrastructure it runs on deserves the same attention.
We used RL to teach a Kimi base model to design medium-power transformers. It met 93% of unseen specs, helping to compress multi-week engineering efforts into a few minutes of inference.
1/5
RL for physical engineering!
AI is now improving its own model architectures and chips. The power infrastructure it runs on deserves the same attention.
We used RL to teach a Kimi base model to design medium-power transformers. It met 93% of unseen specs, helping to compress multi-week engineering efforts into a few minutes of inference.
1/5
Check out NitroGen, recognized with a CVPR 2026 Best Paper Honorable Mention.
Grateful to have contributed to this work alongside Jim Fan, Guanzhi Wang, Loïc Magne, and the entire team.
NitroGen just won CVPR Best Paper Honorable Mention!! We are making strides towards general-purpose embodied agents that master not only the real world physics, but also all possible physics across a multiverse of simulations.
It’s been 4 years since MineDojo, our first embodied agent in Minecraft, won NeurIPS Best Paper. Congrats to everyone on the team!!
Joshua retweeted
The super important thing I haven’t seen mentioned yet as upshot of this:
It’s not just that people won’t HAVE to write code anymore, ITS THAT LITERALLY IT WILL BE UNSAFE TO DO SO
Introducing Project Glasswing: an urgent initiative to help secure the world’s most critical software.
It’s powered by our newest frontier model, Claude Mythos Preview, which can find software vulnerabilities better than all but the most skilled humans.
anthropic.com/glasswing
Joshua retweeted
General Trajectory launches Phoenix: AI drone interceptors
This quoted post is unavailable.
That which can be lifted into latent space will fall all the same
Three days ago I left autoresearch tuning nanochat for ~2 days on depth=12 model. It found ~20 changes that improved the validation loss. I tested these changes yesterday and all of them were additive and transferred to larger (depth=24) models. Stacking up all of these changes, today I measured that the leaderboard's "Time to GPT-2" drops from 2.02 hours to 1.80 hours (~11% improvement), this will be the new leaderboard entry. So yes, these are real improvements and they make an actual difference. I am mildly surprised that my very first naive attempt already worked this well on top of what I thought was already a fairly manually well-tuned project.
This is a first for me because I am very used to doing the iterative optimization of neural network training manually. You come up with ideas, you implement them, you check if they work (better validation loss), you come up with new ideas based on that, you read some papers for inspiration, etc etc. This is the bread and butter of what I do daily for 2 decades. Seeing the agent do this entire workflow end-to-end and all by itself as it worked through approx. 700 changes autonomously is wild. It really looked at the sequence of results of experiments and used that to plan the next ones. It's not novel, ground-breaking "research" (yet), but all the adjustments are "real", I didn't find them manually previously, and they stack up and actually improved nanochat. Among the bigger things e.g.:
- It noticed an oversight that my parameterless QKnorm didn't have a scaler multiplier attached, so my attention was too diffuse. The agent found multipliers to sharpen it, pointing to future work.
- It found that the Value Embeddings really like regularization and I wasn't applying any (oops).
- It found that my banded attention was too conservative (i forgot to tune it).
- It found that AdamW betas were all messed up.
- It tuned the weight decay schedule.
- It tuned the network initialization.
This is on top of all the tuning I've already done over a good amount of time. The exact commit is here, from this "round 1" of autoresearch. I am going to kick off "round 2", and in parallel I am looking at how multiple agents can collaborate to unlock parallelism.
github.com/karpathy/nanochat…
All LLM frontier labs will do this. It's the final boss battle. It's a lot more complex at scale of course - you don't just have a single train. py file to tune. But doing it is "just engineering" and it's going to work. You spin up a swarm of agents, you have them collaborate to tune smaller models, you promote the most promising ideas to increasingly larger scales, and humans (optionally) contribute on the edges.
And more generally, *any* metric you care about that is reasonably efficient to evaluate (or that has more efficient proxy metrics such as training a smaller network) can be autoresearched by an agent swarm. It's worth thinking about whether your problem falls into this bucket too.
2
542
























