
Working on bringing local AI to every phone and laptop. Previously @Meta
Manhattan
Joined August 2026
- Tweets315
- Following356
- Followers581
- Likes1.8K
Pinned Tweet
Local AI caught up way faster than anyone realizes. Sharing my thoughts having spent time in ML at Meta and dedicating the last year to getting smart on local AI.
1/ Barely anyone, not even hardcore AI power users, realizes how fast local models have caught up to the frontier. Qwen3.8 woke a few people up, but the most technical people are so frontier-pilled they've blinded themselves to the breakneck progress.
Meanwhile, normal users don't care about frontier intelligence at all. They want AI that gets the job done, but they have no idea what a local model is or how to run one.
So local is stuck in a weird spot: the people who can run these models don't care to do so, and the people who'd benefit most don't know how to run local.
The paradigm of compute spend will break down completely when this revelation finally reaches the masses.
This is f*cking insane: OpenAI co-founder Andrej Karpathy says a tiny 1B model can beat most of today's giant LLMs.
"A 1B model can be much more effective than most current LLMs. 90% of current models waste a ton of capacity and money on remembering stuff that doesn't matter"
In 64 seconds he explains why intelligence lives in the system around the model, not in its size.
Memorized Internet → Cognitive Core → Tools → Feedback Loop → Agent That Works
Save this before it gets lost in X's algorithms.
Everything below is free:
Neural Networks: Zero to Hero
Karpathy's own course, from scratch
karpathy.ai/zero-to-hero.htm…
nanochat
Karpathy's "best ChatGPT that $100 can buy"
github.com/karpathy/nanochat
Effective Harnesses for Long-Running Agents
The system around the model, by Anthropic
anthropic.com/engineering/ef…
Effective Context Engineering for AI Agents
What to feed the model and what to cut
anthropic.com/engineering/ef…
All of this costs $0
99% will keep waiting for a bigger model.
The 1% who build better systems will win with a smaller one.
Bookmark it and start with the first link tonight
Michael Waldman retweeted
AI in trading is a race along a speed–accuracy Pareto frontier, and firms should try to push that frontier outward at every point.
While you’re thinking, others are trading, so a crude deterministic computation can capture edge before prices adjust as long as it is merely directionally correct AND it is the FASTEST.
Trades which come after (based on the same data) are adversely selected in that they happen only after someone who is faster has already traded a price to what they think is fair.
Slower inference can still be useful but has to earn its latency in that it needs to find additional INFORMATION based on the same DATA to compensate for the opportunities that disappeared during that time.
The race to be first to market with new information happens at every level of inference complexity.
In principle, each point on that Pareto frontier offers a potential edge in which marginal additional intelligence and model complexity contains information the market has yet to priced in.
That suggests firms should invest across the entire speed-accuracy frontier with:
(1) Extremely fast deterministic engines for immediate reactions. (2) Fast local inference for decisions that require some interpretation. (3) Large models on well-connected datacenter clusters for deeper reasoning.
These are complementary capabilities where the fast system acts on the first-order signal while slower system looks for missed implications or mistaken interpretations.
1/ A large hedge fund asked me in to talk about open and local models. The conversation made me realize institutional capital allocation will rapidly change over the next year.
They're following the frontier closely, but wanted to discuss how open and local will affect how capital gets distributed across AI spend.
Another benefit of the shift to open/local that doesn’t get touted enough.
Instead of jobs / capital concentrating to a couple hundred employees at closed labs, it will actually distribute to thousands of IT departments.
Michael Waldman retweeted
✅ BENCHMARKED: Qwen3.8-27B-pi-adapted 🤗 weights on my 64GB M5 Pro Mac mini.
After merging the adapter there is no speed penalty!
I also benchmarked Qwen3.8-27B Q8_0 (llama.cpp, thinking mode disabled) with and without the LoRA (median of 3):
- Prompt processing speed (4K context): 380 (base) vs. 217 (LoRA) vs. 388 tok/s (merged)
- Generation speed (4K context): 9.82 (base) vs. 9.42 (LoRA) vs. 9.97 tok/s (merged)
- Time to first token (32K context): 105 (base) vs. 170 (LoRA) vs. 106 s (merged)
The merged model (28.6 GB; 37.3 GB peak memory usage) fits entirely in the memory of my Mac mini and is on par with the base model in terms of inference speed. Without merging, applying the LoRA seems to come at a cost of ~40% in prompt processing speed.
I also had the base and pi-adapted model solve 12 small Python-related tasks in the pi coding agent (2 times each): Both models solved 22/24 tasks, with the pi-adapted model having a slight edge in terms of solving time (313 s vs. 334 s avg. time per task).
~5.5 hours of results from overnight benchmarks, all on a single Mac mini!
Fine-tune open models for your use-case, merge the adapters and enjoy full-speed inference on your desktop.
Introducing Qwen3.8-27B-pi-adapted 🤗
We adapted Qwen's model specifically for the pi coding agent using reinforcement learning and a LoRA adapter.
The results, measured as pass@1:
🚀 SWE-bench Pro (Hard 51): 29.4% → 62.7%
💻 Terminal-Bench 2.1: 39.5% → 55.8%
⚡ 55–62% fewer tool calls per task
This is the power of open models: you can adapt them to YOUR tools and YOUR use cases. Let’s build! 💛
Open weights: huggingface.co/harness-hub/Q…
Methodology: huggingface.co/blog/harness-…
Awesome release. The coolest part here is getting some huge gains was actually relatively inexpensive.
Post training is orders of magnitude cheaper than pre training yet covers the needs almost any org beyond a lab would have
Introducing Qwen3.8-27B-pi-adapted 🤗
We adapted Qwen's model specifically for the pi coding agent using reinforcement learning and a LoRA adapter.
The results, measured as pass@1:
🚀 SWE-bench Pro (Hard 51): 29.4% → 62.7%
💻 Terminal-Bench 2.1: 39.5% → 55.8%
⚡ 55–62% fewer tool calls per task
This is the power of open models: you can adapt them to YOUR tools and YOUR use cases. Let’s build! 💛
Open weights: huggingface.co/harness-hub/Q…
Methodology: huggingface.co/blog/harness-…
got some wild DMs in response to this, so many finance people want an "AI guy" to inform their investments.
I feel like an Instagram model right now with some offers to get flown out.
1/ A large hedge fund asked me in to talk about open and local models. The conversation made me realize institutional capital allocation will rapidly change over the next year.
They're following the frontier closely, but wanted to discuss how open and local will affect how capital gets distributed across AI spend.
1
11
1,262
one of the first advertised use cases of Astra 6.2 is….. reading email 😭
OpenAI is so lost.
Some of you have been asking very politely. Some of you have just been asking.
GPT-6.2 Astra and Sol are now available.
Super efficient and built to perform where it counts—the real world.
openai.com/index/introducing…
2
8
491
1/ A large hedge fund asked me in to talk about open and local models. The conversation made me realize institutional capital allocation will rapidly change over the next year.
They're following the frontier closely, but wanted to discuss how open and local will affect how capital gets distributed across AI spend.
3/ One example that landed: for some funds, trading decisions are not compute-bound, rather network bound, meaning the roundtrip time of packets is a dominant factor.
This is where local models get really interesting for trading as they can answer quicker than any closed model, and that's a matter of physics, not a problem OpenAI can solve.
1
22
1,746
4/
this hedge fund is overall pretty ahead on AI - so I expect this new understanding washing over large AI capital allocators to continue spreading, and show up in how they're pricing closed labs.
With some other capital allocators I've spoken with, I've been genuinely mind blown in how unread they are on how open models are progressing.
Michael Waldman retweeted
Over the past week, I’ve been overwhelmed by the love and support of friends, industry colleagues, and people I've never met. Thank you to everyone who has shown kindness to my family and me. It means the world to me.
I came to the United States 53 years ago, and becoming an American citizen is one of the greatest gifts and proudest moments of my life. NVIDIA has been my life’s work. Building it into the company it is today alongside my colleagues has been a privilege and an honor. Nowhere but in America could a company like NVIDIA be possible. I love our country, and I love NVIDIA.
I did not live up to the standards expected of everyone at NVIDIA, and I have made serious mistakes in my personal life. I created problems for the Board that I deeply regret. I own my mistakes completely, and I am committed to cooperating with the Board’s independent investigation.
Since my departure, I’ve seen an outpouring of calls from NVIDIA’s employees, investors, business partners, and customers for me to return as CEO. Many continue to reach out to me directly as the company navigates complex technology, market, competitive, and geopolitical dynamics. NVIDIA is a once-in-a-generation company. If the Board of Directors decides that my return is in the best interest of our shareholders, our employees, the company, and America, I would be honored to answer these calls and return as CEO.
Thank you again to everyone who has shared their good wishes. I am profoundly humbled by your support. To my family: I love you, and I will spend the rest of my life being worthy of yours.
Jensen
"you have the ability to revoke access"
God forbid Dario can't revoke people's access to intelligence. Protect his right to do so at all costs
Dario Amodei on open source models:
“If the path continues, I think we could get to a very dangerous place.”
“When you control a model and you're deploying it, you have the ability to monitor its usage.”
“You can revoke a user's access. You can change what the model is willing to do.”
“When a model is released in an uncontrolled manner, there's no ability to do that. It's entirely out of your hands.”
“We should confront the advocates of this with that problem and challenge them to solve it.”
He clearly doesn't believe Intelligence should be in the hand of the masses and Sam Altman also saying "RAM" shouldn't be accessible to everyone indefinitely agreed with him.
1
6
332
It's hilarious how many X "agencies" exist whose entire business model depends on price gouging VC backed founders.
There's a reason most will ask "Have you raised yet" as their first question.
5
285
Events in the last few days have woken people up to the need to have AI that works even if the grid doesn’t.
Loci is the easiest way to own your intelligence.
Send it to your family so that they have AI that works offline.
Alex is right - everyone should have an offline AI for emergencies. Until recently, they have all crashed or recommended models that don’t work. That changes with Loci.
Be prepared for the next time the grid isn’t working - download Loci.
9
462
Stanford study: "local LMs can successfully answer 88.7% of single-turn chat and reasoning queries"
(Tested across 1 million real queries)
The craziest part? This study came out a year ago - local models have both gotten much more accurate as well as way smaller.
1
16
568
Read the full study: arxiv.org/abs/2511.07885
2
70













