Leo the Bayesian (@Leothebayesian) header
Leo the Bayesian (@Leothebayesian) profile picture

Price the tail

Joined February 2024
Photo by Leo the Bayesian (@Leothebayesian)Photo by Leo the Bayesian (@Leothebayesian)Photo by Leo the Bayesian (@Leothebayesian)Photo by Leo the Bayesian (@Leothebayesian)Photo by Leo the Bayesian (@Leothebayesian)Photo by Leo the Bayesian (@Leothebayesian)Photo by Leo the Bayesian (@Leothebayesian)Photo by Leo the Bayesian (@Leothebayesian)Photo by Leo the Bayesian (@Leothebayesian)Photo by Leo the Bayesian (@Leothebayesian)
Leo the Bayesian retweeted
Looped models are weird. A layer can write to itself or read from "future" layers. @thinkymachines gave us early access to Inkling 2 Mini so we could study its inner mechanisms. We uncover how it coordinates information flow across layers, blocks, and loops. Studying Inkling 2 Mini gave us cause for optimism: as transformers continue to evolve, we believe our ability to understand their internal mechanisms can evolve with them. Read the full post: goodfire.com/research/interp…
Some of you thought that Inkling 2 Small was too big at 594B parameters. We heard you: Inkling 2 Mini is a looped transformer with just 26B parameters, small enough to run on your laptop, powerful enough for agentic work with a score of 20.1 on AA Index. thinkingmachines.ai/news/ink…
18
178
24
1,453
107,596
Leo the Bayesian retweeted
With all the noise around formal verification today, it is easy to forget that the Lean or Dafny approach is not the only game in town. When you look closely at how different paradigms connect specifications, proofs, and executable code, four distinct philosophies emerge—each differing on who does the work of finding the witness. 1. Program, Then Prove (Lean, Dafny) In Dafny or Lean, you write an algorithm that computes the value you want—the witness—alongside a proof that the witness satisfies your specification. In Dafny, the specification and implementation languages are distinct. Yet even in Lean, which fuses them via dependent types, the proof is erased at compile time; at runtime, the machine simply executes the deterministic algorithm you wrote. 2. Program Calculation (Squiggol / Bird–Meertens) In the Squiggol style, you start with an obviously correct, albeit inefficient, executable program. There is no separate specification and implementation—the specification is the implementation. From there, you apply algebraic, correctness-preserving transformations (like fusion and promotion) to calculate an efficient algorithm. Every equational step is its own proof, though you are still manually deriving the algorithm that computes the witness. 3. Constructive Proof Extraction & Refinement (Rocq, Isabelle/HOL) Rocq (formerly Coq) and Isabelle/HOL take this a step further through code extraction. In Rocq’s constructive logic, you don't even have to write the algorithm first: you can prove an existential theorem (∃ witness), and Rocq will erase the purely logical steps to extract the computational skeleton of your proof directly into an OCaml program. Classical systems like Isabelle/HOL achieve a similar result through stepwise data refinement, transforming abstract mathematical specs into executable code. Still, the human must construct the proof—and thus the underlying algorithm—ahead of time. 4. Proof Search as Execution (Logic Programming) Logic programming collapses the entire divide at runtime. You state the axioms, and your query is the existential theorem you want to prove. Execution is the proof search itself, using unification and resolution to dynamically construct the witness you are after. In the first three approaches, the human invents or vibe codes the algorithm (or proof) that produces the witness, and the machine verifies it. Only in logic programming do you state what makes a witness valid and let execution itself find both the proof and the witness together—which is why logic programming is my favorite paradigm for formal verification.
33
63
8
531
23,188
Leo the Bayesian retweeted
New paper with Alex Imas (@alexolegimas) and Shane Legg (@ShaneLegg). We build a model of how intelligence is organized into firms, who wins and loses, and when the economy will be transformed: arxiv.org/pdf/2611.05417 Blog post: institute.deepmind.com/essay…
16
114
12
549
53,109
Leo the Bayesian retweeted
Swarm riders and Swarm-native companies AI-use even among highly AI-native people is power-law distributed. There seems to be a new class of people — call them “swarm riders” — who are capable of consistently commanding large swarms of agents to productive ends. This is a new kind of managerial skill, analogous to the skill it takes to manage large organizations of people, but wholly different in nature and of course more technical. Based on the swarm riders we know (including many at @Tempo and @Paradigm), these people are capable of productive token spend that is 1000x-10000x higher than median software engineers, and 100x higher than top 5% Claude Code users. The art of swarm riding is different from software engineering and more akin to machine learning, where practitioners grapple with black box systems: defining reward functions and evals, transmuting non-verifiable problems into verifiable problems, running experiments, etc. Swarm-native companies are a new class of company founded by and populated by swarm riders. These companies are organized around swarms in a way that conventional companies will struggle to replicate, as the cost of internal human coordination is sand in the gears of an effective swarm. Effective swarm-native companies will define the right reward functions and evals, remove functional and bureaucratic barriers, and then let the swarms cook. We’ve gotten to see a few swarm-native companies up close — reshaping financial rails @Tempo, building a new gaming platform @Spawn, cyber swarms at Cathedral Systems — and it’s clear these teams wield a significant advantage over pre-AI teams. Some questions I’ve been wondering about: 1) How fast will swarm riding diffuse through the economy? The implications for token spend could be significant. 2) Can swarms be effectively adopted by existing companies or will it generally track the formation of new swarm-native companies? 3) What becomes scarce as swarm capabilities themselves commoditize (through frontier models becoming better at swarming, swarm capabilities built into harnesses, etc)? Cultures and teams that can define reward functions well and otherwise get out of the way?
144
80
115
1,174
284,198
Leo the Bayesian retweeted
Replying to @dhsusj144284
dIstRiBuTiOn iS tHe mOaT
1
1
125
4,356
Leo the Bayesian retweeted
Replying to @drwconvexity
So in the future, every company will operate like a trading desk, systematically managing the risk exposures and residual values of compute, tokens, and other factors of production on its balance sheet.
1
1
233
Leo the Bayesian retweeted
Hi! this is Rohan Anil, I develop training algorithms among other things - my work trains various frontier models. At CoreAuto, I was fascinated by the problem of training on compute islands, eventually in space training. I hope you all enjoy this write up!🪐🛰️ coreauto.com/hitchhikers-gui…
66
137
39
1,581
193,782
Leo the Bayesian retweeted
Replying to @FredaDuan
Astra is NOT robotics’s GPT-3 moment
3
1
62
6,327
Leo the Bayesian retweeted
This article is a good step toward understanding how things really work. However, quite a few factors related to the experimental setup need to be taken into account if you want to draw more reliable conclusions: 1. The optimal shape of each architecture may differ under compute-optimal settings. 2. The optimal initialization of each parameter may differ across architectures. 3. Different architectures may respond quite differently to different optimizers. 4. Differences in training stability and the health of “internal metrics” can lead to drastically different results, even for the same architecture. 5. Architectures may respond differently to varying degrees of overtraining (tpp). 6. Different architectures may respond quite differently to different data distributions (i.e., different learning objectives). 7. The difficulty of scaling up can vary considerably across architectures. There are many more nuances that need to be carefully addressed to obtain consistent results.
Finally releasing the results of the arch ablations I was running with bux over the weekend
3
34
2
328
40,859
Leo the Bayesian retweeted
Your tweets are so funny dude. They make me laugh so much man. Banger after banger. You should come to my event later this week. Well it's not actually my event, it's my friends event. Okay I was just kidding. It's my friends' friends event. I'm just tagging along. It's going to be sick dude. There are going to be so many AI researchers at this event bro, the scene is fire. You have to be there or else you'll miss the next big data company at the seed bro. Last week, I went to this other party and there was a brand new data company that was incorporated at the start of the party and already was at 100m run rate by the end of the party. It was insane. Dylan Patel led the pre-seed and the ingroup consortium of Elad, Lachy, Sholto, and Dwarkesh led the seed. The round was so hot man. You could feel the heat. Did I get into it? No no. I tried, but the founder wasn't taking angel checks. I'm not actually legally allowed to say his name, before we left the party, they made us all sign NDAs so we wouldn't leak the info to the VCs, but don't worry. I'll tell you, just keep it on the down low. Also just make sure to come to my party later this week, you don't want to miss the next ripper. Well, not my party, my friends' party. Oh actually, maybe let me just make sure that I can bring you as a plus one. You should definitely plan on coming though, everyone from the scene is going to be there. Even the Citrini guy that took the scorpion string straight on his dick. Apparently he had to go to the hospital, but its all good, the antivenom was real. Did you hear about the antivenom that Claude made for the sting? No I don't know anything about it, I just heard about it last week at my friends friends friends party. Just join the scene bro, it's going to be awesome, there's so many events. Every week is tech week when you're in San Francisco. I love it here so much. Want to grab coffee with a YC founder later? I can ask if you can be my plus one
19
9
3
256
11,661
Leo the Bayesian retweeted
Replying to @januarycomputer
Everything is O(1) to an oracle
6
12
538
21,830
Leo the Bayesian retweeted
Replying to @aleximm
actually you don’t just need live eval. you also need to forecast the world state after a chain of agent actions. that takes a really strong general forecaster.
1
1
53
Leo the Bayesian retweeted
Introducing Statebook: Markets for All States of the World. For the past few months, I’ve been exploring a simple market-design question: What becomes possible if we start with the exposure someone needs, rather than the contract an exchange happens to list? This question is becoming increasingly relevant as @HyperliquidX, @Kalshi, @Polymarket, and vertically specialized exchanges such as @Architect_Fi expand the universe of tradable underliers, and the boundaries between perpetuals, options, and event contracts begin to blur. My answer: structured-product heaven. The working paper and interactive research interface are below. Thank you to @0xsmac, @0xfishylosopher, and Chris Saunders for reviewing and contributing to the initial draft. statebook.app
40
21
15
265
37,203
Leo the Bayesian retweeted
I must state again, there is lot more progress if we rethink gradient descent for training.
23
13
2
383
52,216
Leo the Bayesian retweeted
On recruiting great ppl: - great people are (a) rare, and (b) always have tons of options - therefore once you decide a candidate is great you cannot hesitate or delay. You must drop everything and rearrange heaven and earth to get them - there are small things (sending flowers), medium things (comp), and big things (their identity and dreams) - you want the barbell strategy-- put all your energy into small things and big things. Leave medium things till the end Small things: - ppl massively underestimate the power of small gestures. TL;DR: love-bomb the candidate - spend more time than money. Expense accounts are abundant but time is scarce. It's an expensive signal no one can fake - stay in close contact over text and phone - send them a first edition used book or an analog oscilloscope or some other random thing they mentioned in passing they like. Make these unique, everyone sends wine - invite them to computer history museum or moma or something. Spend time with them! Treat these meetings as dates, make them unique. Everyone buys dinner - be persistent but not annoying. Sometimes ppl need time alone, you should respect that. Big things: - there is a gap between what the candidate is doing in life and how they see themselves in the privacy of their dreams. Your job is to help them bridge this gap - to bridge the gap you must first understand it. Take the candidate on walks to museums and parks. Walks help the candidate drop their guard so you can learn about them. Try hard to get to know them, to understand their dreams - use your mission, your culture, your people, your company to paint a world for them in which the gap no longer exists. You may need to adjust the role or even change it altogether. For great ppl it is worth it - don't assume ppl want what you want, dreams can take all kinds of shapes. Maybe they want their work to matter, maybe they want a bigger team, maybe they want to code and avoid meetings, maybe they want to be in the center of the action, maybe they want to learn, maybe all or some or none of the above. Find out! Medium things (aka closing): - you've constructed a world the candidate is excited to live in. The purpose of this phase is to remove mundane obstacles to make this world easy to enter - I don't think I ever sold a candidate on a dream and then lost them on comp. Ppl just want to be compensated fairly and work out some life constraints - I don't have much to add to all the comp advice out there. My philosophy on comp/closing is to get this part done and move on, don't penny pinch too much, and give more equity --- Failure modes/don't do this: - do not ignore candidate's spouse/parents. They have an outsized role in the decision. Most of what I said about the candidate applies to them too (love bomb them, understand their dreams, etc.) - counterintuitive, but do not talk about the company or even the mission too much. The candidate and their dreams are front and center. At this stage you, the role, the mission are only relevant to the degree they can make candidate's dreams come true - same w/ perks, what the day is like, etc. Don't blindly talk about these-- use them as tools to reenforce how the candidate seems themselves - don't get mad if ppl reject you, it happens. Maybe it's not a fit for them now but will be a great fit four years from now. Maybe they become a customer. Maybe you work for them ten years later. Play the long game.
63
126
41
2,666
267,264
Leo the Bayesian retweeted
you are 1 pivot away from being a billionaire
Cursor, 4 years ago
45
256
11
4,976
486,198
Leo the Bayesian retweeted
Introducing Blanket! Blanket helps small businesses find relevant @Kalshi markets for hedging risk and running promotions. Describe your business risks and it will help identify the Kalshi markets that fit. Blanket is a research product and will be open sourced after testing.
143
47
66
1,046
320,214
Leo the Bayesian retweeted
Crypto feels firmly in its own post-dot-com era. Risk assets are down, companies and funds are closing shop, and sentiment is low. Momentum traders, the hype cycle, and culture has moved on to AI for good reason (the commodity + EM/China hype in the 2000s was a powerful narrative too). What followed the dot-com era blues, however, was relentless growth of cloud and mobile infra and apps powered by the internet. Trillions of enterprise value created. Salesforce, ServiceNow, and Shopify. AWS, Azure, and GCP. Hundreds of publicly traded companies. Stablecoins, perps, tokenized assets, credit protocols have been battle tested over the last decade and are open-sourcing capital markets to the world. Peering beyond the post-GENIUS horizon, financial services and payments will be completely transformed. We’re seeing this today with companies across the Fortune 500 gearing up to go live @withAUSD. Across use cases like trading, treasury, and embedded financial services. Nearly all of the recent breakout companies like @raincards, @turnkeyhq, @HyperliquidX, @fomo are driven by meaningful revenue not tokenomics. The crypto era had its chapters driven by experimentation and belief. “It was the best of times, it was the worst of times, it was the age of wisdom, it was the age of foolishness.” The digital assets era is just beginning and it won’t look anything like what came before. That’s exactly what makes it exciting.
12
15
7
247
26,853