Theorem (@theoremlabs) profile picture

Accelerate software verification. https://nitter.cf/t.co/ITzBccyRcT

San Francisco
Joined May 2025
Photo by Theorem (@theoremlabs)Photo by Theorem (@theoremlabs)Photo by Theorem (@theoremlabs)Photo by Theorem (@theoremlabs)Photo by Theorem (@theoremlabs)Photo by Theorem (@theoremlabs)Photo by Theorem (@theoremlabs)Photo by Theorem (@theoremlabs)
the good news is that we can now start writing fuzz harnesses that exercise the full range of production behaviors. because we won't be running them forever, we'll just prove them.
really cool post. essentially taking existing fuzzing harnesses, and "extending" them via formal verification to cover more inputs nothing more nothing less unsurprisingly, there's lots of trivial bugs that these harnesses miss. so it might be a while until we're "vulnzero"...
3
9
1
33
2,450
Replying to @jessi_cata
It didn’t make the final draft, but imo the most exciting story here is that LLMs have unlocked a new *kind* of FV. A human would balk at binary-level verification, because of how tedious it makes the proofs. But LLMs don’t care about tedium. It plays to their strengths.
3
1
26
1,992
The bitter lesson of ML is that scale eats all. The bitter lesson of PL is that [efficient] brute force eats all.
The results from Anthropic + Theorem feel like a “bitter lesson” moment for formal verification. They just went for semantics-level proofs. No fancy program logics, proof techniques or algorithms. Just bare metal proofs with a vanilla Hoare logic over RISC-V.
4
17
3
108
6,620
Theorem retweeted
I am a total fan of @theoremlabs and what it does to add formal verification to probabilistic LLM's! Important concept broadly.
New on our Frontier Red Team blog: using Claude to formally verify software at large scale. Formal verification can prove that code is free of certain bugs, but historically it has only been feasible for small codebases (or required a lot of expert time). With AI models, however, this is beginning to change. In collaboration with @theoremlabs, we used Claude to prove security properties in over one hundred real-world open-source libraries over the span of a few weeks. Read our research in full here: anthropic.com/research/forma…
15
22
2
358
42,664
Highlights for security folks: - We're open sourcing thousands of theorem statements from 437 libraries, our Linux process semantics, and ASan for static binaries. Let's turn a lot more fuzz tests into theorems, and get cracking at the proofs! - OSS Verify: We’ll verify safety properties of critical projects for free. Please enroll!
New on our Frontier Red Team blog: using Claude to formally verify software at large scale. Formal verification can prove that code is free of certain bugs, but historically it has only been feasible for small codebases (or required a lot of expert time). With AI models, however, this is beginning to change. In collaboration with @theoremlabs, we used Claude to prove security properties in over one hundred real-world open-source libraries over the span of a few weeks. Read our research in full here: anthropic.com/research/forma…
6
16
1
97
10,404
Theorem retweeted
Three days before I was pushed out of OpenAI I posted a Formal Verification Manifesto in a core research Slack channel. It listed OpenAI systems that could benefit from formal verification. The only person who responded was @sama, who was very supportive. Not sure what the moral of this story is, but it's never too late for OpenAI to invest seriously in software & hardware formal verification! On a more serious note, @theoremlabs and @AnthropicAI did something amazing. They showed how to verify software at scale without requiring developers to write specifications in Rocq or Lean. Instead, they used existing fuzz harnesses as specifications and LLMs to generate machine-checked proofs. Of course, fuzz harnesses specify only some correctness properties. In the simplest case, the proofs establish crash-safety, not full functional correctness. We need community-maintained open-source projects with reusable formal specifications and correctness proofs for major libraries, kernels, compilers, virtual machines and hardware. Human review of critical specifications and LLM-generated patches is essential. A model could satisfy a specification simply by removing functionality. As AI writes and rewrites critical infrastructure, formal verification should become a key part of AI control. Critical properties of AI-generated code should be machine-checked rather than taken on faith. AI is making vulnerabilities easier to find and exploit. Frontier LLMs and reusable formal specifications could finally make formal verification practical and economical at scale, while keeping it understandable to humans. Kudos to Theorem co-founders @rajashree and @diagram_chaser!
New on our Frontier Red Team blog: using Claude to formally verify software at large scale. Formal verification can prove that code is free of certain bugs, but historically it has only been feasible for small codebases (or required a lot of expert time). With AI models, however, this is beginning to change. In collaboration with @theoremlabs, we used Claude to prove security properties in over one hundred real-world open-source libraries over the span of a few weeks. Read our research in full here: anthropic.com/research/forma…
21
35
4
417
41,797
Making sure I state it clearly: We can literally patch and prove bug-freeness of the entire software stack running in production. With AI that exists right now for ~$100m. This means no more software-level hacks. I know it sounds too good to be true, but it isn't. AMA.
We formally verified 123 open-source projects in a partnership with Anthropic. Formal verification at scale is here! Now we’re launching OSS Verify to scale it to the software we all depend on. We prove it’s crash-free or send you the patches it needs. Free for open-source. Let’s bootstrap the verified software ecosystem: theorem.dev/oss-verify
50
52
25
822
91,700
If I recall correctly, I met @diagram_chaser and @rajashree at a YC event sometime around F24, which I believe was their batch. They were the second FM YC company, after us, and the first to do interactive theorem proving. To the best of my knowledge, they were the second overall company to do LLM-based ITP after @HarmonicMath . We were dealing with gpt-4o in those days, and I couldn't even get the model to write a good fuzz test, or set up a sandbox (or an AWS lambda or similar) with someone's repo. There was a bunch of demand -- we launched agentic PBT and __immediately__ had >100 users, which was way more than we could keep up with at the time -- but the tech wasn't even remotely there. I thought Jason and Rajashree were fucking nuts to be trying to actually prove stuff, when literally just testing was borderline impossible. Later, I heard rumors of research projects with Redwood and maybe ARIA, and I thought, huh, I guess that's a reasonable way to keep your company alive and learn things without taking on more non-dilutive funding. Good for them, they found a way to do the research lab thing for a while and have some fun. FM is about the fun we have along the way! But I was hella wrong to be remotely skeptical. The reality is that, Jason and Rajashree spent two years becoming state of the art at large scale agentic verification. Later, when a biglab asked me "who should we talk to if we want to do formal methods", I was like, "I mean obviously, Theorem." I think a **lot** of other people said the same thing. Who else? (To be clear, there are a couple other quite meaningful players in this space, like @axiommathai and @mathematics_inc , but they've largely positioned themselves as AI neo-labs themselves, which makes them awkward partners for a biglab, so ...) I'm not convinced that a world where we formally verify assembly code is really ideal. I am concerned it enables a world where we LLM-generate assembly, which I view as an uninterpretable and harmful outcome for humans (here I disagree with both Elon and my employer ..). I would like to believe that there are all kinds of really elegant, important properties that are best expressed over more abstract representations. But a world where we've proven *anything at all* about a giant proportion of software is actually a **massive improvement** over the counterfactual. So even if I feel inclined to nitpick the approach, the reality is that this looks like a phenomenal overall uplift in cybersecurity and a totally worthwhile and important endeavor. This is a great time to start learning formal methods. And it's a great time to work in formal methods. I really couldn't be more excited about where we're at, and can't wait to see what's next from the folks re-inventing the field!
New on our Frontier Red Team blog: using Claude to formally verify software at large scale. Formal verification can prove that code is free of certain bugs, but historically it has only been feasible for small codebases (or required a lot of expert time). With AI models, however, this is beginning to change. In collaboration with @theoremlabs, we used Claude to prove security properties in over one hundred real-world open-source libraries over the span of a few weeks. Read our research in full here: anthropic.com/research/forma…
5
7
68
2,973
Lots of very sharp folks seem to have joined Theorem lately Not surprising if you've met @rajashree but still cool to see!
4
6
55
5,377
Theorem retweeted
Amazing, they bypassed the need for a formally verified compiler by verifying at the assembly code level.
New on our Frontier Red Team blog: using Claude to formally verify software at large scale. Formal verification can prove that code is free of certain bugs, but historically it has only been feasible for small codebases (or required a lot of expert time). With AI models, however, this is beginning to change. In collaboration with @theoremlabs, we used Claude to prove security properties in over one hundred real-world open-source libraries over the span of a few weeks. Read our research in full here: anthropic.com/research/forma…
11
21
1
632
38,378
Theorem retweeted
How to find ALL PHP bugs: 1. Compile code (e.g. to RISC-V, x86, LLVM IR) 2. Vibe prove ASAN and non-ASAN versions have same behavior across all inputs. Hard, but Claude can do it! 3. Find a LOT of bugs Super proud of our work. A big step towards finding ALL memory safety bugs
New on our Frontier Red Team blog: using Claude to formally verify software at large scale. Formal verification can prove that code is free of certain bugs, but historically it has only been feasible for small codebases (or required a lot of expert time). With AI models, however, this is beginning to change. In collaboration with @theoremlabs, we used Claude to prove security properties in over one hundred real-world open-source libraries over the span of a few weeks. Read our research in full here: anthropic.com/research/forma…
24
61
5
1,190
55,436
The thing that excites me the most about formal verification is that we can section off parts of our software and be like “boom, no more bugs here” Asymmetric defense at scale
New on our Frontier Red Team blog: using Claude to formally verify software at large scale. Formal verification can prove that code is free of certain bugs, but historically it has only been feasible for small codebases (or required a lot of expert time). With AI models, however, this is beginning to change. In collaboration with @theoremlabs, we used Claude to prove security properties in over one hundred real-world open-source libraries over the span of a few weeks. Read our research in full here: anthropic.com/research/forma…
19
18
4
181
11,001
We formally verified 123 open-source projects in a partnership with Anthropic. Formal verification at scale is here! Now we’re launching OSS Verify to scale it to the software we all depend on. We prove it’s crash-free or send you the patches it needs. Free for open-source. Let’s bootstrap the verified software ecosystem: theorem.dev/oss-verify
New on our Frontier Red Team blog: using Claude to formally verify software at large scale. Formal verification can prove that code is free of certain bugs, but historically it has only been feasible for small codebases (or required a lot of expert time). With AI models, however, this is beginning to change. In collaboration with @theoremlabs, we used Claude to prove security properties in over one hundred real-world open-source libraries over the span of a few weeks. Read our research in full here: anthropic.com/research/forma…
38
118
25
1,066
134,776
guess we'll just verify all of the software then
Last week some of South Korea's biggest banks were hit by a cyberattack. Thanks to a report from CrowdStrike tonight, we now know the entire hack may have been done by a single person. He used a combined stack of an open-source AI penetration tool named ARTEX, DeepSeek v4.1-Flash, GLM-5.3, Grok 4.6, and Claude Code.
3
18
1,179
Theorem retweeted
We’re organizing Vibecheck, a formal methods hackathon with some of the best folks in the space - @maxvonhippel, @qd_forall, @nolanlwin, @jessemhan, @emiyazono, and @workersio. You’ll get a weekend to build real production software, and formally verify it. We’ll have people who really know their stuff around to help, and you can come with a team, find one there, or just hack on something yourself. We are glad to have a phenomenal group of sponsors, including @harmonicmath, @mathematics_inc, @theoremlabs, @thegp, @omnicom, @primeintellect, @astriolabs, @buildwithparty, @wearerandomlabs, and @lanyon_ai, as well as some more we will announce in the coming days! If you’re a hacker, a formal methods person, or just someone who thinks “how do we know it works?” is an interesting question, we’d love to have you. Signup Now: fmxai.org/vibecheck/
21
41
21
206
77,650
Let's verify all the software -- finally published a botec on how much it might cost. Some headline numbers and estimates: - ≈$400M: all of nixpkgs in ≈5 yr on a $100M/yr budget, if AI capability keeps doubling every 8 months - ≈$134M: 95% of the code by end of 2027, if we speed that up 4× for verification - program size is heavy-tailed, most software is small; the median binary is about 30 KB, and verifying just 9 MB is enough to cover all deps of ⅔ of all code - verification of Linux+Firecracker would cost ≈$1M theorem.dev/blog/verified-so…
13
15
1
88
6,219
We scheduled all 123,471 binaries in dependency order. At $100M a year, with model capability doubling every eight months, the last package would finish in just under five years, for about $404M. Explore when Python, curl, or another package you care about gets verified.
2
1
23
1,310
More parallelism speeds us up, up to ~$80M a year. The largest binaries are the bottleneck. After that, better models get us there faster. If capability on this task doubles every two months, 95% of nixpkgs could be proved by the end of 2027, at $134M in total.
1
18
610
What would it take to formally verify all the software a real system runs on? We modeled one ecosystem: nixpkgs, with 28,650 packages and 154 GB of machine code. theorem.dev/blog/verified-so…
7
13
6
129
22,940
A proof of OpenSSL can be reused by the nearly 16,000 packages that run on it. It would cost about $220K once, or about $14 per package that reuses it. If every package verified its own dependencies separately, the total would be about $190B.
1
1
21
804
Verified software will be built from verified software. Nixpkgs records each package's dependencies. We'll verify in dependency order at the binary level, reusing as we go. Just 9 MB of shared libraries is what two thirds of its code depends on.
2
1
42
5,782