A.I. news · October 24, 2026
Claude Found Bugs Trillions of Tests Missed. Now What?
Anthropic says Claude proved certain crashes can't happen in 123 open-source libraries, and it also found old bugs that trillions of test runs missed. But Anthropic is clear that this doesn't make the code bug-free.
🔎 What Anthropic says it did
Anthropic says Claude proved certain crashes can't happen in 123 open-source libraries. It published the research on October 22, 2026. The proofs cover 239 entry points and are machine-checked.[1][2]
The surprise: Claude also found bugs that had eluded detection for years, despite trillions of test runs.[1]
So is the code now bug-free? Anthropic says no. Its own post says the result doesn't mean the libraries contain no bugs.[1]
Theorem was Anthropic's collaborator on the project.[1]
🧮 How the proofs work
Proofs have been costly. Verifying the seL4 kernel took 20 person-years, versus two to build it. That's why this has only been practical in mission-critical work, like Airbus avionics code.[1]
Anthropic's idea was to turn each library's existing fuzz test into a spec, which is a precise statement of what correct means. The rule it proves: the modeled RISC-V fuzz test never crashes on any input under 64 MB.[1]
Claude writes the proof. The Rocq proof checker confirms it meets the spec. Anthropic says virtually none of the proof code needs manual review.[1]
The proofs total about 180 million lines of Rocq. A typical proof runs about 600,000 lines, covering binaries with a median of 18,000 RISC-V instructions. It all happened over a few weeks, and Claude patched bugs along the way.[1]
🐛 The bugs Claude found
In one library, Claude found a counter underflow in a decompressor. It caused an out-of-bounds write, but only on rare inputs of around 20 MB.[1]
Claude also rediscovered CVE-2026-56132 in libexpat.[1]
Anthropic says one widely used PDF reader yielded as many security bugs in one week as are typically reported in a year. The post doesn't name the reader.[1][3]
Anthropic also had Claude prove that the Linux kernel's zlib implementation correctly implements its spec. It repeated the process for half a dozen other libraries.[1]
In one experiment, five of seven injected bugs survived random testing, but all seven broke the proof. Our take: that's one small test, but it suggests proofs and tests could work best together.[2]
💡 Why it matters: cost and reach
Anthropic says most software is riddled with bugs that A.I. can easily find and exploit. It points to the Linux kernel, which recently had over 1,000 vulnerabilities disclosed.[1]
Anthropic predicts formal verification will soon be as practical as fuzzing. That's a prediction, not a result, and we'd treat it that way.[1]
Cost could matter more than the headline. Proofs can take hundreds of millions of tokens to produce. Anthropic says cost-efficiency is critical to reaching the whole software ecosystem.[1]
There are early signs of savings. On 36 small tasks, Opus 5.5 proved half of them with about 40% fewer tokens than Opus 5. Agents sharing a workspace needed 17% fewer tokens than agents working alone.[1]
🚀 What you can do with this
Theorem offers the pipeline free to qualifying open-source projects through OSS Verify. Enrollment is by pull request that adds three files. Bugs are privately disclosed by email with no deadline, and proofs can take multiple days.[2]
Anthropic applied the process to hundreds of OSS-Fuzz projects. It says developers who tried formal verification with less capable models should try again. Theorem is also publishing a Claude skill to help developers get started.[1]
Our take: if you write code, check whether your project has fuzz tests, since that's where this method starts. If you run a business that depends on open-source code, ask your developers which libraries have been verified, and under what assumptions.
⚠️ What we don't know yet
The proofs assume a single-threaded static binary with a fixed memory budget. Most people run these libraries on x86 or ARM and dynamically link against glibc. Anthropic also didn't verify all the entry points for each library.[1]
Anthropic says remaining bugs are in trusted components or outside the proof's scope. It's also holding back proofs that may reveal an unpatched bug.[1]
Claude could make any theorem provable by rewriting a program so it never crashes. Anthropic says it penalized and audited patches to discourage that.[1][2]
The File covered the story but says it didn't re-run the proofs or open the outside review the post links.[3]
Our take: watch for independent checks, proofs on more libraries, and whether costs keep falling. We also can't say whether the bugs are fixed upstream.
💡 Our take: what this could mean for you
- Ask which open-source libraries your product depends on, and whether any have been formally verified.
- Treat a proof as one extra layer, not a seal of approval. Keep patching.
- If you maintain open-source code, look at Theorem's free OSS Verify program.
Have a business or a business idea? 🚀
Trinity builds a free, personalized website prototype for it — no credit card.
Get my free prototype