Project Glasswing fixes: 217 of 37,966 findings

One ratio explains most of the argument over Project Glasswing: 217 fixed out of 37,966 findings. That’s under 0.6%.

It comes from VulnCheck, a vulnerability intelligence company. Its researcher Patrick Garrity published a seven-month review on November 9 and read Anthropic’s public ledger the way a defender would. The ledger, as of Anthropic’s November 2 update, lists 37,966 findings, and just 217 are marked fixed. His post is titled “After 7-months, Glasswing Overwelms Maintainers with Underwhelming Findings” (spelling his). Infosecurity Magazine ran the same numbers.

Anthropic’s dashboard, which VulnCheck is reading, says something different on the same date. So I went and read both.

What VulnCheck counted

Garrity tracks CVEs credited to Anthropic in an open repo he started in April. After seven months he has 442 of them, about 0.5% of the 93,450 CVEs published this year. Only two of the 442 appear in VulnCheck’s catalog of vulnerabilities exploited in the wild.

The ledger is the bigger part of his case, and it’s the one with a table:

Measure	September 8	November 2
Total findings	26,153	37,966
Made public	2,736	7,349
Marked fixed	202	217
Duplicates	2	1,670
Claimed true positive rate	91.4%	94.2%

(All figures from VulnCheck’s post. In the text it gives 1,654 duplicates and in the table 1,670, so I’m using the table for the trend and the text for the snapshot.)

In two months, then, Anthropic added nearly 12,000 findings and 15 fixes. Garrity also finds it odd that the claimed true positive rate went up while duplicates and withdrawn entries (1,654 and 419 at the latest count) outnumbered fixes.

What Anthropic’s dashboard counts

Anthropic’s page, last updated November 2, doesn’t use the word “findings” the way VulnCheck does. Its glossary defines the 37,966 as “candidates”: every crash or vulnerability hypothesis Claude produced before any triage. From there the numbers narrow.

6,376 candidates were reviewed by outside security firms, and 6,009 were confirmed valid. The 94.2% true positive rate is that second number over the first.
5,087 were reported to maintainers: 1,862 after outside review, and 3,225 sent directly, which the dashboard flags as possibly containing false positives.
3,635 were acknowledged by a maintainer.
629 were patched upstream.

So the fix count is 217 on the ledger and 629 on the dashboard. Anthropic’s own pages don’t reconcile those two figures in anything I read, and I can’t say why they differ. My arithmetic: 629 is 12.4% of the 5,087 reported and 1.7% of all candidates.

The dashboard also admits where the limit is. Triage by humans, it says, “is the rate limiting step.”

The curl test

The one place where the unreviewed reports can be checked in public is curl. Maintainer Daniel Stenberg received a batch of 45 reports from Anthropic’s new OSS Scanner, which VulnCheck says Anthropic itself describes as model-generated without human review, and began posting them as public issues.

By the time of VulnCheck’s post, 34 had been published. Garrity counts 19 as not a vulnerability, 6 as confirmed low severity and 9 as still waiting. He calls that a 55.9% false positive rate. That figure is his tally from a partly finished batch, and “not a vulnerability” isn’t the same as “useless report”: a maintainer can reject something as a security issue and still fix it as an ordinary bug. I didn’t count curl’s issues myself.

My read

My read: neither side is wrong about its own number. They’re counting different things. VulnCheck counts what a defender can see, which is CVEs, fixes on the ledger and exploitation. Anthropic counts its pipeline, and by its own account most of that pipeline hasn’t been reviewed by a person.

What does bother me is the 94.2%. Anthropic’s glossary says it includes duplicates and findings a maintainer won’t fix, and it applies only to the 6,376 outside-reviewed findings. A reader who sees “94.2% true positive” next to “37,966 findings” will assume it covers all of them. It doesn’t. That’s a presentation problem on Anthropic’s side, not a lie, and the glossary is upfront about it if you click through.

Still, the cost has moved. A model writes a report in minutes, and a person still has to reproduce it. I wrote earlier about Claude finding the Xen VM-escape flaws, where the human validation step was explicit. And the GLM-5.3 cyber-exploit post covers why Anthropic held Mythos back in the first place. The Xen bugs were real. The open question is how many of the other 37,000 are.

Anthropic’s next dashboard update will show whether the patched count moves. I’d watch that number, not the findings total.

Note on sources

I read VulnCheck’s November 9 post and Anthropic’s dashboard and About page in full. The Infosecurity Magazine piece I saw only as an excerpt, and it matches VulnCheck on the headline numbers. I did not open curl’s issue tracker or Stenberg’s own posts, so the curl figures are VulnCheck’s.

Project Glasswing has 37,966 findings and 217 marked fixed. Anthropic's own dashboard says 629.