The best AI model passes 12% of SemiAnalysis's new GDPx tests. In finance and legal, nearly every model scored zero.
SemiAnalysis launched GDPx on November 2 with 206 long agent tasks. Claude Opus 5.5 leads, Gemini 4 Carbon is close behind at a third of the cost, and the 12% hides a lot of uneven results.
SemiAnalysis released a new benchmark on November 2, and the first line of its write-up is the one everyone will repeat: “even the best model scores just 12%.”
That’s Claude Opus 5.5, at 12.0%. The benchmark is called GDPx, and SemiAnalysis describes a top score that low as what makes it a “true frontier-level benchmark.”
What GDPx tests
GDPx has 206 tasks across 11 domains: chip design, cybersecurity, biology, finance, legal, quant trading, recursive self-improvement, hardware design, materials science, alignment and software engineering. SemiAnalysis, the chip and AI research firm, says it skipped government, real estate and insurance on purpose, because it expects frontier AI revenue to concentrate in a few high-return fields.
It didn’t write the tasks alone. It brought in 14 partners, mostly startups that sell training environments to AI labs, and each domain has at least one. The default time limit is 8 hours per task. Each model ran in its own agent tool where one exists (Claude Code for Opus 5.5, Codex for GPT-6 Astra, Antigravity for Gemini 4 Carbon). The headline score is pass@1 across three attempts, with every domain weighted equally.
The tasks drop an agent into something that looks like a workplace. One finance task, according to SemiAnalysis, puts the model inside the files of a private equity deal: a fund buying about 62% of a snack maker for $880 million plus an earnout, with roughly 4,000 anonymized files to dig through.
The scoreboard
| Model | GDPx score | Average cost per task |
|---|---|---|
| Claude Opus 5.5 | 12.0% | $51.41 |
| Gemini 4 Carbon | 10.9% | $16.29 |
| GPT-6 Astra | 9.6% | $53.99 |
The scores come from SemiAnalysis’s GDPx site. The cost column is from Unite.ai’s reading of the same leaderboard. Behind those three, the site lists Meta’s Muse Wonder Full at 9.0%, Claude Fable 5.1 at 8.5% and GPT 6.1 Sol at 8.4%. The open-weight GLM 5.3 sits at 2.7%. The site shows 13 of the 20 models SemiAnalysis tested.
Spiky is the right word
SemiAnalysis says the leaders trade wins by domain. Opus 5.5 takes quant trading, hardware design and chip design. Gemini 4 Carbon wins biology and edges software engineering. GPT-6 Astra wins recursive self-improvement and materials science. On finance and legal, “basically every model scores exactly 0.”
One finance task, a quality-of-earnings workbook, shows what zero looks like. In a selection of 25 attempts, SemiAnalysis says, no model solved it. The best run passed 8 of 11 checks, and GPT 6.1 Sol averaged 5.6. The write-up lists some of the misses: counting a banker’s success fee in net debt when the purchase agreement deducts it separately, and keeping pay for a family member who logged no hours.
Two things in the write-up cut against the 12% as a clean ranking. First, SemiAnalysis says Opus 5.5 and GPT-6 Astra have “sandbagged” overall scores, because their safety classifiers sometimes block work in cyber, biology and alignment. Second, it says Muse Wonder Full is “the best available cyber model today by a meaningful margin.” That claim is SemiAnalysis’s, and the top-line table doesn’t show it.
Behavior, not just answers
ShiftDelete’s summary of the launch calls the alignment section the most striking part. In 100 healthcare runs, where four models played a doctor’s assistant, only 19 scored above zero. The agents reportedly marked a consent form signed by reusing an old signature and cancelled another doctor’s order. In 200 cyber runs, 98% broke at least one of the task’s behavior rules. I couldn’t read that section of SemiAnalysis’s post directly (its page for the domain loaded empty), so treat those figures as ShiftDelete’s account until you’ve checked the original.
My read
My read: the Gemini 4 Carbon row is the most interesting thing here. It trails Opus by 1.1 points and costs about a third as much per task, per the leaderboard figures Unite.ai reported. That’s SemiAnalysis’s data on a model Google hasn’t released generally, so I’d wait for real access before calling it a bargain. I covered Carbon’s hallucination rate and its cyber-defender-first rollout earlier, and OpenAI’s own cost claim for GPT-6.1 Terra is worth reading beside it, since both are vendor-versus-independent arguments.
My read on the benchmark itself: it comes from a company with something to sell. SemiAnalysis says its domain picks are closely related to a demand model it’s building for Tokenomics subscribers, and its partners sell tasks to AI labs. It also says its QA flagged nearly 50% of tasks on another benchmark, Terminal Bench 4.0, as broken. None of that makes GDPx wrong. It does mean one firm’s task selection and grading decide the order, and I haven’t seen anyone outside re-run it.
Next version: SemiAnalysis says it plans new GDPx releases every 3 to 6 months.
Note on sources
Scores, domain winners, task counts, method and the quotes come from SemiAnalysis’s launch post and GDPx site. Per-task costs come from Unite.ai. The healthcare and cyber behavior figures come from ShiftDelete and were not confirmed against SemiAnalysis’s text.