Gemini 4 Carbon benchmarks: all 19 tests and where it loses
Google reports Gemini 4 Carbon leads in 17 of 19 benchmarks. Here’s the full score table, what each test measures and where rivals win.
👁 117 reads
Google published a report with Gemini 4 Carbon’s results across 19 tests. It compares Carbon with Gemini 4 Argon, GPT-6 Astra, Claude Opus 5.5 and Claude Fable 5.1. I’ve reproduced the full table, explained what each test measures and noted where Carbon doesn’t come out on top. These are results reported by Google. I haven’t tested Carbon myself because it’s available only through the Fairwind programme.
Full results: 19 benchmark tests
The best result in each row is in bold. A dash means no result was reported for that model.
| Test | Carbon | Argon | GPT-6 Astra | Opus 5.5 | Fable 5.1 |
|---|---|---|---|---|---|
| Artificial Analysis Index v5.0 | 59 | 50 | 57 | 58 | 49 |
| Vals Index v2.2 | 70.4% | 69.2% | 65.0% | 70.2% | - |
| SWE-sweep | 9.6% | 4.3% | 9.5% | 3.6% | 2.9% |
| FrontierSWE 2 | 61.2% | 55.0% | 65.5% | 62.3% | 56.3% |
| Terminal-Bench 4.0 (AA) | 65.2% | 57.1% | 59.1% | 59.6% | 52.0% |
| Agents’ Last Exam v1.1 | 77.4% | 69.0% | 73.1% | 71.6% | - |
| OSWorld-2.1 (offline) | 81.5% | 74.7% | 80.9% | 87.2% | 76.8% |
| AutomationBench (Zapier) | 53.9% | 51.3% | 41.4% | 42.5% | 31.4% |
| Vending-Bench 2 (balance in USD) | 16,484 | 13,718 | 15,515 | 9,235 | 5,422 |
| Chartography | 72.5% | 71.6% | 71.0% | 66.3% | 46.2% |
| Blueprint-Bench 2 | 58.6% | 54.4% | 49.7% | 51.2% | 41.9% |
| GraphWalks 256k-1M | 96.2% | 84.2% | 71.8% | 66.8% | 65.0% |
| RiemannBench | 80.0% | 76.0% | 72.0% | 69.6% | 65.6% |
| Terminal-Bench-Science 0.1 (AA) | 64.8% | 52.9% | 63.3% | 59.0% | 43.3% |
| CWE-bench | 71.0% | 68.0% | 68.0% | 67.0% | 58.0% |
| ExploitGym | 54.8% | 42.0% | 42.4% | 34.5% | - |
| SEC-bench Pro | 88.2% | 83.4% | 85.4% | - | - |
Carbon comes out on top in 17 of the 19 tests. It loses FrontierSWE 2 to GPT-6 Astra and OSWorld-2.1 to Claude Opus 5.5.
What each benchmark measures
General intelligence
- Artificial Analysis Index v5.0 - an independent ranking that combines multiple tests; version 5.0 includes tools. Carbon scores 59, Opus 5.5 scores 58 and GPT-6 Astra scores 57. This is the only row in the table sourced entirely from an external company, and Carbon’s lead is smallest here.
- Vals Index v2.2 - a composite index of professional tasks from Vals AI. Carbon scores 70.4% and Opus 5.5 scores 70.2%, making them practically tied.
Coding
- SWE-sweep - very difficult coding tasks where every model scores in the single digits. Carbon scores 9.6%, compared with 9.5% for GPT-6 Astra. Google calculated Carbon’s score itself using the mini-swe-agent tool, while competitor scores came from the public leaderboard.
- FrontierSWE 2 - software engineering tasks from the public Proximal leaderboard. GPT-6 Astra leads here with 65.5%, compared with Carbon’s 61.2%. Opus 5.5 also scores higher, at 62.3%.
- Terminal-Bench 4.0 - a benchmark for agents working in a terminal, with results from Artificial Analysis. Carbon scores 65.2%, almost six points ahead of Opus. This is the most convincing coding result because it doesn’t come from Google.
Computer use
- Agents’ Last Exam v1.1 - 122 agent tasks completed within a five-hour window. Carbon scores 77.4%. Google calculated its score itself and took Astra and Opus scores from the public leaderboard.
- OSWorld-2.1 - the model interacts with a real operating system using only screenshots, across 82 tasks without internet access. Opus 5.5 leads with 87.2%, compared with Carbon’s 81.5%. If you’re building an agent to operate desktop apps, Claude is still stronger here.
Office and business tasks
- AutomationBench - Zapier-style automation tasks. Carbon scores 53.9%, while Opus 5.5 and GPT-6 Astra each score around 42%, putting them well behind.
- Vending-Bench 2 - the model runs a simulated snack vending machine over an extended period, with the final balance determining its score. Carbon earned USD 16,484; Opus 5.5 earned just USD 9,235. The test gives a sense of how well a model maintains a strategy over many steps.
Images and charts
- Chartography - reading charts without tools. Carbon scores 72.5%, just ahead of Argon at 71.6%.
- Blueprint-Bench 2 - understanding architectural drawings and plans, scored by similarity. Carbon scores 58.6%.
Long context
- GraphWalks 256k-1M - the model navigates a graph described in text ranging from 256,000 to one million tokens long. Carbon scores 96.2%; the strongest competitor, GPT-6 Astra, scores 71.8%. That’s the largest lead in the table. One caveat: Google calculated the scores for all models itself. I’ve explained how Argon’s one-million-token limit works in Gemini 4 Argon’s one-million-token context.
Science and maths
- RiemannBench - maths tasks from the public Surge leaderboard. Carbon scores 80.0%.
- Terminal-Bench-Science 0.1 - science tasks in a terminal, from Artificial Analysis. Carbon scores 64.8%, compared with 63.3% for GPT-6 Astra.
Cybersecurity
- SEC-bench Pro - finding real vulnerabilities in a JavaScript engine across 183 tasks. Carbon scores 88.2%, compared with 85.4% for GPT-6 Astra.
- ExploitGym - testing whether a vulnerability can be exploited. Carbon scores 54.8%, more than 12 points ahead of Astra at 42.4%.
- CWE-bench - fixing the underlying cause of a vulnerability, on a leaderboard maintained by Collinear. Carbon scores 71.0%; Argon and Astra each score 68.0%.
Google ran the ExploitGym and SEC-bench Pro tests itself in the Antigravity environment, without internet access and with the model’s safeguards disabled. Competitor results came from their own reports, not from the same test run.
How to read the benchmark table
- Who ran the tests? Some results come from independent leaderboards: Artificial Analysis, Vals, Surge, Zapier, Andon Labs and Collinear. Google calculated others itself: SWE-sweep, Agents’ Last Exam, OSWorld, GraphWalks, ExploitGym and SEC-bench Pro. Carbon’s biggest leads happen to be in tests calculated by Google.
- Test settings. Carbon and Argon ran at their highest reasoning settings through the Gemini API, with one attempt. Competitors generally also ran at their highest settings, but the figures come from their own reports.
- Small score gaps. On Artificial Analysis, Carbon scores 59, Opus 58 and Astra 57. Differences this small can easily reverse in the next leaderboard update.
In its earlier Argon report, Google showed results for 19 tests, with Argon winning 13. I covered those results in the Gemini 4 Argon benchmarks. Carbon’s tally is 17 out of 19, with its biggest gains over Argon in long context, science and cybersecurity. For a full overview of the model, pricing and access, see the Gemini 4 Carbon hub.
Google · Your sourcesQuantslant higher in your Google - one clickAdd to preferred sources →Sources
- Google DeepMind: Gemini 4 Carbon - Model evaluation, methodology and results (PDF), 28 October 2026
- Google: Gemini 4 Carbon: our next frontier model, 28 October 2026
- Artificial Analysis Intelligence Index
Checked: 28 October 2026.
More on this from QUANTSLANT
ai tools·Gemini 4 Carbon: Benchmarks, Price and AccessGemini 4 Carbon leads 17 of 19 Google-reported benchmarks, but access is currently limited to selected cyber defenders in Fairwind.