GPT-6 Astra Benchmarks: Three Tests Where It Loses
We analyzed every GPT-6 Astra benchmark. While it dominates most tests, three losses reveal more about the AI market than the record-breaking wins.
👁 118 reads
We analyzed every GPT-6 Astra benchmark. While it dominates most tests, three losses reveal more about the AI market than the record-breaking wins.
Every model launch follows the same ritual. The company publishes charts showing their new model beating everything, media outlets copy the three most impressive numbers, and a week later it turns out the picture was more complex. So we went through GPT-6 Astra’s complete benchmark table, including the results that didn’t make the headlines.
Let’s start with an honest statement: in most tests, this model’s advantage is real and sometimes enormous. But there are three places where it doesn’t win, and those tell us more about the market than the records do.
| Test | GPT-6 Astra | Fable 5.1 | Opus 5 | GPT-5.6 Sol |
|---|---|---|---|---|
| FrontierMath Tier 4 | 97.6% | 87.8% | 73.2% | - |
| ARC-AGI-3 | 99.9% | - | 30.2% | 7.8% |
| Terminal-Bench 4.0 | 57.7 | 55.8 | 52.3 | 37.3 |
| AutomationBench | 41.4 | 31.4 | 26.9 | 18.1 |
| ExploitBench | 100% | 70.0% | - | 78.5% |
| SRE-Bench | 88.0% | 12.5% | - | 55.9% |
| Humanity’s Last Exam (with tools) | 57.2% | 65.0% | 63.6% | - |
| FrontierCode 1.1 Extended | 64.5% | - | - | - |
| HealthBench Professional | 63.4% | 56.6% | - | 60.5% |
| MRCR v2, 8 needles, 512K-1M | 96.3% | - | - | 73.8% |
First Loss: An Exam Where Broad Knowledge Matters
Humanity’s Last Exam is a test composed of very difficult questions across many domains, designed so you can’t pass it with computational ability alone. In the variant with tool access, Astra achieved 57.2 percent. Claude Fable 5.1 achieved 65.0 percent, the older Fable 5 got 63.8 percent, and Opus 5 scored 63.6 percent.
This isn’t a margin-of-error difference, but nearly eight percentage points against the model that achieves 97.6 percent in the highest-level mathematics in the same table. In other words, we’re dealing with a highly specialized model, not one that’s uniformly better at everything.
Second Loss: Same Exam, Half-Price Model
In the same test, Astra is beaten not only by Anthropic’s flagship. Claude Opus 5, sold at $5 and $25, half of Astra’s rate, achieved 63.6 percent, which is also over six points more. If your work involves answering difficult questions across many domains rather than computing, you’re paying twice as much for a worse result.
Third Loss: Code in Extended Version
In the FrontierCode test in the extended variant, Astra achieved 64.5 percent, while the older Claude Fable 5 scored 64.9 percent. The difference is symbolic but telling, because we’re talking about a model launched nearly three months earlier at the same price.
The Most Impressive Number, and It’s Not Math
If I had to point to one result from this table that truly changes how you work, it wouldn’t be the 97.6 percent in mathematics or even the 99.9 percent in abstract reasoning. It would be the test for finding eight pieces of information hidden in very long text. In the range from half a million to one million tokens, Astra found 96.3 percent of them, while its predecessor achieved 73.8 percent there.
This is the difference between a model you can throw a complete project documentation at and trust it won’t miss anything, and a model where you have to divide material into chunks and monitor it manually. For daily work with long documents, this means more than a record in a mathematics olympiad.
The Second Number Nobody’s Talking About
In the set of tests checking model honesty, there’s an item measuring how often the system tries to cheat on a task instead of performing it when given the opportunity. Astra scored zero percent there. Its predecessor, GPT-5.6 Sol, tried to cheat in 48.2 percent of cases. Similarly in the safety test for computer control: 2.4 percent versus 22 percent.
If these numbers are confirmed in practice, this will be the most important change in this launch. A model that actually performs the task instead of looking for shortcuts is worth more than a model a few points better in benchmarks but gaming behind the user’s back.
Google · Your sourcesQuantslant higher in your Google - one clickAdd to preferred sources →What This Means When Choosing a Tool
Three conclusions. Astra is clearly best in mathematics, abstract reasoning, automation, and cybersecurity, and there its high price is justified. In tasks requiring broad general knowledge, Anthropic’s models still win despite identical pricing. And for everyday writing and editing, neither of these two models is necessary and you’re better off spending five times less.
All numbers come from materials published at the September 3, 2026 launch and compilations prepared by independent sites. We haven’t done our own measurement yet because the model hasn’t reached the resellers we test through. We’ll do it as soon as possible. We’ve collected the model’s complete data in our GPT-6 Astra guide.
More on this from QUANTSLANT
benchmarks·GPT-6 Astra Test: Non-English Writing, Code, 150K ContextWe tested GPT-6 Astra against Fable 5.1, Opus 5, and Gemini 3.8 Flash on six language tasks, code challenges, and needle-in-haystack retrieval. Times, costs, and tokens.