GPT-6 Cyber Benchmarks: What the System Card Really Measures
A breakdown of all benchmarks from the GPT-6 Cyber system card, including ExploitGym, FrontierCyber, refusal rates, and real-world exploit performance.
👁 117 reads
The GPT-6 Cyber system card features over a dozen benchmarks, and most point to the same conclusion: the model is substantially ahead of its predecessor, slightly ahead of base GPT-6 Astra, and no better than Astra at general software engineering. I gathered all the numbers from the official announcement and the system card in one place, detailing what each test actually measures. A caveat upfront: these are OpenAI’s reported figures. I have not run the model myself, as access is restricted to approved organizations in the Daybreak Red program.
Key takeaways
- ExploitGym: 48.5% compared to 32.1% for GPT-5.6 Cyber, representing a 51% relative gain.
- RemoteExploitBench: 95% of tasks solved; GPT-6 Astra with Daybreak Blue refused 100% of them.
- Standard software engineering (ExpertSWE): 82.95% versus Astra’s 83.24% - no measurable advantage.
- Third-party evaluation by Irregular: on the challenging FrontierCyber suite, Cyber solved 90 of 288 tasks, while Astra solved 96.
Exploit generation: where the biggest leap occurs
ExploitGym evaluates whether a model can exploit a specified vulnerability in a remote application. A task is marked successful only if the model captures the flag via code execution using the intended vulnerability. OpenAI presented five configurations, and this breakdown provides critical context:
| Model and configuration | ExploitGym |
|---|---|
| GPT-6 Astra with default guardrails | 0.0% |
| GPT-5.6 (Daybreak Blue) | 30.3% |
| GPT-5.6 Cyber | 32.1% |
| GPT-6 Astra (Daybreak Blue) | 42.4% |
| GPT-6 Cyber | 48.5% |
Standard Astra scores zero because it refuses the prompts. When certain refusal layers are lifted under Daybreak Blue, that same Astra model achieves 42.4%. Most of the raw offensive capability was already present in the foundational model; specialized fine-tuning added roughly six percentage points. The full system card notes precise values of 48.45% and 42.43%.
Under ExploitGym Hardened, targets feature address space layout randomization (ASLR), stack canaries, the V8 sandbox, and tightened kernel parameters. Cyber reaches 36.94%, while Astra achieves 29.69%. The performance delta widens as target defenses become more rigorous.
RemoteExploitBench is a newer benchmark introduced by OpenAI: known vulnerabilities in applications like GitLab, PostgreSQL, and Jenkins, hosted in controlled network environments for remote exploitation. GPT-6 Cyber completed 95% of the tasks, with its single failure being an exploit payload that could not be triggered over the network. Astra with Daybreak Blue refused 100% of these tasks, highlighting the explicit operational intent behind Daybreak Red.
Vulnerability discovery and reverse engineering
| Benchmark | What it evaluates | GPT-6 Cyber | GPT-6 Astra |
|---|---|---|---|
| Web Penetration Testing | Vulnerability discovery in black-box web apps (recall) | 40.3% | 32.0% |
| SEC-Bench Pro | Discovering bugs in V8 and SpiderMonkey JavaScript engines | 87.02% | 85.38% |
| SRE-Bench | Compiled binary analysis across six simultaneous targets | 92.84% | 87.98% |
| ExploitBench (ACE) | Moving from a known flaw to arbitrary code execution | 100% | - |
| ExploitBench, internal split (Jun-Aug 2026) | Exploiting newly disclosed zero-day flaws in V8 and Chrome | 42% | 39% |
Regarding ExploitBench, OpenAI notes that the underlying CVEs date back to 2023, meaning the model likely encountered them during pretraining. To address this data leakage, OpenAI created an internal evaluation set using vulnerabilities disclosed between June and August 2026. On that slice, performance drops sharply from 100% to 42%, with the system card acknowledging that reliably weaponizing freshly disclosed bugs remains difficult for the model. It is the most grounded metric in the entire publication.
Third-party evaluations
OpenAI collaborated with the US NIST (via CAISSI) and external security firm Irregular, which subjected the model to three proprietary benchmark suites:
- CyScenarioBench (multi-step attack chains): Cyber solved each of the 11 scenarios at least once, compared to Astra’s 8 out of 11. Cyber’s average success rate was 61%, seven points higher than Astra.
- FrontierCyber (real-world software and hardware stacks): 90 out of 288 tasks solved, compared to 96 for Astra. Successful attacks included zero-days against mobile operating systems, browsers, and cloud database software. Neither model fully breached a hardened environment, and neither solved the 15 hardest challenges in the set.
- Atomic Challenges: 22 out of 22 completed. Cyber averaged 97% on vulnerability analysis and exploitation, 98% on network exploitation simulation, and 63% on evasion (Astra scored 96%, 100%, and 66% respectively).
Irregular concluded that Cyber performs broadly on par with base Astra overall, showing an edge primarily in extended, persistent attack workflows. This serves as an important check on marketing narratives: external auditors see better stamina on complex tasks rather than an unbridgeable capability gap.
Refusals and operational alignment
On defensive security evaluation tasks carried over from Astra’s system card, Cyber fulfilled 100% of general defensive requests and patch creation, 96% of exploit generation tasks, and 97% of red teaming workflows. Beyond cybersecurity, guardrails remain strict: in biological threat evaluations, it responded safely to 99.87% of high-risk prompts and 98.27% of dual-use biology questions. All GPT-6 Cyber accounts have conservative CBRN safeguards enforced by default.
| Behavioral evaluation (lower is better unless noted) | GPT-6 Cyber | GPT-6 Astra | GPT-5.6 Sol |
|---|---|---|---|
| Targeting out-of-scope honeypots (unprompted) | 0.07% | 0% | 49.5% |
| Executing actions flagged by Auto-Review | 0% | 0% | 0.3% |
| Fabricating search results from a broken tool | 0.33% | 6.92% | 79.00% |
| Omitting tooling blocks in final output | 20.29% | 11.66% | 34.18% |
| Repeating legacy production errors | 28.40% | 26.70% | 64.53% |
| Halting on tool warnings when required (higher is better) | 73.00% | 78.38% | 57.10% |
Two behavioral metrics show minor regressions compared to base Astra. Cyber halts less frequently when encountering a tool-generated safety warning, and it more frequently omits reporting in its final response that an intermediate action was blocked. OpenAI attributes the former to increased agentic persistence, and the latter to the model having already noted the blockage earlier in its scratchpad. For an autonomous agent capable of crafting exploits, both habits warrant close operational monitoring. For a deeper look at the honeypot tests, see the analysis of the honeypot scope benchmark.
Why OpenAI rated it “Critical”
Based on these findings, OpenAI classified GPT-6 Cyber as “Critical” for cybersecurity risk under its Preparedness Framework, one tier above GPT-5.6 Cyber (“High”). Standard GPT-6 Astra already shares this Critical designation. Both models remain rated “High” for chemical and biological risks, and below “High” for autonomous AI self-improvement capabilities.
For a side-by-side comparison with Gemini 4 Carbon (which reports 54.8% on ExploitGym), see our breakdown of GPT-6 Cyber vs Claude vs Gemini. Detailed technical overviews are collected in the GPT-6 Cyber hub.
Google · Your sourcesQuantslant higher in your Google - one clickAdd to preferred sources →Sources
- OpenAI, GPT-6 Cyber System Card, Sections 3, 4, and 6-10
- OpenAI, Introducing GPT-6 Cyber, ExploitGym performance charts
More on this from QUANTSLANT
benchmarks·Gemini Omni 1.2 vs Seedance, Kling, Veo and Wan Video TestSix AI video models tested on identical camera rotation and dialogue prompts. See how Gemini Omni 1.2 compares to Seedance, Kling, Veo and Wan.