QUANTSLANT_
Benchmarks

GPT-6 Cyber Benchmarks: What the System Card Really Measures

quantslant@ai:~$ cat news_1

› The GPT-6 Cyber system card features over a dozen benchmarks, and most point to the same conclusion: the model is substantially ahead of its predecessor, slightly ahead of base GPT-6 Astra, and no better than Astra at general software engineering.

quantslant@ai:~$ cat news_2

› I gathered all the numbers from the official announcement and the system card in one place, detailing what each test actually measures.

quantslant@ai:~$ cat news_3

› A caveat upfront: these are OpenAI's reported figures.

quantslant@ai:~$ cat news_4

› I have not run the model myself, as access is restricted to approved organizations in the Daybreak Red program.Key takeawaysExploitGym: 48.5% compared to 32.1% for GPT-5.6 Cyber, representing a 51% relative gain.RemoteExploitBench: 95% of tasks solved; GPT-6 Astra with Daybreak Blue refused 100% of them.Standard software engineering (ExpertSWE): 82.95% versus Astra's 83.24% - no measurable advantage.Third-party evaluation by Irregular: on the challenging FrontierCyber suite, Cyber solved 90 of 288 tasks, while Astra solved 96.Exploit generation: where the biggest leap occursExploitGym evaluates whether a model can exploit a specified vulnerability in a remote application.

quantslant@ai:~$ open --full
Read more

GPT-6 Cyber Benchmarks: What the System Card Really Measures

Open article on quantslant.com