Gemini 4 Carbon test: 50+ tasks vs Argon, Claude Fable 5.2 and GPT-6.1 Sol

👁 112 reads

On the day Gemini 4 Carbon became available to everyone (10 November 2026) I ran it through more than 50 tasks and put the same tasks to Claude Fable 5.2, GPT-6.1 Sol and Gemini 4 Argon. Every call went through OpenRouter with the same settings, and every answer was graded by unit tests, an answer key or an automatic checker - not by my impression. Here is what Carbon can do, what it costs and where it trips up.

Prices first

Results on the same tasks

* The grader deducted one point for a correct grammatical form - its own mistake, the same for both Gemini models.

What Carbon did well

It did not invent sources. I asked for 8 peer-reviewed publications with DOIs on two topics. All 16 of Carbon’s papers exist; I checked them in Crossref, OpenAlex and library catalogues. One DOI did not resolve and one report had the wrong year. Argon invented 2 out of 16 on the same prompt in October.

It reads documents selectively. In the agent task Carbon had three tools (list documents, read a document, run Python) and six documents, one of them an outdated 2025 price list. It skipped the outdated file based on its title alone, read the four current ones, wrote a cost function, ran it once and answered in exactly the requested format. 54 seconds, 0.03 USD.

It thinks like an analyst. From 92 lines of access log it found the brute-force IP, the compromised account, the exact second of the successful login, a CSV export 20 seconds later and a path traversal that worked through URL-encoded ..%2f. It did not confuse the blocked /etc/shadow attempt with a leak and flagged an ordinary user with one typo in his password as a false alarm.

It writes careful code. The spreadsheet engine (formulas, ranges, SUM/MIN/MAX/AVG/IF, five error codes, cycle detection and a 900-cell chain without hitting Python’s recursion limit) came out as 639 lines with its own parser and generator-based evaluation instead of recursion. All 34 tests passed.

Where it trips up

It can burn the whole token limit on thinking. On a regular expression task at medium effort Carbon used all 16,000 tokens on reasoning and returned no code (0.16 USD). With a 64,000 limit it solved it after 248 seconds for 0.17 USD. At low effort it solved the same task in 13 seconds for 0.007 USD - 25 times cheaper, same 13/13 result. On the spreadsheet at high effort it spent 47,997 tokens thinking and returned nothing.

No self-check in the plain API. When patching a vulnerable module at medium effort, its fixes were right (parameterised SQL, PBKDF2 with 600,000 iterations, a random salt, constant-time comparison), but one line had an extra space, so Python rejected the file and all 14 tests failed. At high effort it passed 14/14, in 308 seconds for 0.31 USD.

It is slow. On the same tasks Argon was between 1.4 and 8 times faster at similar accuracy.

Which one to choose

• Gemini 4 Carbon for work where mistakes cost more than time: security reviews, research with required sources, hunting a hard bug, coordinating other agents. Start at low or medium and keep the output limit high.
• Gemini 4 Argon for everyday writing, summaries and routine code. Same price per token, much faster.
• GPT-6.1 Sol if you watch costs: cheapest in practice and fast.
• Claude Fable 5.2 if you need code that works without checking: no slip in any task, but four times the price per token.

Prices, access and Google’s own benchmarks are in my Gemini 4 Carbon guide.

// read moreGemini 4 Carbon is available to everyone: price, access and first testAI ToolsClaude Fable 5.2 vs GPT-6.1 Sol vs Gemini 4 Argon: Price, BenchmarksBenchmarksClaude Fable 5.2 vs GPT-6 Astra and Gemini 4 CarbonBenchmarks

Kinetyra