How we test

This page explains where the numbers on Quantslant come from: tests, prices, rankings and status data. Every result reported in an article should be traceable to one of the sources below.

Hands-on tests

• Same task, same input. When we compare models or tools, each one gets the same prompt, the same files and the same settings. Articles show the prompts, so you can repeat the test.
• Dated results. Every test carries the date it was run and the model version or plan that was used. Models change after release, so an old result is a snapshot, not a verdict.
• Things that can be checked. Where possible we count verifiable things - correct answers, facts found or missed, errors in code, response time, the cost of a run - instead of grading by impression. When a judgement is subjective, such as image quality or writing style, the article says so and shows the outputs.
• Local models run on a desktop PC with an NVIDIA RTX 4090 (24 GB VRAM). Articles about local models state the hardware, settings and versions used.
• Language. Some tests were first run for our Polish-language sister publication. When the language of a test matters for the result, the article says which language was used.

Prices

• API prices on model price pages, comparisons and cheapest-model lists are read automatically every day from the OpenRouter price list: price per million input and output tokens, excluding VAT. Each page shows the date of the data. A provider’s own rate card can differ from OpenRouter; when an article relies on it, it links to it.
• Subscription prices (ChatGPT, Claude, Gemini and others) come from official pricing pages, in US dollars, with the date they were checked. Local prices and taxes vary.
• Cost examples. When an article says what a task costs, it shows the token counts and the price used, so the calculation can be checked. The API cost calculator uses the same daily prices.

Rankings and benchmarks

• AI model ranking - scores from the Artificial Analysis Intelligence Index, fetched automatically twice a day. We do not adjust the scores.
• Vendor benchmarks - numbers published by model makers are reported as the vendor’s claims, with a link, and kept apart from our own measurements.
• Our own benchmarks - for example the context window test, which checks whether a model finds facts hidden at different depths of a long document. The method is described on the test page.
• AI service status - data from official provider status pages, refreshed every few minutes, plus our own hourly checks.

What we don’t do

• No made-up numbers or quotes. If a source does not give a figure, neither do we.
• No paid results. Companies cannot pay for a better outcome in our tests or rankings. Sponsored content, when it appears, is labelled as sponsored.
• No promises beyond the test. A test shows how a tool behaved on our tasks on a given day. For legal, financial or medical decisions, check the original sources and ask a professional.

Spotted a problem?

If a number looks wrong or a test cannot be reproduced, write to promptowy@gmail.com. What happens next is described in our corrections policy.