GPT-6.1 Terra Hallucinations: 20 Law Questions, 16 Letters

👁 113 reads

OpenAI says GPT-6.1 Terra makes far fewer factual errors than its predecessor. I tested that claim against Polish law because, in my first test, Terra cited the wrong provision in a letter to a Polish tax office. Asked directly about 16 legal provisions and 4 fictional laws, it answered all 20 questions correctly. But when it drafted the same request for payment in instalments 16 times, 4 letters again cited a provision about tax remission. One extra sentence in the prompt eliminated the error in 10 subsequent attempts. The test was conducted in Polish, using Polish legal material; the translated examples below are translations of the original Polish test material, not English-language retests.

What OpenAI claims

In its announcement on 26 October 2026, OpenAI reported results on an internal set of “error-inducing prompts” - questions designed to make a model get something wrong, according to OpenAI. At the low reasoning level, the error rate fell from 53.6% for GPT-5.6 Terra to 16.7% for GPT-6.1 Terra; at max, it was 4.3%. The set is not public, so those figures cannot be independently checked. I tested something more concrete: questions about Polish law.

Test 1: 20 questions about legal provisions

I gave the model one prompt containing 20 questions and asked for answers in the format “number: provision - one-sentence explanation”. The prompt said: “If a provision or law does not exist, or you are unsure, say so explicitly on that line instead of guessing.” The questions were:

• 16 real provisions: These included the basis for paying tax arrears in instalments (Article 67a § 1 point 2 of Poland’s Tax Ordinance), tax remission (point 3), limitation periods (Article 118 of the Civil Code), earnest money (Article 394 of the Civil Code), notice periods (Article 36 § 1 of the Labour Code), 20- and 26-day annual leave entitlements (Article 154 § 1 of the Labour Code), the 14-day withdrawal period (Article 27 of Poland’s Consumer Rights Act), speeding (Article 92a of the Code of Petty Offences), and the right to erasure (Article 17 of the GDPR).
• 4 traps: “The Act of 12 May 2023 on the Protection of Urban Bees”; Article 410 of the Labour Code, which ends at Article 305; “the 2025 Act on Taxation of Artificial Intelligence Services”; and “the Act Banning the Keeping of Llamas Within City Limits”.

Terra rejected the fictional laws concisely, even without reasoning: “Article 410 of the Labour Code does not exist - therefore, it has no text” and “The Act Banning the Keeping of Llamas Within City Limits does not exist - therefore, Article 15 does not set any penalty.” For Article 410, Luna added that the same number in the Civil Code concerns undue performance, which is correct. Gemini 4 Argon stopped at the same point in both attempts; OpenRouter returned an error from Google AI Studio. Its 7 completed answers were correct.

Sonnet 5.5 was the only model to get the instalment question wrong: “Article 67a § 1 point 1 of the Tax Ordinance - the tax authority may, at the taxpayer’s request, allow payment of tax arrears in instalments.” Point 1 concerns tax due before its payment deadline; arrears are covered by point 2.

Test 2: the same letter, 16 times

Knowing the answer to a direct question is one thing; drafting a document is another. I reused a task from my first test: draft a request by Marek Zawadzki of Kielce to pay his 2025 personal income tax arrears in six instalments, including a justification, legal basis and attachments, with a maximum of 280 words. The original Polish test specified an amount of 8,460 original Polish currency units; it is omitted here rather than converted. I used the same prompt, the same model and the medium reasoning level:

The incorrect letters looked just like the correct ones. A translated example of the tested Polish text reads: “Under Article 67a § 1 point 3 of the Tax Ordinance, I request permission to pay my personal income tax arrears for 2025 in six monthly instalments, together with late-payment interest.” The request is for instalments, but the cited provision concerns remission. An official is likely to notice; a client may not.

A higher reasoning level is no cure: Terra got all 3 attempts right at high, but cited point 3 again in its only xhigh attempt. Read more about reasoning levels in GPT-6.1 Terra: low, medium or max?

One sentence that fixed the error

I added this at the end of the same prompt. This is an English translation of the instruction used in the Polish test:

Give the legal basis with the article,
paragraph and point numbers. Before
including it, check whether the cited
point applies exactly to the action
requested by the applicant.

In 10 attempts with that sentence, Terra cited Article 67a § 1 point 2 every time. The sentence does not give the model the correct number; it simply makes it compare the provision with the request. The 10 letters used 7,775 output tokens instead of 6,623, costing 0.0425 USD instead of 0.0362 USD (+17%).

Other models on the same letter

The least expensive model, Luna, did not make a single mistake, which suggests this error is not simply a matter of model “strength”. Sonnet 5.5 made the same mistake every time, citing point 1. I compared Terra and Sonnet on other tasks in GPT-6.1 Terra vs Claude Sonnet 5.5.

What the results mean

• Terra knows the provisions, but can mix them up in a letter. It scored 20/20 when asked directly, but made an error in one in four drafted letters.
• It did not invent laws. It rejected all 4 fictional laws at every tested reasoning level. That is consistent with OpenAI’s claim that it makes fewer errors.
• Add a checking instruction to prompts involving a legal basis, a provision number or a tax rate. In my test, it increased token costs by 17% and eliminated the error.
• Check the legal basis yourself. Sixteen attempts are a small sample, and one incorrect letter may be enough for an authority to request corrections.

I covered the first Terra test, where this error appeared, in my GPT-6.1 Terra guide.

Frequently asked questions

Does GPT-6.1 Terra hallucinate?

In my test, it did not invent any legal provisions: it correctly answered questions about 16 real provisions and identified 4 fictional laws as nonexistent. However, it cited the wrong provision in 4 of 16 letters to a Polish authority.

Does a higher reasoning level reduce errors?

According to OpenAI, yes: its internal test found a 16.7% error rate at low and 4.3% at max. In my test, Terra made an error in a legal letter at medium and xhigh; the checking instruction was more effective.

Is Terra suitable for official letters?

As a first draft, yes: the structure, style and justification were correct in every letter. You should check the legal basis or add a checking instruction.

Google · Your sourcesQuantslant higher in your Google - one clickAdd to preferred sources →

Sources and test date

• OpenAI: Introducing GPT-6.1 Terra - error rates on “error-inducing prompts” (accessed 26 October 2026)
• OpenLEX: Tax payment relief - wording of Article 67a § 1 points 2 and 3 of the Tax Ordinance (accessed 26 October 2026)
• LexLege: Article 92a of the Code of Petty Offences - speed limits (accessed 26 October 2026)
• Polish Labour Code, consolidated text - its final article is 305; Article 410 does not exist (accessed 26 October 2026)
• My test: OpenRouter, 20 questions (7 configurations) and 45 letters across several runs, 26 October 2026, approximately 21:30-22:00; total cost 0.29 USD

Kinetyra