AI Detectors: How to Read Results and Check Text Properly

👁 115 reads

We treat AI detectors as tools that signal the need for further text verification. We do not consider their output as standalone proof of authorship. Before drawing conclusions, we check what the report actually means, which languages are supported, and how the material was created. This matters especially when the assessment affects article acceptance, assignment grading, or accusations against an author.

What AI Detectors Actually Show

First, we read the documentation of the specific service. We do not assume that a percentage on screen means the probability of AI use. In a Turnitin report, it indicates the proportion of analyzed text classified as likely generated or appropriately transformed by AI. That is a different meaning than certainty of cheating.

Turnitin also acknowledges the possibility of incorrectly identifying human-written text and generated content. The company advises against using the result as the sole basis for actions against a student. For us, this means a conversation and verification of additional materials are necessary.

Source for report interpretation and limitations: Turnitin, “Using the AI Writing Report”, guides.turnitin.com

We also do not equate AI detection with plagiarism checking. Turnitin describes the AI detection result as independent from the similarity score. When we want to assess article integrity, we separately check sources, citations, and the origin of borrowed fragments. Text style alone does not answer these questions.

What Research Shows and What We Don’t Generalize

In the paper “GPT detectors are biased against non-native English writers,” Weixin Liang and co-authors described false flagging of English texts by people for whom English was not their native language. The study concerned specific tools and samples. We do not automatically transfer its results to any detector or to texts in other languages.

Source: Weixin Liang et al., “GPT detectors are biased against non-native English writers”, arxiv.org

In the publication “Different Time, Different Language: Revisiting the Bias Against Non-Native Speakers in GPT Detectors,” Adnan Al Ali, Jindřich Helcl, and Jindřich Libovický examined the topic including in the context of Czech. In the detectors studied, they found no systematic bias against non-native speakers. The authors point to the importance of language and study conditions.

Source: Adnan Al Ali, Jindřich Helcl, and Jindřich Libovický, “Different Time, Different Language: Revisiting the Bias Against Non-Native Speakers in GPT Detectors”, arxiv.org

Our conclusion is limited: we assess a specific tool in a specific application. We do not write that all detectors are useless. We also do not write that a positive result closes the case. To assess text in any given language, we would need data concerning that language and similar types of materials.

How We Would Check Suspicious Text

1. Record the tool name, settings, and full report. Do not base assessment on a percentage screenshot taken out of context.
2. Check whether the service supports the language and type of material being examined.
3. Read the highlighted fragments and verify the facts stated in them.
4. Ask the author to explain their working method, sources, and available drafts.
5. Compare explanations with document history, if preserved.
6. Separate confirmed problems from assumptions about the tool used.

If we find a fabricated citation, we can point to a specific error regardless of authorship. If the text is correct but the detector flagged it, the result alone does not give us grounds to construct a history of its creation. Lack of drafts is also not sufficient for us to draw such a conclusion.

Ready-Made Prompt for Content Verification

Analyze the following text for accuracy.
Do not assess authorship based on style.
List claims requiring sources, inconsistencies, and citations to verify.
Separate errors that can be demonstrated from matters you cannot establish.
Do not invent sources. For each problem, indicate the text fragment.
Text: [paste material]

We would use such a prompt as an aid. Ultimately, we would open sources and check the indicated places. A chatbot’s response does not replace a report from actual verification. We also do not ask it to recognize the author based on “AI sound.”

What Not to Expect

• Do not expect error-free determination of authorship. Turnitin documentation explicitly allows for mistakes.
• Do not treat a low score as confirmation of factual accuracy or correct citation.
• Do not accept results for English as proof of effectiveness for other languages.
• Do not average percentages from different services without knowing their meaning.
• Do not assume the report will distinguish every form of human-tool collaboration, from proofreading to writing an entire draft.

How We Would Compare Tools Before Use

We would prepare a set of materials with known origin, similar to the texts we need to assess. We would also preserve information about editing and translation. We would not mix essays, short product descriptions, and academic texts in one result without separating these categories.

We would separately record cases of wrongly flagging human work and failing to detect generated text. We are interested in both types of errors. Before the trial, we would also establish what action the report should trigger. To direct a fragment to manual review, we need a different basis than to make an accusation against an author. Even with good results from our own trial, we would maintain this distinction.

What We Would Do After False Flagging of Our Text

We would request the full report and assessment criteria. Then we would gather available notes, source materials, and successive versions of the document. We would also explain whether we used proofreading, translation, or generation of fragments. We would not change the text just to fit the indicator.

When assessing someone else’s work, we would first establish the permissible scope of AI use. We discuss permitted language correction differently than presenting someone else’s or generated text as independent work. The rules should be known before starting the assignment.

AI detectors can help us select a fragment to check, but we do not entrust them with independently determining authorship. If we want to assess text fairly, we combine the report with verification of content, sources, and the writing process. Only on this basis do we formulate a conclusion.

Related Topics

If you are looking for a related application, we have separate articles: how artificial intelligence works. Each has a section on tool limitations, because that usually determines whether something is worth implementing.

// read moreHow to Tell if AI Wrote a Text: 7 Signs and How to CheckGuidesAI Detector: What Your Results Really MeanGuidesAI Detectors: How They Work, Which Are Reliable & What to DoGuides

Kinetyra