How AI detectors work

1. 1

   Score predictability

   A language model reads the text and asks, word by word, how likely each word was. AI models pick likely words, so their text is unusually predictable.

2. 2

   Measure variation

   People mix short and long sentences and change rhythm. AI drafts are more even. Detectors call this variation burstiness.

3. 3

   Classify

   A model trained on large sets of labeled human and AI texts combines these and other signals into a probability, often with sentence highlights.

Perplexity measures how surprised a language model is by each word. After “The meeting has been moved to”, the word “Thursday” is expected and “the greenhouse” is not. A text full of expected words has low perplexity. AI models are built to choose likely words, so their writing scores low.

The catch: careful, formal human writing is predictable too. Lab reports, legal text, cover letters and essays by people writing in a second language often use common words in common orders, which is one reason human writing gets flagged.

Burstiness describes how much sentence length and structure change across a text. People write a long sentence, then a short one. They start one sentence with “But” and the next with a date. AI drafts tend to keep a steady rhythm: sentences of similar length, each paragraph opening with a topic sentence and closing with a summary.

Structure matters as much as wording. A run of short, parallel bullet points, or a five-paragraph essay that recaps itself at the end, looks machine-made even when every word is fine.

Most commercial detectors now use their own trained models rather than perplexity alone. They learn from large sets of examples where the answer is known, and they rarely publish exactly how they decide. That is why two detectors can give the same text very different scores.

Watermarking is a different approach: the AI model marks its text as it writes, in a pattern of word choices a checker can test for. Google DeepMind has published SynthID Text, which works this way. A watermark can only be checked for text from models that add one.

• OpenAI’s own detector. OpenAI released an AI classifier in January 2023. It correctly labeled 26% of AI-written text as likely AI and wrongly labeled 9% of human text as AI. Its update note reads:

  “As of July 20, 2023, the AI classifier is no longer available due to its low rate of accuracy.”

  Source: OpenAI, New AI classifier for indicating AI-written text, 2023

• Non-native English writers. A 2023 Stanford study ran 91 TOEFL essays written by people learning English through 7 detectors. On average the detectors labeled more than half as AI-generated, and all seven agreed on 18 of them. The authors conclude that

  “these detectors consistently misclassify non-native English writing samples as AI-generated, whereas native writing samples are accurately identified”

  Source: Liang et al., Patterns, 2023

• Languages other than English. In our test, ZeroGPT gave 20 English AI drafts an average score of 97% AI and flagged all of them. For 10 Spanish and Portuguese AI drafts the average was 76%, and one was read as human before any rewriting.
• Detectors disagree. Each detector is trained differently, so the same text can score high on one and low on another. A single score is one opinion, not a verdict.

• Writing in a second language. A smaller working vocabulary means more common words, which reads as predictable.
• Formal templates. Lab reports, abstracts, cover letters and policies follow fixed patterns.
• Short texts. A few sentences give a detector little to go on, so scores swing.
• Heavy editing tools. Grammar and rewrite tools smooth rhythm and word choice toward the average.
• Generic topics. An essay that says what every essay on the topic says, in the usual order, looks like a model’s first draft.

• Ask which detector was used and what the score was. A percentage is a probability, not evidence of who wrote the text.
• Show your process: outlines, notes, sources and version history in Google Docs or Word.
• Offer to talk through your argument. Explaining your own reasoning is the clearest proof that the work is yours.
• Check the policy. Turnitin itself says its AI score should not be the only basis for action against a student.