How Do AI Detectors Work? Perplexity, Burstiness and Classifiers in Plain English

Every AI detector gives you a number. Almost none of them explain, in the moment you need it, what the number is a measurement of. This article takes the machinery apart: perplexity, burstiness and classifier models — the three mechanisms the industry is built on — in plain English, using the vendors’ own explanations (GPTZero’s guide to how detectors work and Turnitin’s detection FAQ) and the peer-reviewed literature that tested them. Once you see the mechanism, the famous failures stop looking like mysteries and start looking like arithmetic. For the accuracy evidence and the policy picture, read the pillar, AI Detectors Explained, after this one.

The core idea: predictable text looks machine-made

Start from how the text being detected is produced. A large language model writes by predicting, at every step, the most likely next word given everything so far — Turnitin’s FAQ puts it as: “LLMs are designed to predict the next most likely word in a sequence, and they consistently do so,” while humans select words “in a much more idiosyncratic way.” Detection is the attempt to run that process in reverse. If a passage consistently consists of the words a model would most probably have chosen, the passage probably came from a model — or from a human writing in a model-like way. That “or” is not a footnote; it is the entire technical story of AI detection. Every mechanism below is a different instrument for measuring the same underlying property: how model-like the statistical texture of the writing is. None of them observes the writing being done. None of them can. The detector sees a finished artefact and infers a process, and inference fails wherever two different processes produce similar artefacts — which, in writing, is constantly.

Perplexity: the surprise meter

Perplexity is the oldest and most intuitive measure. GPTZero, which says it pioneered the term’s use in detection, defines it as “how surprising the text is to a language model” — if the model finds the text familiar and predictable, perplexity is low; perplexity over a whole document is “the average of its perplexities over each sentence.” Walk through it concretely. Given “The committee decided to postpone the…” a language model assigns high probability to “meeting” or “vote” and low probability to “pineapple.” A writer who consistently takes the high-probability branch produces low-perplexity text. Chatbots do this by construction — probability is literally their selection rule — so their output clusters at the predictable end. Human writers also take probable branches often (most sentences are ordinary), but they veer: personal references, odd metaphors, local knowledge, mistakes, flair. The trap is now visible: anyone writing conservatively — a scientist following a journal’s house style, a student taught a five-paragraph template, a non-native speaker using the vocabulary they are sure of — writes low-perplexity prose by training or necessity. Liang and colleagues proved the consequence experimentally in Patterns (2023): essays by non-native speakers that detectors flagged had significantly lower perplexity, and the flagged rate for those essays averaged 61.22% across seven detectors. Perplexity measures predictability faithfully. It just cannot tell you whose predictability it is.

Burstiness: the rhythm measure

Burstiness looks at shape rather than word choice: how much the text varies in sentence length and complexity as it goes. GPTZero describes it as a measure of text variation, observing that human-written sentences show greater variation while AI writing stays “more consistent.” Human prose really does lurch — a two-word sentence. Then a long, clause-stacked, comma-heavy unwinding that takes its time because the thought itself is unpacking as it goes. Short again. Models, sampling word by word at a steady confidence, tend toward a metronome: medium sentences, evenly built, paragraph after paragraph. (This paragraph is demonstrating the human pattern; an unedited chatbot paragraph on the same topic would feel noticeably smoother.) Burstiness is a genuinely useful signal for naive machine output, and it explains the folklore that “adding some short sentences” changes scores — rhythm is one of the measured features. But notice the same symmetry as perplexity: disciplined human styles suppress burstiness on purpose. Legal writing, technical documentation, academic prose in a second language, journalism on a style guide — all deliberately flatten rhythm. A detector cannot distinguish a flattened style that was taught from a flattened style that was sampled. The mechanism is honest about what it measures; trouble starts when institutions read the measurement as more than it is.

Classifiers: learning the fingerprint instead

The modern industry mostly runs on the third mechanism: train a machine-learning model on large labelled collections of human and AI text, and let it learn whatever statistical features separate the two — vocabulary habits, punctuation patterns, syntactic preferences, subtler signals no one hand-designed. GPTZero says it adopted a deep-learning architecture in autumn 2023, processing text in sequential layers with sentence-level classification, and now also cross-checks text against web sources to catch copy-paste generation. Turnitin’s FAQ describes the same family of approach from the education side: likely-AI sentences receive scores between 0 and 1 evaluated in the context of the surrounding document, and the document’s headline figure is the percentage of sentences above the AI threshold. Classifiers outperform the hand-crafted measures — they catch more, subtler signals — and inherit two structural weaknesses instead. First, they are only as representative as their training data: Turnitin notes pointedly that its data includes writing by second-language learners precisely to minimise bias, an admission that a classifier trained on the wrong corpus learns the wrong boundary. Second, they are opaque: a classifier’s sentence score arrives without an explanation a student can examine or contest, which is why Vanderbilt cited having “no insight into how it works” among its reasons for switching Turnitin’s detector off. The mechanism-by-mechanism accuracy record — vendor benchmarks against independent tests — is assembled in how accurate are AI detectors?

Where all three mechanisms break

The failure modes follow from the designs. Short text: statistics need volume; GPTZero warns that short passages are harder to evaluate and more prone to error, and Turnitin only displays scores at all above a 20% AI-writing threshold and requires at least 300 words of evaluated sentences before scoring — the vendors themselves fence off the unreliable zone. Edited or paraphrased machine text: human revision injects exactly the surprise and variation the measures look for, so performance collapses — Weber-Wulff and colleagues found precisely this in their 2023 test of 14 tools, where accuracy on paraphrased AI text fell sharply. Predictable human text: the mirror failure, documented in the Patterns study and explored fully in AI detector false positives. Mixed documents: a human essay with one AI-drafted paragraph resists the whole document-level verdicts the tools print, which is why sentence-level scores exist — and why a single percentage for a long document should always be read as an average of local judgements, not a measurement of the whole. Understanding these edges is not a route to gaming them — the same edits that change a score are called “rewriting” when a human does them to their own draft, and this site’s position is that the fairness problem is the story, not evasion technique. It is, instead, the knowledge a teacher needs before acting on a score, and a student needs before panicking about one. The institutional picture — who uses scores and how — is in what schools and employers actually do with AI detectors.

What this means

The mechanisms are not frauds; they are measurements marketed past their meaning. Perplexity really does capture something true about how models write, burstiness captures something true about human rhythm, and classifiers genuinely separate unedited chatbot output from typical human drafts most of the time. The failure is categorical: a measurement of textual resemblance is being consumed, in too many classrooms and hiring pipelines, as a determination of personal conduct. For US readers — students submitting through Turnitin, applicants whose cover letters pass through a checker, teachers handed a dashboard — the mechanism knowledge in this article converts into three durable rules. A low score is not proof of innocence and a high score is not proof of guilt; both are statements about style statistics. Scores are least trustworthy exactly where the stakes literature shows they are used most confidently: on short, formal, second-language or heavily templated writing. And the only trustworthy adjudication of authorship remains the boring evidentiary one — drafts, version history, process, conversation — which is why institutions like Vanderbilt concluded the detector added risk rather than removing it. Detectors will keep improving at the easy end of their job, catching copy-pasted chatbot essays. The hard end — judging careful writers fairly — is not an accuracy problem awaiting a patch. It is built into what the instruments measure, and no version number changes what a surprise meter is.

None of this makes the mechanisms useless — it locates their legitimate use. As triage, a detector can tell an overworked instructor which of two hundred submissions deserves a closer human look first, in the same way a spell-checker’s squiggle prioritises attention without proving an error. As research instruments, perplexity and burstiness genuinely illuminate how machine prose differs from human prose at population scale, which is how the Liang study could use detector outputs to expose the bias in the first place. The instruments become harmful at exactly one moment: when their output is relabelled from “this text resembles machine writing” to “this person cheated,” crossing an evidentiary gap the mechanism cannot bridge. Readers who hold that distinction — who can say “the score measures resemblance, and resemblance is not authorship” and insist that processes honour it — understand these tools better than most of the institutions currently deploying them, and are equipped to insist on the fairer reading wherever a score next appears.

Bottom line

Detectors measure resemblance, not authorship. Perplexity and burstiness are real signals — they are simply not verdicts, and anyone using a score as one has misunderstood the instrument.

Read Next

• How Accurate Are AI Detectors? What Studies and the Vendors’ Own Tests Actually Show
• AI Detectors Explained: How AI Text Checkers Work and Why They Sometimes Get It Wrong

Sources

• How AI Detectors Work — GPTZero (accessed 7 November 2026)
• Turnitin’s AI writing detection capabilities — FAQs (accessed 7 November 2026)
• Liang et al. (2023), “GPT detectors are biased against non-native English writers,” Patterns 4(7) (accessed 7 November 2026)