AI Detectors Explained: How AI Text Checkers Work and Why They Sometimes Get It Wrong

Somewhere in your life there is probably a score you cannot see: a percentage attached to something you wrote, estimating the probability that a machine wrote it instead. AI detectors — tools that claim to distinguish human writing from AI-generated text — now sit inside school submission systems, hiring pipelines and publishing workflows. They are also, by the admission of the institutions that tested them hardest, unreliable in ways that have ended real students’ and writers’ good names. This guide explains how the detectors actually work, what the published evidence says about their accuracy, why false positives happen and to whom, and how schools and employers really use the scores. The honest summary: AI detection is a probability estimate with documented failure modes, not a verdict — and the difference matters enormously when the estimate is wrong. For the technical mechanism in depth, see how AI detectors work; this pillar is the full picture.

What an AI detector actually is

Strip away the branding and every detector is a statistical instrument pointed at text. It does not contain a registry of AI outputs, cannot compare your essay against “what ChatGPT would say” directly, and has no access to how the document was produced. It measures properties of the finished text — how predictable its word choices are, how its sentences vary, how closely it resembles labelled examples of machine writing — and converts those measurements into a score. Turnitin, the largest player in education, describes its scale plainly: writing is scored sentence by sentence, and the document score is the percentage of sentences scored as likely AI-written. GPTZero, the best-known independent tool, explains its own measurement in terms of perplexity and burstiness before describing its newer machine-learning approach. The crucial point for everything that follows: a detector outputs the likelihood that text resembles machine writing. Resemblance is not authorship, and the distance between the two is where all the controversy lives. A student who writes carefully, formally and predictably — as students are taught to do — produces text that resembles machine output for structural reasons that have nothing to do with how it was written.

The three mechanisms, in plain English

Perplexity is a “surprise meter,” in GPTZero’s phrase: how unexpected is each word given the words before it? Language models write by choosing the most probable next words, so their text tends to be unsurprising; human writing wanders, digresses and picks odd, personal words, so it tends to surprise more. Burstiness measures variation — humans alternate short punches with long winding sentences, while models generate at a more even tempo, and detectors compare the pattern of variation against norms. Classifiers skip hand-crafted measures: a model is trained on large collections of labelled human and machine text and learns whatever statistical fingerprints separate them, then classifies new documents, sometimes sentence by sentence. Turnitin’s FAQ describes the same underlying observation all three rest on: “LLMs are designed to predict the next most likely word in a sequence, and they consistently do so,” while humans choose words “in a much more idiosyncratic way.” GPTZero says it moved from pure perplexity-and-burstiness measurement to a deep-learning architecture in autumn 2023, adding sentence-level classification — a migration the whole field has made as simple measures proved too easy to fool and too prone to error. The deep dive into each mechanism, including why paraphrasing breaks them, is here.

Accuracy: what the evidence actually says

Three bodies of evidence exist, and they do not agree, which is itself the most important fact in this guide. The vendors’ own testing reports reassuring numbers: Turnitin states a false positive rate “of less than 1% for documents with over 20% AI writing,” validated before every model release against more than 700,000 academic papers written before ChatGPT existed — that is, documents guaranteed human. GPTZero claims 99% accuracy distinguishing AI from human text with a 1% false positive rate, and 95.7% of AI text identified on the RAID benchmark. Independent academic testing is colder: Weber-Wulff and colleagues, testing 14 detectors in the International Journal for Educational Integrity in 2023, found none more than 80% accurate and rated most below that, concluding the tools are “neither accurate nor reliable” — and that accuracy collapsed further on paraphrased machine text, the realistic evasion case. The technology’s own creator delivered the bleakest data point of all: OpenAI launched its own AI Text Classifier in January 2023, reporting it caught only 26% of AI-written text while falsely labelling 9% of human text, and withdrew it that July — its announcement page still carries the note that it is “no longer available due to its low rate of accuracy.” The full sourced comparison — vendor claims, the RAID benchmark, the university studies — is assembled in how accurate are AI detectors? The fair reading of all three together: detectors catch naive, unedited AI output fairly well, struggle with edited or paraphrased output, and carry a small but real false positive rate whose consequences are anything but small.

False positives: the error that lands on people

A false positive means the detector flags writing that a human produced unaided. The reason it deserves its own section — and its own article, AI detector false positives — is asymmetry: a missed AI essay is an institutional embarrassment, but a false accusation is a personal crisis, and the burden of proof tends to land on the accused. The published research maps exactly who bears the risk. The Stanford-led study by Liang and colleagues (Patterns, 2023) ran seven detectors over 91 TOEFL essays written by non-native English speakers and 88 essays by US eighth-graders. The US essays were classified near-perfectly; the TOEFL essays were misclassified as AI-generated at an average rate of 61.22%, with all seven detectors unanimously flagging nearly one in five — because second-language academic writing uses a narrower, more predictable vocabulary, which is the precise statistical signature of machine text. The study’s authors conclude the detectors are biased against non-native writers. Scale completes the picture: Vanderbilt’s famous calculation noted that at Turnitin’s stated sub-1% false positive rate, its own 75,000 annual paper submissions implied roughly 750 papers potentially wrongly labelled per year. Small percentages, applied to millions of documents, become thousands of people. Turnitin itself states the correct institutional posture: its tool “does not determine misconduct,” the score “should not be used as the sole basis for adverse actions,” and final decisions belong to educators who know the student.

What schools and employers actually do with the scores

Practice varies more than the marketing suggests, and our companion piece documents it in detail. The pattern across education is a split: many universities adopted Turnitin’s detector because it arrived inside a tool they already used, while a significant minority — Vanderbilt most visibly, in a public 2023 explanation — switched it off, citing the lack of insight into how it works and the false-positive arithmetic above. Where detectors remain on, stated policy at most institutions treats the score as a prompt for conversation and further evidence — drafts, version history, an oral discussion of the work — rather than an automatic finding; Turnitin’s guidance to instructors pushes the same direction, and notably, only instructors and administrators can even see the AI indicator. Employers use the tools differently and more loosely: screening cover letters and writing samples in hiring, and checking submitted content in publishing and marketing, usually without any of education’s due-process habits. And a large share of both worlds quietly uses detectors for the least discussed purpose: as a deterrent, where the existence of the score matters more than its precision. Readers navigating a specific flag — a student, a teacher, an applicant — will find the practical questions collected in our detector FAQ. One boundary this site observes throughout: we explain how detection works and what the evidence shows about fairness; we do not publish techniques for disguising AI text to deceive a checker, because the legitimate problem is the score’s reliability, not its inconvenience.

What this means

The spread of AI detectors is best understood as institutions automating a suspicion they cannot otherwise process. Schools cannot interview every student about every essay; employers cannot hand-verify every cover letter; so a percentage steps in where judgement used to be, and the percentage’s errors become the system’s errors — distributed, as Liang’s study shows, with statistical precision onto the writers least able to contest them: second-language students, formula-trained writers, anyone whose honest style is careful and predictable. For US readers the stakes are concrete, because American education and hiring both run on records: an academic-integrity finding follows a student, and a quiet rejection follows an applicant, with none of the visibility that would let the falsely flagged appeal. The hopeful reading is also in the sources above. The vendors themselves — Turnitin’s “not the sole basis” rule, GPTZero’s “no detector is perfect” — now publish the caveats; a flagship university disabled the tool on explicit fairness arithmetic; and the company that built ChatGPT withdrew its own classifier for inaccuracy. The technology consensus, unusually, includes its critics. What remains is for the humans operating the systems to act on what the evidence already says: a detector score is a reason to look closer, never a reason to stop looking. Students are entitled to processes that examine drafts and histories rather than percentages; institutions owe their members the Vanderbilt calculation before adoption, not after the first wrongful flag. Until that discipline is standard, understanding these tools — their real mechanisms, their published error rates, their documented bias — is not academic curiosity. It is self-defence for anyone whose writing passes through a machine’s judgement, which is now nearly everyone.

A practical starting point follows for each side of the desk. Students: assume any institution you write for may run a detector, and keep the artefacts of your process — drafts, version history, notes — the way earlier generations kept receipts. Teachers and managers: before the next flagged document arrives, decide what your process does with a score, write it down, and publish it; the worst outcomes in this field have come from improvised responses to a number nobody had agreed how to read. Neither step requires new software or new policy from anyone above you. Both convert a tool famous for producing anxious ambiguity into what it should always have been — one input among several, inside a process a human being owns.

Related Guides

• How Do AI Detectors Work? Perplexity, Burstiness and Classifiers in Plain English
• How Accurate Are AI Detectors? What Studies and the Vendors’ Own Tests Actually Show
• AI Detector False Positives: Why Human Writing Gets Flagged and What It Can Cost
• What Schools and Employers Actually Do With AI Detectors
• AI Detector FAQs: Turnitin, GPTZero and the Questions Students and Teachers Ask Most

Bottom line

Detectors automated a suspicion institutions could not process by hand — and their errors became the system’s errors. Use the score as a question, insist on a fair process around it, and never let a percentage be a verdict.

Read Next

• How to Check AI Answers: A Fact-Checking Checklist That Catches Bad Claims
• ChatGPT Explained: What It Is, How It Works and How to Use It Well

Sources

• Turnitin’s AI writing detection capabilities — FAQs (accessed 7 November 2026)
• How AI Detectors Work — GPTZero (accessed 7 November 2026)
• Liang et al. (2023), “GPT detectors are biased against non-native English writers,” Patterns 4(7) (accessed 7 November 2026)
• Guidance on AI Detection and Why We’re Disabling Turnitin’s AI Detector — Vanderbilt University (accessed 7 November 2026)