A student in Washington state writes a lab report the old-fashioned way, at midnight and on too much coffee, and gets called into an academic integrity hearing anyway. That isn’t a made-up worst case. At Washington State University, 33% of the AI-related hearing cases between 2023 and 2025 ended with the student found “not responsible,” according to a February 2026 memo from the Provost’s office. A few weeks after that memo, the university cancelled its contract for Turnitin’s AI detection.
So what’s going on? Why does software built to catch machine writing keep pointing at people? The short answer is that detectors don’t spot AI. They spot patterns that AI tends to produce, and plenty of humans write that way too.
What an AI Detector Is Actually Measuring
Most detectors work on probability. They look at your text and ask a simple question: how predictable is each next word? Language models pick likely words, one after another, so their output tends to be smooth and low on surprises. Detectors score that smoothness and call it a signal.
The trouble is obvious once you say it out loud. Lots of good human writing is predictable on purpose.
Predictable Writing Isn’t the Same as Machine Writing
Think about the kinds of text that follow strict conventions:
- Lab reports and methods sections
- Legal summaries and compliance documents
- Product descriptions with a fixed format
- Five-paragraph essays taught in high school
All of these reward plain words, standard structure and few stylistic risks. That’s exactly the profile a detector reads as “probably AI.” A careful writer who follows the rules can score worse than a sloppy one who doesn’t.
The Study That Exposed a Bias Against Non-Native Writers
The clearest evidence came from Stanford. In 2023, James Zou’s team, led by Weixin Liang, ran seven popular detectors on two sets of essays: 91 TOEFL essays written by non-native English speakers, and 88 essays by US eighth graders.
The detectors were close to perfect on the American essays. On the TOEFL essays, they fell apart.
Share of human-written TOEFL essays wrongly labelled as AI by seven detectors (Liang et al., Stanford, 2023).
The average false positive rate across the seven tools was 61.22%. Of the 91 essays, 89 (97.80%) were flagged by at least one detector, and 18 (19.78%) were flagged by all seven at once. Every one of those essays was written by a person.
Why Vocabulary Size Matters So Much
The researchers traced the problem to something called perplexity, the measure of how surprising a text’s word choices are. Non-native writers often use a smaller, safer vocabulary and simpler sentence structures. That isn’t a flaw in their writing. It’s a natural stage of learning a language. But it lowers perplexity, and low perplexity is what detectors treat as a red flag.
The team ran a telling follow-up. When they used ChatGPT to enrich the vocabulary of the TOEFL essays, false positives dropped sharply. In other words, the “fix” for being flagged was to sound fancier, not more human.
What Turnitin Says About Its Own Error Rates
Turnitin is the detector most students and teachers meet, so its numbers matter. In a June 2023 post, the company said its document-level false positive rate was under 1%, but only for documents with 20% or more AI writing. At the sentence level, it put the rate at around 4%.
Those figures sound small. They stop sounding small when you multiply them by a campus.
What “Only 1%” Looks Like at Scale
Vanderbilt University did that math in August 2023. It submitted about 75,000 papers to Turnitin the year before, so even a 1% false positive rate would have meant roughly 750 papers wrongly labelled as AI. Vanderbilt switched the detector off.
Washington State ran a similar calculation in 2026. With 148,547 assessments submitted in Fall 2024, a 1% error rate would mean about 1,485 students wrongly flagged in a single term. The memo also noted that Turnitin’s claimed 1 to 2% false positive rate is disputed.
Universities Are Quietly Walking Away
Vanderbilt was early, not alone. The University of Waterloo stopped using Turnitin’s AI detection as of September 2025. Its EdTech Hub said the tool had flagged human-written text as 100% AI-generated, and it named bias against students whose first language isn’t English as a concern.
Even OpenAI gave up on its own text classifier in 2023. By the company’s own figures, it caught only 26% of AI-written text and wrongly flagged human writing 9% of the time. A peer-reviewed study that same year by Debora Weber-Wulff and colleagues tested 14 detection tools and concluded they were “neither accurate nor reliable.”
To be fair, the tools have improved since then. A September 2025 working paper from the National Bureau of Economic Research found that the best commercial detectors now beat open-source ones by a wide margin, and some report very low false positive rates. But “better on average” still leaves real people on the wrong side of the line.
Who Gets Flagged Most Often
The risk isn’t spread evenly. A 2024 Common Sense Media survey, reported by Education Week, found that 20% of Black teens said their schoolwork had been wrongly flagged as AI-generated, compared with 10% of Latino teens and 7% of white teens. Those are self-reported accusations rather than measured detector errors, but the gap is hard to ignore.
Beyond classrooms, the same problem hits freelance writers and marketers. Clients increasingly paste delivered copy into a free detector before paying. A writer with a clean, corporate style can end up defending work they wrote line by line.
How Writers Can Protect Themselves
You can’t control which detector someone uses. You can control your paper trail and, to a degree, your style.
Keep Evidence of Your Process
Draft in a tool that saves version history, such as Google Docs or Word with tracked changes. Keep your outlines, notes and source lists. If a score is ever challenged, a timeline of edits is far more persuasive than any detector percentage.
Write With More Texture
Vary your sentence length. Use specific examples from your own experience. Let your opinions show. Detectors read uniformity as machine-made, and real voice is the best counterweight.
Handle AI-Assisted Drafts Openly
Plenty of professionals now start with an AI draft and rewrite it heavily. If that’s your workflow, disclose it where your client or employer expects disclosure. For marketing and business copy, a rewriting model such as phrasly ultra can help turn stiff draft language into more natural prose while keeping the meaning. Treat its output as a starting point you still edit and fact-check. Its published pass rates come from the company’s own September 2026 benchmark, and results vary by text. Students should follow their course policy on AI use, full stop.
Know How to Push Back
If you’re accused based on a detector score, ask what tool was used, what the score was and whether any other evidence exists. Point to the published error rates above. Many institutions now say a detector score alone shouldn’t decide a case.
It also helps to know why one detector says 8% and another says 80% on the same paragraph. This breakdown of which AI detector comes closest to Turnitin explains how the tools differ and why no free checker can mirror Turnitin exactly.
A Score Is a Clue, Not a Verdict
AI detectors aren’t useless. They can prompt a conversation or highlight a passage worth a second look. What they can’t do is read intent, and they can’t tell the difference between a machine and a person who simply writes cleanly, plainly or in a second language.
The numbers make that clear. A 61% false positive rate on non-native writers in 2023. Hundreds or thousands of possible wrong flags per campus, even at 1%. A third of hearings at one university ending in “not responsible.” Until detectors can explain their reasoning, the fairest way to treat their scores is as one clue among many. For writers, the best defense is a visible process and a voice that’s unmistakably your own.
The post Why Do AI Detectors Flag Human Writing? Inside the False Positive Problem appeared first on The Hype Magazine.

1 week ago
13
