Why AI Search Answers Go Wrong: Wrong Citations, Made-Up Links and Confident Mistakes

An AI search answer can be wrong in a way a list of links never could: fluently, specifically, and with a citation attached that appears to prove it. Understanding why requires the one study that took the problem apart piece by piece. In March 2025, the Tow Center for Digital Journalism at Columbia published “AI Search Has a Citation Problem” in the Columbia Journalism Review — 1,600 carefully constructed queries across eight AI search tools. Its findings remain the reference map of how AI answers fail: wrong articles credited, URLs fabricated, copies cited instead of originals, and nearly all of it delivered in a tone of complete assurance. This guide explains each failure mode, why the technology produces it, and what a reader can do about it. It is part of our pillar series on AI search explained.

The test, briefly — and its limits

The Tow Center’s method was simple and strict. The researchers took ten articles from each of twenty news publishers, extracted a direct quote from each, and asked every tool the same thing: identify this article’s headline, publisher, publication date and URL. An answer counted as correct only when all three scored attributes — article, publisher, URL — were right. Excerpts were chosen so that a plain Google search returned the original in the first three results: retrieval, on paper, should have been easy. Two limits deserve stating before the results. The test measured one narrow skill — tracing news text to its source — not general question-answering, and it ran in early 2025 on products that have since been revised. Neither limit rescues the findings, because the failures it exposed are structural, as each mode below shows.

Failure mode 1: The confident wrong answer

The study’s most quoted sentence is about manner, not accuracy: “Most of the tools we tested presented inaccurate answers with alarming confidence, rarely using qualifying phrases.” ChatGPT is the documented extreme — 134 wrong identifications, fifteen hedges, zero refusals. The researchers also found a premium paradox: paid models such as Perplexity Pro ($20/month) and Grok 3 “answered more prompts correctly than their corresponding free equivalents” yet “paradoxically also demonstrated higher error rates,” because they preferred a definitive wrong answer over declining. Why does this happen? Because of how these systems are built and tuned. A language model generates the most plausible continuation, and assistant tuning rewards helpful, complete, decisive replies; “I couldn’t find that article” scores badly against a fluent guess unless the product deliberately engineers refusal in. The reader-facing consequence is severe: tone carries no information. A right answer and a wrong one sound identical, so your confidence must come from checking, never from how the answer feels.

Failure mode 2: The citation that doesn’t support the claim

A citation in AI search is not retrieved like a footnote in a book; it is generated — the model writes an answer and attaches links it judges relevant. The Tow Center found the judgement wanting at scale. Tools “had a common tendency to cite the wrong article”: DeepSeek credited the wrong source 115 times out of 200. Even correct identifications often linked wrongly, and the researchers note that when chatbots cite trusted brands, “audiences are more likely to trust the answer — even if it’s incorrect” (a point they quote from the BBC’s own research into AI assistants). A subtler variant: the answer cites a copy — “syndicated versions of articles on platforms like Yahoo News or AOL rather than the original sources,” in one case a republished version cited despite the tool’s maker having a partnership with the original publisher, the Texas Tribune. For a reader, a plausible citation is worse than none: it converts your scepticism off. The defence is mechanical and quick — open the citation and confirm the page says what the sentence claims — which is why our checking checklist makes “the source must support the sentence” a standalone step.

Failure mode 3: The fabricated or broken link

The purest generation failure: links that do not exist. “More than half of responses from Gemini and Grok 3 cited fabricated or broken URLs that led to error pages. Out of the 200 prompts we tested for Grok 3, 154 citations led to error pages.” Grok 2 often linked merely to a publisher’s homepage instead of an article. A model that has learned what URLs look like can manufacture one that reads perfectly — right domain, plausible slug, wrong everything — because it is predicting text, not consulting a directory. This is the retrieval/generation split from our pillar guide made concrete: the retrieval half found (or failed to find) real pages; the generation half then wrote link-shaped text over the gaps. Any citation you plan to reuse, quote or forward deserves the two seconds it takes to click it. A link that 404s tells you the answer’s other claims deserve a second look too.

Failure mode 4: The invisible retrieval failure

Some answers fail before generation begins. The study found tools answering questions about publishers whose content their crawlers were blocked from reading — correctly identifying, in Perplexity Pro’s case, nearly a third of ninety excerpts it should not have had access to, apparently via copies and references elsewhere — and, in the opposite direction, failing or declining on publishers that fully permitted access; Google’s Gemini “only provided a completely correct response on one occasion” for the ten publishers that allowed its crawler, and declined politics-related queries outright. The reader sees none of this: a blocked crawler, a missing index entry and a genuinely unanswerable question all surface as the same fluent paragraph — or are papered over by it. The practical lesson is that absence of evidence in an AI answer is not evidence of absence, especially for new, niche or paywalled material: run the classic search before concluding something doesn’t exist.

What this means

Step back from the percentages and the picture is coherent. AI search fails the way it is built to fail: a retrieval layer that can miss, misattribute or be blocked, feeding a generation layer that is fluent by design, confident by tuning, and capable of writing citations as text rather than fetching them as records. None of that makes the tools useless — the same study shows the best tool wrong “only” a third of the time on an adversarial test, and everyday questions are kinder. It makes them junior: fast, well-read, eager, and in need of the supervision any junior’s work gets before it leaves the building. For US readers the stakes scale with the habit Pew measured — answers consumed without clicks, sessions ended on the summary — because an unexamined failure mode multiplied by hundreds of millions of daily questions becomes part of the information environment itself, which is why the Tow Center frames citation quality as a publisher-and-public problem, not just a product bug. Your personal protocol is short enough to keep: prefer tools whose citations you can open fastest, open them for anything that matters, treat confidence as decoration, and run a classic search when the answer decides something. The technology will keep improving its retrieval; the generation layer’s fluency is permanent. Learning to read AI answers the way editors read copy — interested, grateful, and sceptical — is the durable skill this era is asking for.

Publishers, for their part, are learning the reader’s lesson from the other side: being the source an AI answer garbles is worse than being absent from it, which is why provenance — clear authorship, stable URLs, accessible originals — is becoming a competitive concern for newsrooms rather than housekeeping. The Tow Center addressed its recommendations to the platforms, but the study’s real audience is anyone who reads: the era when a citation could be taken as a promise is over, and the small print now has to be read by the person it protects.

Bottom line

AI search fails the way it is built to fail: retrieval that can miss, feeding generation that is fluent by design. Useful for orientation, dangerous for citation — check the source, not the summary, when it counts.

Read Next

• How to Ask AI Search Better Questions: Prompts That Get Specific, Sourced Answers
• How to Check AI Answers: A Simple Fact-Checking Checklist

Sources

• AI Search Has a Citation Problem — Tow Center for Digital Journalism, Columbia Journalism Review (accessed 7 November 2026)
• AI Features and Your Website — Google Search Central (accessed 7 November 2026)
• Getting Started with Perplexity — Perplexity Help Center (accessed 7 November 2026)