AI Fabricated a Citation: How to Catch It Before Publish

An arXiv study found 100 fabricated citations in 53 published NeurIPS 2025 papers, roughly 1% of accepted papers, each of which had passed review by 3–5 expert researchers (arxiv.org). Experts missed them. A quick skim before you hit publish will miss them too.

This post gives you a three-check routine you can run on any AI-assisted draft before it goes live. You will be able to catch the three ways AI citations actually fail: hallucinated URLs, real sources saying something different, and dead links. You need the draft, its list of citations, and about twenty minutes. Nothing else.

The failure rate is high enough that spot-checking isn't optional

A study in Critical Care Explorations found 55% of LLM-generated neurocritical care references contained inaccuracies, with 28.3% entirely fabricated (pubmed.ncbi.nlm.nih.gov). Those are medical references, written with the whole weight of "get this right" behind them. Blog drafts get less care, not more.

If you assume your model of choice is the safe one, the same study argues against that. Hallucination rates varied sharply by model: DeepSeek-V3 at 23%, GPT-5.3 at 69%, Grok-4 at 73%. There is no safe default.

The scale is worse than most people expect. A study of 111 million references across 2.5 million papers estimated about 146,932 hallucinated citations entered the literature in 2025 alone (arxiv.org). SourceVerify reports research consistently showing 30% to 70% of AI-generated citations are fabricated. A wide range, but the floor is not zero..

Read those numbers as an operating instruction, not trivia. Any citation in an AI draft is guilty until verified.

Check 1: The URL looks real and isn't

Otio lists fabricated URLs as a core failure mode: AI citations can mimic real journals, titles, and authors but cannot be found in the scholarly record. The fake ones borrow real structure. A plausible DOI. A real journal name. An author who exists. A paper that does not.

Concrete example: a draft cites "Journal of Digital Marketing, 2024, DOI 10.1234/jdm.2024.0887." The journal is real. The DOI resolves to nothing, or to a paper about soil chemistry. The author has published eleven papers, none with that title.

This is hard to catch by eye, and the NeurIPS study explains why. In its taxonomy, Total Fabrication accounted for 66% of fabricated citations, and 100% of them exhibited compound failure modes: the fake parts are wrapped inside real-looking details. A fabricated citation is not a blank where a citation should be. It is a citation-shaped object with one wrong component.

The check: paste every DOI or URL directly into the publisher's site or a DOI resolver. If it does not resolve to the exact paper cited, it is fabricated. Do not accept "the title looks familiar." Familiar is what fabrication is built from.

Pitfall to avoid: checking the domain instead of the destination. journals.sagepub.com being real tells you nothing about whether the article exists at that address.

Check 2: The source is real but says something different

Citeability calls this misattributed claims: the AI cites a real source that does not actually support the specific claim made. This is the most common failure in drafts that survive Check 1, because the citation resolves and the link opens. Everything works. The claim is still wrong.

Concrete example: a draft says "most marketers say AI has cut their content budget" and cites an industry survey. The survey is real. The claim is not in it. The survey asked enterprise CMOs about hiring plans, and it is years old. Real details welded to the wrong claim.

The NeurIPS taxonomy has a name for this: Partial Attribute Corruption, 27% of fabricated citations: a real paper with a wrong detail attached. Same shape in blog citations, just with industry reports instead of journals.

The check: open the source and find the exact figure or claim in the draft. Not a similar figure. Not a figure about an adjacent topic. The figure, on the page the citation points to. If you cannot find it, the citation is misattributed, and the fix is either a corrected claim or a deleted claim.

Pitfall to avoid: trusting the summary. If the AI draft quotes the source, compare the quote to the source word for word. Paraphrases drift, and drifted paraphrases are misattributions with extra steps.

Check 3: The link worked when the AI wrote it and is dead now

Otterly notes dead links as a failure mode: cited URLs become inactive over time, and studies show a significant percentage of AI-generated citations are no longer active. This failure mode has a time delay, which is why it survives the first two checks. The draft was fine on Tuesday. The press release was moved to an archive on Thursday. You publish on Friday with a 404 in your evidence.

Concrete example: a draft cites a vendor's pricing page to support a market-size claim. The vendor rebranded and killed the page. Or it cites a PDF that lived on a university lab site that got reorganized. Corporate pages, press releases, and moved documentation are the usual casualties.

There is a nastier variant. CiteTruth flags citations to retracted or corrected papers — a live link to a source that no longer stands behind its own claim. The link opens. The paper is still there. It has a retraction notice on it.

The check: click every link at publish time, not draft time. Publish time means the day it goes live, every time. For the retracted-paper case, check the publisher's page for a correction or retraction notice; Retraction Watch covers the high-profile ones.

Pitfall to avoid: treating this as a one-time check. Sources rot continuously. A draft you verified last month needs its links re-clicked before publish.

Manual checking doesn't scale, so verification is becoming automated

The NeurIPS study proposes mandatory automated citation verification at submission, arguing peer review lacks effective citation verification. Peer review, with 3–5 experts per paper, missed 100 fabricated citations. If the academic world concludes that human review needs machine backup, your publish queue needs it more.

The tooling exists. Verifing's Citation Verification tool resolves scholarly identifiers and labels each item VERIFIED, RETRACTED, HALLUCINATED, or NEEDS REVIEW, including dead-link detection. Veru audits citations against a database of over 250 million academic papers, cross-referencing CrossRef, OpenAlex, PubMed, and Google Scholar. CiteTruth checks citation metadata against more than 140 million academic records. SwanRef detects fake AI-generated citations from tools like ChatGPT and Claude.

Note the limit: these tools are built for academic citations: DOIs, journals, PubMed records. Blog posts cite industry reports, vendor pages, and news articles. The check is the same, but the tooling for it is thinner. Which brings us to the actual fix.

What a traced-to-source fact sheet changes

All three checks exist because citations are generated after the prose. The model writes a claim, then invents evidence for it. Reverse the order and the failure modes mostly disappear.

ContentRails builds a cited fact sheet as part of its per-project pipeline, before drafting (contentrails.ai). The full sequence is plan, research, cited fact sheet, outline, draft, checks, editor, in that order, each step logged. The draft can only cite what was retrieved. It cannot invent a DOI for a claim that has no source, because there is no step where it writes prose first and finds sources later.

ContentRails states that every figure in a draft comes from a source it actually read, with automated checks and an editor pass before you see the draft. That is the same three checks above, run by the pipeline instead of by you at midnight.

Your review changes shape. Instead of hunting through prose for invented citations, you spot-check the fact sheet against the draft: does each figure match its source? That is Check 2, applied to a short list, instead of applied to a finished essay.

How to run this across multiple sites without it eating your week

If you operate several sites, the routine has to survive repetition. Three things make that work.

First, the publish gate. ContentRails drafts are not published until you approve them unless full autopilot is on, and every draft carries its sources (contentrails.ai). That approval step is where your spot-check lands, and the sources attached to the draft are what you check against.

Second, be clear on what the tool does and does not promise. ContentRails' terms state figures are checked against retrieved sources, but generated content may still contain errors, and you are responsible for reviewing before publishing (contentrails.ai). That is the correct posture. The fact sheet narrows your review. It does not remove it. Anyone who tells you their tool makes verification unnecessary is describing the NeurIPS reviewers.

Third, set the standard per site. Automation levels are Off, Suggest topics, Write drafts, and Full autopilot, set per project. A site where a wrong statistic costs you credibility stays on approval. A lower-stakes site can run further. Each project runs the pipeline independently, in that site's own voice and on its own evidence, with nothing bleeding between sites, so the same citation standard applies everywhere without one site's facts contaminating another's.

If you want to see the pipeline before paying for it, the free plan runs one project end to end with three posts a month (contentrails.ai).

How to know it worked, and what to do next

You know the routine worked when the count is boring: every figure in the published draft has a source that opens, says the thing the draft says, and was live on publish day. Zero surprises after publish is the metric.

Decision rule: no draft gets published until every figure in it traces to a source you (or your tooling) actually opened and read.

Next step: pick one recent AI-assisted draft and run the three checks. Paste the URLs. Find the figures. Click the links. Count what fails. If you would rather not do that by hand for every site you run, set up one project on ContentRails' free plan and compare its cited fact sheet against your manual pass. The difference is your weekly time budget, measured.