Observed Signal · May 22, 2026 · Research Study · Source: t3n · Impact: 2/5 · Sentiment: Neutral
Study Finds ~147K Fake Citations from LLM Hallucinations
A new study analyzed 111 million references across 2.5 million scientific papers and identified 146,900 fabricated or untraceable citations, attributing a steep rise in such false references to the widespread use of large language models (LLMs). The fake citations appeared across preprint repositories including arXiv, bioRxiv, SSRN and PubMed Central. Researchers from Cornell University and the University of California conducted the analysis. In response, arXiv has tightened submission rules—requiring stronger author verification and recommending sanctions (including potential one‑year bans) for works that show authors did not verify LLM-generated content. Scientists and platform leaders warned that AI hallucinations dilute the scientific record and undermine trust in research literature.
Findings highlight a significant rise in LLM-driven misinformation within scientific literature and a publisher (arXiv) policy response; relevant for organizations relying on AI-generated content verification but not a direct AdTech platform policy or major platform technical release.
Track arXiv Signals & Market Shifts in Real-Time
Polaris7 autonomous intelligence agents track regulatory filings, primary sources, executive changes, and deal flow 24/7. Create your free Explorer workspace to monitor these entities.
Key Takeaways & Evidence Grounding
- Researchers from Cornell University and the University of California analyzed 111 million references from 2.5 million scientific articles.
- The study found 146,900 fabricated or untraceable references (fake citations) in the dataset.
- Fake citations appeared across preprint repositories including Arxiv, Biorxiv, SSRN and Pubmed Central.
- Arxiv has tightened submission rules: first authors must provide a recommendation from established authors and may face a one‑year ban if submissions show unverified LLM-generated results.
Connected Companies & Entities
3 Entities mappedOntology Mapping & Concepts
Related Market Signals & Shifts
Recent verified developments and strategic activity across this market segment.
arXiv tightens rules for AI-generated research papers
arXiv, the open preprint repository formerly hosted by Cornell University and now transitioning to an independent nonprofit, has introduced stricter rules governing submissions that rely on large language models (LLMs). The platform requires first authors to have an endorsement from an established author and places responsibility on authors to verify any AI-generated content. arXiv warns that submissions containing clear evidence that authors did not check LLM outputs — for example, fabricated citations — may trigger sanctions: a one-year ban and a requirement that subsequent submissions be accepted by a recognized peer‑review platform. Moderators and subject-area chairs must confirm violations before penalties are applied, and appeals are possible. The change follows broader concerns about increasing fabricated references in biomedical literature and widespread use of AI tools in universities.
Measuring AI Citation Hallucinations in Production
Cihangir Bozdogan published a field report describing tooling and a measurement methodology for "citation hallucination" in LLMs. He defines four distinct classes of citation failure—fabricated URLs, retrieve-then-misquote, URL substitution, and anchor-text drift—and explains detection signatures and remediation for each. Bozdogan ran the methodology on roughly one thousand grounded queries across Anthropic, OpenAI, and Google/Gemini-powered grounding, and shares a worked example (500 grounded responses, ~1,200 citations) with empirical class rates and mitigation patterns. Recommended operational mitigations include hard-blocking non-retrieved URLs, sentence-level embedding/NLI verification, retrieval prompt nudges, discouraging substitution via system prompts, and an async verification UI pattern. The post emphasizes that citation-faithfulness middleware is essential for high‑stakes domains and provides production-ready implementation patterns, trade-offs (latency/cost), and monitoring guidance. Published 2026-05-10.
Why AI Hallucinates
This explainer article defines and explains AI 'hallucinations'—instances where generative models produce false, misleading, or fabricated information presented confidently. It outlines primary causes (models predict language patterns rather than verify facts; incomplete or outdated training data; lack of real-world understanding; ambiguous prompts; and model overconfidence). The piece gives real-world consequences (fake legal cases, invented research citations, incorrect medical/financial advice) and lists mitigation approaches such as improving training data quality, integrating fact-checking or live databases, using human feedback/moderation, and clearer prompting. The article is an educational overview aimed at helping readers understand the limitation and responsible use of LLMs.
Track Real-Time Market Signals & Shifts
Set up custom watchlists to receive automated, evidence-grounded executive digests whenever material signals or shifts occur across your tracked landscape.