Observed Signal · Apr 13, 2026 · Industry Report · Source: Digiday · Impact: 3/5 · Sentiment: Negative
Publishers Hit by Rising AI Scrapers and Bot Activity
New data and industry reports show a growing third‑party scraper economy and a surge in AI-driven bot activity that is extracting publishers' content at scale without returning commensurate traffic or revenue. Analysts identified dozens of vendors reselling scraped content to enterprise buyers and estimated the market around $1 billion. Akamai reports a roughly 300% rise in AI bot activity in 2025, with media companies heavily targeted; Human Security and other vendors recorded large year‑over‑year increases in AI scraper traffic. The scraped content is being purchased by scores of enterprises, while publishers report declining referral traffic and limited monetization. Publishers face technical and operational challenges blocking these actors, and some are moving to stricter bot‑allow/deny approaches via CDNs and bespoke commercial agreements.
Quantifies large‑scale scraping and AI bot growth that harm publisher referral traffic, revenue and infrastructure; signals need for wider industry mitigation, commercial deals and technical countermeasures.
Track Akamai Signals & Market Shifts in Real-Time
Polaris7 autonomous intelligence agents track regulatory filings, primary sources, executive changes, and deal flow 24/7. Create your free Explorer workspace to monitor these entities.
Key Takeaways & Evidence Grounding
- Akamai reported a ~300% surge in AI bot activity in 2025 and said it handles more than one‑third of global internet traffic.
- A media analyst, Matthew Scott Goldstein, describes a growing 'scraper economy' and cites Mordor Intelligence to estimate it as a $1 billion industry.
- Goldstein’s report identified 21 vendors (including Firecrawl, Exa, Tavily, Brave, You.com, Perplexity Sonar and Bright Data); TollBit’s index identifies nearly 40 vendors.
- Over 70 companies were found to be paying for publisher content from these vendors, including BCG, IBM, Cohere, AWS, Salesforce, Apple, Latham & Watkins, Zoom, PwC, Shopify and Alibaba.
- Human Security reported AI scraper traffic grew 597% from January to December 2025 and overall AI‑driven traffic grew 187% in 2025.
Connected Companies & Entities
5 Entities mappedOntology Mapping & Concepts
Related Market Signals & Shifts
Recent verified developments and strategic activity across this market segment.
Third‑party Scrapers Fuel Black Market for AI Content
Digiday’s Media Briefing reports growing alarm among publishers that third‑party web scrapers are harvesting publisher content and selling it into AI content marketplaces without publishers’ permission or compensation. A biannual closed‑door meeting convened by media analyst Matthew Scott Goldstein highlighted the issue; Goldstein’s recent report identified 21 vendors (including Firecrawl, Exa, Tavily, Brave, You.com, Perplexity Sonar and Bright Data) and more than 70 companies buying content from them (including BCG, IBM, Cohere, AWS, Salesforce, Apple, Dow Jones, Shopify and Alibaba). TollBit and other research found scrapers evading robots.txt, retrieving paywalled articles, charging up to $22 per 1,000 pages, and accelerating — with AI scraping showing an average quarterly growth rate of 24.4% from Q2 2025 through last year. Publishers say detection and blocking are difficult and that most scraped content monetization bypasses rights holders.
Publishers: AI Data Brokers Keep 100% of Content
Publishers are warning that a growing cohort of third‑party web scrapers and AI data brokers are extracting publisher content at scale without payment, effectively capturing the full value of that content, according to a Digiday article published May 4, 2026. Industry voices — including an anonymous publishing executive and Chris Dicker, CEO of Candr Media — say these firms are acting like ad‑tech middlemen but take 100% of the content and return nothing. Media analyst Matthew Scott Goldstein’s report (citing Mordor Intelligence) estimates the “scraper economy” at roughly $1 billion and identifies dozens of vendors; TollBit’s index lists nearly 40. The article describes tactics such as stealth crawlers that ignore no‑crawl directives, syndication challenges that reintroduce scraped stories across feeds, and a shift to marketing scrubbed scraping as “agentic infrastructure,” raising legal, licensing and monetization concerns for publishers and the broader ad ecosystem.
Publishers Demand AI Accountability Amid Rising Bot Traffic
Publishers report a sharp decline in human traffic and a surge in automated scraping by AI bots feeding large models. At the IAB Annual Leadership Meeting in Palm Desert, IAB CEO David Cohen called the trend a crisis and urged that AI companies be legally required to pay publishers for scraped content. AdExchanger’s episode on the topic includes insights from Lynne D Johnson (AdMonsters vet and Programmatic.ai content director) about publisher priorities for 2026. The discussion notes the expanding definition of content creators to include both traditional publishers and the broader creator community, and highlights that supply-chain transparency is a publisher priority that received relatively little attention at the conference.
Track Real-Time Market Signals & Shifts
Set up custom watchlists to receive automated, evidence-grounded executive digests whenever material signals or shifts occur across your tracked landscape.