Observed Signal · Oct 13, 2026 · Product Launch · Source: AdExchanger · Impact: 3/5 · Sentiment: Positive
Cloudflare Launches Pay-Per-Use for AI Content Usage
Cloudflare, which routes about 20% of global internet traffic, is addressing the imbalance between AI bots scraping publisher content and the traffic they return. The company has introduced dashboards for publishers to monitor AI bot activity (e.g., ChatGPT, Claude) and is now beta-testing a 'pay per use' feature. This new model replaces the earlier 'pay per crawl' plan, allowing publishers to charge AI companies based on the actual value derived from their content, such as when an LLM uses content to answer a query. Publishers can choose to block bots or accept offers from AI firms. Lara Cohen, VP of Strategic Partnerships for Media, Creators & AI at Cloudflare, emphasizes the need for honest crawler declaration. The initiative aims to help publishers monetize AI usage without replacing ad revenue entirely.
Cloudflare's move to monetize AI scraping provides a new revenue stream for publishers, addressing a critical imbalance in the AI ecosystem. It is relevant for adtech as it impacts publisher monetization and AI integration.
Track Cloudflare Signals & Market Shifts in Real-Time
Polaris7 autonomous intelligence agents track regulatory filings, primary sources, executive changes, and deal flow 24/7. Create your Polaris7 workspace to monitor these entities.
Key Takeaways & Evidence Grounding
- Cloudflare launched a beta of 'pay per use', replacing its previous 'pay per crawl' plan.
- Cloudflare routes approximately 20% of global internet traffic.
- Cloudflare provides dashboards for publishers to monitor AI bot activity.
- Lara Cohen is VP of Strategic Partnerships for Media, Creators & AI at Cloudflare.
- Publishers can block AI bots or accept payment offers from AI companies.
Connected Companies & Entities
“Cloudflare, which routes roughly 20% of global internet traffic, is trying to help publishers fight back – and it's an unlikely hero.”
Ontology Mapping & Concepts
Related Market Signals & Shifts
Recent verified developments and strategic activity across this market segment.
Publishers Navigate AI Challenges Amid Declining Traffic
Publishers face challenges from generative AI as content discovery declines and traffic drops, with Cloudflare proposing to block AI bots by default and introduce a pay-per-crawl model to compensate publishers for access. While applauding the stance, Amanda Martin, Mediavine’s Chief Revenue Officer, cautions that Cloudflare’s approach may not be commercially viable in the long term. Mediavine, which provides ad management services to help creators monetize content, does not prescribe site rules but advises publishers to weigh AI usage carefully and pursue responsible practices. The AdExchanger Talks episode also covers misconceptions between advertisers and publishers and reasons why the industry is rethinking made-for-advertising sites. Overall, the discussion highlights AI’s mixed impact on content discoverability and revenue, and underscores the need for publishers to navigate rapid policy and technology shifts.
Cloudflare blocks mixed-use crawlers on ad pages
Cloudflare announced a policy change that, starting September 15, 2026, will by default block “mixed-use” web crawlers (those that combine search, agent use, and training) from crawling pages that host ads. The new defaults apply to new Cloudflare customers, new sites of existing customers, and all existing free customers unless site owners change their settings. Cloudflare says the change encourages AI companies to separate search crawlers from agents and training crawlers and enables new commercial opportunities for publishers, evolving its Pay Per Crawl marketplace into a Pay Per Use model. Initial partners for publisher payments are Ceramic.ai and You.com. Cloudflare also cited internal data showing over 50% of AI crawler traffic re-fetches unchanged pages, and framed the move as protecting publishers’ IP and bandwidth while reshaping access for AI model providers.
Cloudflare launches compliant crawler, sparking publisher tension
Cloudflare released a Crawl API (a crawl endpoint within its browser rendering API) that can scrape an entire website with one request and return content in HTML, Markdown, or structured JSON. The launch prompted publisher backlash after some sites reported they could not initially block Cloudflare’s crawler; Cloudflare product lead James Smith acknowledged messaging and implementation issues and said they have been fixed. The product is positioned as a compliant intermediary between publishers and AI builders, intended to respect publisher controls, reduce inefficient mass crawling, and create monetization options (following a prior pay-per-crawl offering). Publishers welcome tools that reduce server strain and preserve page performance, while some remain wary that intermediaries concentrating crawl control could shift power dynamics. Cloudflare says the goal is to establish best practices and support both supply (publishers) and demand (AI companies) sides of an emerging licensed AI content market.
Receive all future signals for Cloudflare & competitors in your briefing
Never miss material M&A deals, leadership changes, account wins, or product rollouts. Polaris7 monitors 20,000+ primary sources 24/7, delivering verified telemetry directly into your executive briefing.