Observed Signal · Jul 29, 2024 · Technical Release · Source: OnlineMarketing.de · Impact: 3/5 · Sentiment: Neutral
Reddit Content Vanishes From Major Search Engines
Bing and DuckDuckGo no longer display current Reddit content, as Reddit blocks crawling on these search engines. Google remains able to show Reddit content due to a content-cooperation deal reportedly worth $60 million per year. Reddit updated its robots.txt to limit third-party crawling, with a rollout in coming weeks and ongoing rate-limiting; Reddit also stated it blocks crawlers that won’t commit to not using crawl data for AI training. Separately, Reddit expanded monetization with a Lead Gen Ads beta, enabling advertisers to generate high-quality leads via Reddit, aided by AI automation through Zapier. Reddit highlights its scale, noting over 82 million daily active users, more than 16 billion posts and comments, and more than 100,000 active communities. OpenAI is also leveraging Reddit content under a cooperation, underscoring Reddit’s influence on AI training data and search.
Covers crawling policy changes and a major content deal affecting search visibility and AI training data; also notes a new Lead Gen Ads feature and AI automation integration.
Track Reddit Signals & Market Shifts in Real-Time
Polaris7 autonomous intelligence agents track regulatory filings, primary sources, executive changes, and deal flow 24/7. Create your free Explorer workspace to monitor these entities.
Key Takeaways & Evidence Grounding
- Google has a content-cooperation deal with Reddit worth about $60 million per year, enabling Reddit content to appear in Google Search results.
- Bing, DuckDuckGo and other engines no longer show newer Reddit content due to crawlers being blocked; Google remains an exception.
- Reddit updated its robots.txt with a forthcoming update, continuing rate-limiting and blocking unknown bots; researchers and the Internet Archive will have access for non-commercial use.
- Reddit states it blocks crawlers unwilling to commit to not using crawl data for AI training.
- Lead Gen Ads are in beta on Reddit, with AI automation via Zapier to help advertisers generate high-quality leads.
Connected Companies & Entities
4 Entities mappedOntology Mapping & Concepts
Related Market Signals & Shifts
Recent verified developments and strategic activity across this market segment.
Reddit May Not Renew Google AI Content Deal
Reddit is weighing cutting Google’s AI access as a $60 million-per-year content licensing deal that allowed Google to train models nears expiration. The move follows reports that Google’s AI Overviews—displayed above search links—have kept readers on Google and reduced referral traffic to publishers, prompting Reddit to reassess the partnership. Reddit shares slid after the report amid investor concern, and other publishers (Politico, CNN, Business Insider) have reported steep declines in Google-driven traffic. Reddit also licenses content to OpenAI and will report second-quarter results on July 30, 2026. The broader newsletter flagged risks to publisher monetization, viral AI-generated content trends, and investor worries over large AI capital spending by firms such as Alphabet and Tesla, while the U.S. Department of Energy selected 278 projects for its Genesis AI program.
Reddit CEO: Google's AI Overviews Can't Replace Blue Links
Reddit CEO Steve Huffman said Google’s AI Overviews — AI-generated summaries that push traditional blue links down the page — have not produced the same positive referral-traffic impact that the classic search results have delivered for publishers. Huffman made the comments alongside Reddit’s quarterly earnings release; despite revenue and earnings beating estimates and ad revenue jumping 61% year-over-year, Reddit shares fell in after-hours trading. Reddit has data-licensing partnerships with OpenAI and Google and has considered cutting off Google’s access to its data, though Huffman said the company's thinking hasn’t changed. Reddit is focusing on product improvements and user retention to convert search traffic into more valuable direct traffic while keeping capital spending low.
Reddit Is Training The Robots
In a podcast interview published June 30, 2026, Reddit COO Jen Wong discussed how Reddit’s billions of posts and comment threads have become a major data source for AI search and large language models. The article notes Reddit signed multimillion-dollar data licensing deals in 2024 with Google and OpenAI and is reportedly exploring a usage-based pricing model for data access. Wong emphasized that human-generated content remains central to Reddit’s value and highlighted efforts to keep AI-generated or manipulated content from contaminating Reddit’s corpus. The episode also covered community feedback’s role in purchase decisions and Reddit’s measures to prevent forum gaming that could distort AI search results.
Track Real-Time Market Signals & Shifts
Set up custom watchlists to receive automated, evidence-grounded executive digests whenever material signals or shifts occur across your tracked landscape.