Cloudflare vs LiteLLM

Cloudflare Enables Separation of AI and Search Bots

Cloudflare has introduced a new feature that allows website operators to distinguish between AI crawlers and traditional search engine bots. This tool enables more granular control over which bots can access and scrape website content, addressing concerns about unauthorized use of content for AI training. The feature is part of Cloudflare's broader efforts to give publishers more control over their content in the age of generative AI.

• Cloudflare introduced a feature to differentiate AI crawlers from search bots.
• The feature allows website operators to set separate policies for AI and search bots.
• This aims to give publishers more control over content usage for AI training.

Investor Steve Grasso buys Cloudflare stock on AI growth

CNBC Pro contributor Steve Grasso has purchased shares of Cloudflare, a cybersecurity and network services company, citing its superior growth rate and new opportunities from AI. Grasso notes Cloudflare's revenue grew 36% year-over-year to $696 million in the last quarter, outpacing competitors CrowdStrike and Zscaler. He highlights that existing customers are increasing spending by about 20%, and AI is creating new revenue streams, such as helping companies manage AI bot traffic and using OpenAI models for security. Despite the stock trading at a premium of 40 times sales and being up 68% year-to-date, Grasso believes momentum can continue, though he advises using stop orders to manage risk. The next catalyst is Cloudflare's third-quarter earnings report.

• Cloudflare's revenue grew 36% year-over-year to $696 million in the last quarter.
• Cloudflare's stock is up about 68% this year and recently hit an all-time high.
• Cloudflare trades at roughly 40 times sales, higher than cybersecurity peers.

Cloudflare introduces new controls for search crawlers, AI training, and agents.

Cloudflare is introducing new controls that let website operators separately decide whether their content can be used for traditional search, AI training, or AI agents. This aims to resolve the trade-off between being discoverable in search and protecting content from being used to train AI models. The company is replacing its 'Block AI Bots' toggle with three independent settings, including a 'Disallow AI Training' option. It is also introducing an 'Accountable' status for AI crawlers, which requires them to support independent opt-outs for AI training and search. Apple, Google, and Microsoft are already compliant or have committed to a timeline. Additionally, Cloudflare is replacing 'Managed Robots.txt' with 'Bot Preference Sync' for centralized control and is working on an open standard called 'ai-prefs' for expressing site preferences.

• Cloudflare introduces separate controls for search, AI training, and AI agents.
• A new 'Disallow AI Training' setting allows blocking AI training while still allowing search access.
• 'Block AI Bots' is being replaced by three separate toggles for search, training, and agents.

Auto Router: 45% Lower Cost on 25 SWE-bench Tasks

We solved 23 of 25 SWE-bench Verified tasks with LiteLLM's experimental capability router for $11.15, compared with $20.27 using Opus 5.

Introducing LiteLLM Fusion: 56% More Tasks Solved Than Fable 5

LiteLLM Auto Router Fusion ran three models on the same task and synthesized their work, solving 14 of 21 Terminal-Bench tasks against 9 for Claude Fable-5 alone. Total spend rose 36%, cost per solved task fell 12%, and turn latency went up 5x.