Observed Signal · Jul 31, 2024 · Technical Release · Source: Trending Topics · Impact: 3/5 · Sentiment: Neutral
AppleBot Crawls Web for AI Training Data, Respects robots.txt
Apple has revealed details about its training data for its Apple Foundation Models (AFM), including a 3-billion-parameter on-device LLM. The company uses licensed publisher data, curated public datasets, and web crawls via Applebot, while claiming to respect robots.txt exclusions. It also denied using YouTube videos from 'The Pile'. Separately, CloudFlare released data on the most active AI scraping bots, with Bytespider from TikTok accessing 40.4% of websites, followed by GPTBot at 35.46%.
Highlights the growing use of web scraping for AI training and provides data on bot activity, relevant to publishers and ad tech regarding content protection.
Track Apple Signals & Market Shifts in Real-Time
Polaris7 autonomous intelligence agents track regulatory filings, primary sources, executive changes, and deal flow 24/7. Create your free Explorer workspace to monitor these entities.
Key Takeaways & Evidence Grounding
- Apple introduced Apple Foundation Models (AFM) including a 3B parameter LLM.
- Apple uses Applebot to crawl public web data for AI training, respecting robots.txt.
- Apple licenses data from publishers and uses curated public datasets.
- Apple denied using YouTube videos from the training database 'The Pile'.
- CloudFlare data shows Bytespider as the most active AI bot, accessing 40.4% of websites.
Connected Companies & Entities
6 Entities mapped“Apple revealed details about its Apple Foundation Models and the use of Applebot for AI training data....”
“Mentioned as one of the companies that train AI models similarly to Apple....”
“Mentioned as one of the companies that train AI models similarly to Apple....”
“Mentioned as one of the companies that train AI models similarly to Apple....”
“CloudFlare provided data on the most active AI scraping bots....”
“Listed as the origin of the Bytespider bot, the most active scraper....”
Ontology Mapping & Concepts
Related Market Signals & Shifts
Recent verified developments and strategic activity across this market segment.
OpenAI Fires Safety Researchers; GPT-6.1 Ultrafast Launch
This AINews newsletter covers several significant developments: OpenAI has fired three safety researchers linked to a METR audit, citing mishandling of confidential information. OpenAI also launched GPT-6.1 Sol Ultrafast, a faster and cheaper model, and introduced an intelligent UI for ChatGPT. Anthropic released Claude Haiku 5.5 and cut Sonnet 5.5 cache-read prices. Additionally, Arena raised $200M, and there are reports about OpenAI's revenue figures. The newsletter also includes various model launches, benchmarks, and safety research updates.
GreenCore Solutions Opens London Office for A2A-Grocery Agentic Commerce
GreenCore Solutions Corp. (GSC) announced the opening of a sales office in London, UK, under a new entity, GreenCore Solutions (UK), to support its agentic commerce hub, A2A-Grocery.co.uk. The company introduced its 0-100 Agentic Density Scale, indicating that the UK and Europe account for 84% of grocery makers, while the USA accounts for only 16%. GSC's AI agents have processed 100 million transactions year-to-date, with 40% from Europe, 20% from the USA, and 40% from the rest of the world. The London office aims to connect European makers with AI agents buying for grocery retailers across 20 markets. GSC operates on Microsoft Azure and Google Cloud Enterprise, with data residency in each market.
Google's $10M Bid for Spirit Airlines Data for AI Training
Google has won a bankruptcy auction with a $10 million bid to acquire Spirit Airlines' internal business data, including emails, Microsoft Teams messages, spreadsheets, and other records, for AI training and product development. The purchase, pending court approval, highlights the growing value of corporate data as training material for AI systems. The article, however, is primarily an opinion piece reflecting on the meaning of work and the implications of AI on human productivity. It does not provide additional factual details about the deal, such as the timeline for court approval or specific terms. The article also promotes the author's newsletter and MCP server, which are not related to the core news event.
Track Real-Time Market Signals & Shifts
Set up custom watchlists to receive automated, evidence-grounded executive digests whenever material signals or shifts occur across your tracked landscape.