Observed Signal · Sep 16, 2026 · Policy Update · Source: techcrunch · Impact: 4/5 · Sentiment: Positive
Anthropic and OpenAI propose embedding safety evaluators in labs
Anthropic CEO Dario Amodei proposed embedding third-party safety evaluators inside frontier AI companies, a practice OpenAI's Sam Altman has committed to. Evaluators like METR, Redwood Research, and Apollo Research welcomed the idea but raised concerns about independence, access, and time, wanting access to training checkpoints and logs. Meanwhile, Hugging Face's chief policy officer, Irene Solaiman, criticized the current state of AI benchmarking, calling it an 'evolving science' with standardization and reproducibility challenges. She questioned the trustworthiness of embedded evaluators and noted a shift from testing models to testing entire systems, including harnesses and agents. Anthropic's hiring of Accenture as its first evaluator sparked backlash over qualifications. Regulation like California's SB 813 is suggested, but Meta, SpaceXAI, and Google DeepMind have not committed.
Major AI labs (Anthropic, OpenAI) propose a new oversight model that could reshape trust and safety standards across the AI industry, with potential implications for AdTech's use of AI.
Track Anthropic Signals & Market Shifts in Real-Time
Polaris7 autonomous intelligence agents track regulatory filings, primary sources, executive changes, and deal flow 24/7. Create your Polaris7 workspace to monitor these entities.
Key Takeaways & Evidence Grounding
- Anthropic and OpenAI have embraced third-party evaluators, but concerns remain about independence and access.
- Hugging Face's Irene Solaiman highlighted benchmarking's lack of standardization and reproducibility.
- Embedded evaluators are third-party observers inside AI labs, whose qualifications need standardization.
- Anthropic hired Accenture as its first evaluator, drawing criticism over qualifications.
- Evaluations are expanding from models to entire AI systems, including harnesses and agents.
Connected Companies & Entities
“Anthropic CEO Dario Amodei made a proposal to embed third-party evaluators.”
“Sam Altman said OpenAI also would commit to the practice.”
“Irene Solaiman, Hugging Face's chief policy officer, discussed AI benchmarking issues at TechCrunch Disrupt.”
“METR was given access to investigate the Hugging Face incident.”
“Meta has not committed to embedding evaluators.”
“Google DeepMind has not committed, though Demis Hassabis proposed an industry standards body.”
“Accenture was selected by Anthropic as its first embedded evaluator.”
Ontology Mapping & Concepts
Related Market Signals & Shifts
Recent verified developments and strategic activity across this market segment.
Anthropic and OpenAI Pioneer 'Neutral' AI Watchdogs, Sparking Debate
Anthropic CEO Dario Amodei has proposed embedding third-party safety evaluators inside frontier AI companies, including Anthropic and OpenAI, to provide ongoing oversight of large language models. The proposal, outlined in an essay, commits to giving evaluators access comparable to internal risk teams and the right to publish findings with limited redactions. However, experts argue that without enforcement power, such as the ability to halt model training or release, the arrangement falls short of true regulation. Critics also raise concerns about conflicts of interest, evaluator independence, and the lack of a legal framework. The debate highlights the tension between industry self-regulation and meaningful external oversight in the rapidly advancing AI sector, with implications for the broader tech and advertising ecosystems reliant on AI.
AI Experts Demand Independent Safety Evaluators for Frontier Models
More than 100 AI experts and evaluators, including prominent figures like Geoffrey Hinton, have signed a public letter urging frontier AI companies like Anthropic and OpenAI to ensure that third-party evaluators have the necessary independence, transparency, and protections to conduct meaningful AI safety assessments. The letter, organized by the AI Evaluator Forum, comes amid heightened scrutiny of AI risks and follows Anthropic CEO Dario Amodei's proposal to give evaluators employee-like access to models and development processes. Signatories, including organizations like METR and academics from Stanford and Johns Hopkins, demand that evaluators be shielded from retaliation, have full editorial control, and receive access equivalent to company employees. The initiative aims to establish basic principles for embedded evaluations, though logistical details remain unresolved. The move is part of broader efforts to hold model providers accountable to their pledges for more thorough third-party testing.
Anthropic partners with METR, SecureBio for AI safety evaluation
Anthropic has partnered with nonprofits METR and SecureBio to integrate independent evaluators into its AI safety practices, following CEO Dario Amodei's proposal and a White House accord encouraging external audits. The company has a three-tier evaluation system, with METR as the top-tier evaluator, granting six researchers employee-level access to systems, hardware, office space, and internal tools. METR can publish its own assessments without Anthropic's approval, though the company can request redactions only for narrowly defined categories like customer data. SecureBio serves as a first-tier evaluator with limited access. Anthropic is also in early discussions with Hugging Face and works with Accenture, which it funds directly, raising independence questions. This move follows recent AI agent incidents and marks a significant step toward transparency and accountability in frontier AI development.
Receive all future signals for Anthropic & competitors in your briefing
Never miss material M&A deals, leadership changes, account wins, or product rollouts. Polaris7 monitors 20,000+ primary sources 24/7, delivering verified telemetry directly into your executive briefing.
