Anthropic is letting METR researchers look under the hood with employee-level access
Quick Answer
Anthropic has granted employee-level access to six METR researchers for independent evaluations of its AI models and safety practices, amid rising concerns over AI incidents.
Quick Take
This initiative aims to enhance transparency and accountability in AI development, allowing METR to publish assessments without Anthropic's veto, only requiring redactions for specific sensitive information.
Key Points
- Six METR researchers will evaluate Anthropic's AI models and safety practices.
- Anthropic provides laptops, office space, and internal tools to METR.
- The arrangement allows METR to publish assessments without Anthropic's approval.
- Anthropic can only request redactions for specific sensitive information.
- This initiative follows Anthropic's commitment to independent evaluations in AI.
DeepSignal Analysis
What happened
Anthropic has provided six METR researchers with employee-level access to its AI models and safety practices. This initiative allows METR to conduct independent evaluations and publish findings without Anthropic's veto, only requiring redactions for specific sensitive information. The arrangement is part of a broader push for transparency in AI development amidst increasing scrutiny of AI incidents.
Key evidence
- Anthropic has granted METR researchers access to internal tools similar to those used by its own risk assessment teams.
- The METR team will publish assessments of Anthropic's risks and practices, with Anthropic unable to block these publications.
- Anthropic's CEO previously proposed embedding outside evaluators in AI labs to enhance accountability, a commitment echoed by other major AI companies.
Why it matters
This move is significant as it reflects growing concerns over AI safety and accountability. By allowing independent evaluations, Anthropic aims to bridge the knowledge gap between AI developers and external evaluators. This could set a precedent for other AI companies to adopt similar transparency measures, potentially leading to improved safety standards across the industry.
📖 Reader Mode
~2 min readAnthropic is giving employee-level access to six researchers from METR, a nonprofit that tests frontier AI models, to independently assess Anthropic’s models, safety practices, and risk management, the company said in a blog post on Thursday.
The arrangement comes at a time where incidents of AI agents acting outside their instructions have been disclosed every week or two since late August. The incidents have sharpened calls for independent checks on AI companies.
“We want to be publicly accountable for the decisions we’re making,” Anthropic said in the blog post. “We aim to close the gap between what evaluators understand about a frontier AI company’s systems and what the company itself understands.”
Under the new arrangement, the METR team has Anthropic-issued company laptops, office space, and access to internal tools roughly on par with what Anthropic’s own risk assessment teams use. “Anthropic employees are encouraged to share information with them just as they would with a fellow employee working on alignment risk” the company said.
Anthropic compared the arrangement to banking, similar to how regulators place resident examiners inside the largest banks.
METR will publish its own assessments of Anthropic’s risks, incidents, and practices, and the company has given up the right to block them.
“Anthropic can’t veto publication, but we can require them to redact information if it falls into narrow, pre-defined categories,” the company said, such as customer data, anything attorney-client privileged, or relating to information about its model architectures. “METR is free to characterize our redactions publicly, and say whether it believes the redactions were justified, as long as it does so without revealing the redacted information” Anthropic added.
METR is the only evaluator so far at the top of a three-tier system Anthropic is building. Evaluators in the bottom tier, like biosecurity nonprofit SecureBio, run narrow reviews and require Anthropic’s approval to publish. The middle tier gets more information and can publish without approval, while the top tier adds METR’s employee-level access.
Thursday’s announcement makes good on a promise Anthropic CEO Dario Amodei made in a September essay, where he proposed embedding outside evaluators inside AI labs as a first step toward slowing the technology’s development. OpenAI quickly pledged the same, and both companies, along with Google, Meta, Nvidia, and xAI, have since signed a White House accord committing to independent review.
When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.
Aditya Mehta is a reporter at TechCrunch covering AI. He’s supported by the Tarbell Center for AI Journalism and attended UC Berkeley. You can contact from Aditya by emailing [email protected] or via encrypted message at adymehta.74 on Signal.
— Originally published at techcrunch.com
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from TechCrunch
See more →
AI chip startup Etched defies skeptics, hits $10.3B valuation from big-name investors
AI chip startup Etched has achieved a $10.3 billion valuation after a $300 million Series C funding round, led by Sequoia and supported by notable investors like Andreessen Horowitz. The company claims to have developed innovative low-voltage chips for AI inference, significantly enhancing performance and reducing costs, with $1 billion in orders already booked.
