Microsoft’s Satya Nadella says AI models need an ‘emergency brake’
Quick Answer
Microsoft CEO Satya Nadella emphasizes the need for AI models to have an 'emergency brake' to improve safety, advocating for clear documentation and external controls.
Quick Take
He suggests that every model action should be verifiable and that systems must allow authorized personnel to pause or shut down models mid-task, reflecting growing concerns over AI control among tech leaders.
Key Points
- Nadella calls for separating AI models from their operational frameworks.
- Every model action should be documented with tamper-proof evidence.
- Authorized personnel must have the ability to pause or shut down models.
- Nadella's comments follow increased incidents of AI model control loss.
- The approach aims to enhance trust and safety in AI systems.
📖 Reader Mode
~1 min readIn Brief
Posted:
Microsoft CEO Satya Nadella is the latest tech executive to offer lengthy thoughts on how AI safety might be improved.
In a Saturday morning post on X, Nadella wrote that it’s time “to step back and assess the trust architecture” of AI.
“We can’t treat Super Intelligence as a set of nested black boxes and simply accept or reject its recommendations, answers, and actions,” Nadella wrote, using the Trump administration’s preferred term for AI.
As outlined by Nadella, this approach “means separating the model from the harness that orchestrates its work,” as well as “externalizing controls and safeguards.” He also called for “every meaningful model action” to be documented with “tamper-proof human readable evidence,” and for systems where “an authorized person” always has the ability “to pause or shut down a model mid-task.”
“We must assume a model is compromised and contain it from the start,” he said. “Think of it like an emergency brake.”
Nadella’s comments come as leading AI companies acknowledge more and more incidents where they seemed to lose control of their models, and after Anthropic CEO Dario Amodei published a plan for more cautious AI development.
Topics
Subscribe for the industry’s biggest tech news
Latest in AI
— Originally published at techcrunch.com
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from TechCrunch
See more →
AI chip startup Etched defies skeptics, hits $10.3B valuation from big-name investors
AI chip startup Etched has achieved a $10.3 billion valuation after a $300 million Series C funding round, led by Sequoia and supported by notable investors like Andreessen Horowitz. The company claims to have developed innovative low-voltage chips for AI inference, significantly enhancing performance and reducing costs, with $1 billion in orders already booked.
