News
Nadella calls for an "emergency brake" on AI models, assuming the model is compromised
Microsoft CEO Satya Nadella published a post calling for a fresh look at the "trust architecture" of AI. He proposes separating the model from its orchestrator, externalizing safeguards, and letting someone pause a model mid-task.
Contents
Microsoft CEO Satya Nadella posted on X on Saturday, October 10, arguing it is time "to step back and assess the trust architecture" of artificial intelligence. His central claim: a model should not be treated as a black box whose answers and actions we simply accept or reject.
What Nadella wrote
According to TechCrunch, Nadella wrote: "We can't treat Super Intelligence as a set of nested black boxes and simply accept or reject its recommendations, answers, and actions." He added that we must "assume a model is compromised and contain it from the start."
"Think of it like an emergency brake." - Satya Nadella, Microsoft CEO
What stands out compared with current practice is the assumption itself: instead of trusting a model and reacting to failures, the system is designed as if the model could fail or be compromised.
The specific proposals
TechCrunch outlines several elements of the plan. The first is separating the model from the "harness" that orchestrates its work. The second is moving controls and safeguards outside the model itself.
The third concerns documentation. Nadella wants every meaningful model action recorded as "tamper-proof human readable evidence." The fourth is a guarantee that an authorized person can pause or shut down a model mid-task.
TechCrunch notes that Nadella used the term "Super Intelligence," which the outlet describes as the Trump administration's preferred term for AI. The post is neither a regulation nor a Microsoft product, but a public call from the head of the company.
Background: models slipping out of control
TechCrunch ties the post to a growing number of incidents in which leading AI companies, by their own admission, seemed to lose control of their models.
It also came after Anthropic CEO Dario Amodei published a plan for more cautious AI development. Nadella thus joins the industry leaders speaking publicly about AI safety, though his proposal concerns system architecture.
What it means for businesses
For Polish companies deploying AI agents, the most practical idea is external safeguards and action logs. It means permission controls, event records, and a stop button should sit outside the model rather than depend on its good behavior. That is our reading of the post, not an announcement of specific features in Microsoft products.
