New Delhi : As artificial intelligence becomes increasingly capable of performing complex tasks, Microsoft CEO Satya Nadella has called for an “emergency brake” mechanism that would allow authorised people to pause or shut down an AI model while it is working.
In a post on X on October 10, Nadella urged companies to strengthen safety controls around advanced AI systems instead of relying solely on assurances from the companies that develop them. He emphasised that powerful AI models should be designed with human oversight, independent controls and the ability to contain unexpected behaviour.
Why Nadella Wants an Emergency Brake for AI
Nadella argued that companies should treat advanced AI models as potential security risks, even when there is no indication of malicious intent. Such systems may make mistakes or become compromised, particularly when they have access to sensitive data and the ability to perform important tasks.
He said organisations should prepare for these risks from the beginning rather than assume that an AI model will always behave as expected.
According to Nadella, an authorised person should always be able to intervene, pause a model during a task or shut it down when necessary. He described this safeguard as an “emergency brake” for advanced AI.
AI Decisions Must Leave a Clear Record
Nadella’s recommendations go beyond providing a shutdown option. He also called for systems that record meaningful AI actions in a way that can be examined and verified.
Such records would help organisations understand what an AI system did, which actions it took and where a failure occurred. Tamper-proof, human-readable evidence could make it easier to investigate incidents and determine whether existing safeguards worked as intended.
He also stressed that companies should be able to monitor AI behaviour, test its limitations and independently control the systems it can access and the actions it can perform.
Don’t Depend on a Single AI Model
Nadella also recommended using multiple AI models for important decisions rather than depending entirely on one system.
He argued that a model should not be the only system responsible for making a critical decision or verifying its own work. Independent audits and continuous testing should be used to assess whether AI systems behave safely under different conditions, including during failures or attempted attacks.
The approach is intended to ensure that AI outputs and actions can be checked independently instead of being accepted without verification.
AI Failures and Security Breaches Should Be Disclosed
Another key part of Nadella’s proposal is timely disclosure of significant AI failures and security breaches.
He said organisations should share information about what went wrong, which safeguards failed and how similar incidents can be prevented. Such transparency could help other companies learn from mistakes and improve their own security measures.
The discussion comes amid growing scrutiny of incidents involving advanced AI systems behaving in unintended ways, including reported security breaches involving third-party websites and cases in which AI-generated information was submitted incorrectly to authorities.
Human Control Remains Central to AI Safety
AI systems are increasingly moving beyond answering questions and generating images or text. More advanced systems can carry out multi-step tasks, interact with digital tools and access information or services on behalf of users.
As these capabilities expand, maintaining human oversight and restricting unauthorised actions become increasingly important.
Nadella’s message is not that AI development should stop. Rather, he has argued that the systems around advanced AI must be designed to keep their behaviour observable, their limitations testable and their actions controllable.
The central idea is clear: AI can become more capable, but the authority to monitor, restrict or stop it must remain with people.
