Nadella: Assume AI Models Are Initially Out of Control

Microsoft chief executive Satya Nadella wrote in a lengthy post on X that developers and regulators should assume advanced artificial intelligence models are, from the outset, a potential hazard and must be designed with controls to contain them. He argued that the “architecture of trust” around AI needs to be reexamined, and that current practice of treating powerful models as a set of nested black boxes is inadequate.

Separating models from control and oversight

Nadella said we should stop accepting a model's recommendations, responses or actions without structural safeguards. He proposed creating architectures in which models are separated from the systems that manage and execute tasks so that containment measures and safety controls can sit outside the model itself. That separation, he wrote, would make more independent oversight and monitoring of model behaviour possible.

On the question of compromise, Nadella recommended assuming a model may be out of control from the start and building containment accordingly. He framed this as an operational safety requirement: "We should assume the model is out of control from the start and we must contain it. Think of it like an emergency brake. An authorized person should always be able to stop or shut the model down in the middle of performing a task." He added that more capable models will require more advanced containment technologies and that common standards for those technologies should be developed.

Alongside stop controls, Nadella urged extensive logging of models' important actions. He called for records that are both human readable and resistant to tampering so that performance and decisions can be examined and audited. Such records, he said, would support review and assessment of how models operate in practice.

Many elements of Nadella's proposals align with recommendations raised elsewhere in the AI community. He noted that issues already under discussion include timely disclosure of incidents, independent auditing, verifiable data provenance and mechanisms to contain models when they behave unexpectedly.

Nadella did not offer technical blueprints or timelines in the post, and his remarks were framed as proposals and principles rather than specific policy prescriptions. He emphasized the need for shared standards and stronger external controls to enable more robust, independent oversight of advanced AI systems.