California Wants a Kill Switch for Frontier AI. The Order Never Says What One Is.

A kill switch is an easy thing to picture. Somewhere in a data center sits a big red button, and if a frontier AI model slips its constraints, somebody walks over and presses it.

California Governor Gavin Newsom signed an executive order on September 18 that has been read as asking for exactly that. Engadget wrote that the governor "wants to implement a kill switch for frontier AI models." Gizmodo called the idea "eyebrow-raising." The order itself doesn't say what the switch is, how it works, or who gets to pull it.

What the order actually does

Newsom gave his own office two months to convene national experts and come back with AI safety recommendations. Those recommendations are meant to inform updates to California state law, which is where the consequences live. An executive order can direct the governor's staff. It can't write the statute.

The rest of it is less quotable and more specific. The order calls for independent verification organizations to be embedded inside AI labs, running periodic audits. It calls for AI companies to file risk assessments with third-party organizations. And it mandates disclosure of what it calls "loss-of-control incidents."

The dull parts are the enforceable parts

A loss-of-control incident is the thing a kill switch would exist to stop. Requiring a company to report one is a smaller ask than requiring it to halt one, and it's an ask you can check. Either the disclosure got filed or it didn't.

The audits work the same way. An outside organization sitting inside a lab with a standing right to inspect produces a paper trail, and the risk filings add to it. None of that stops a model mid-run. All of it builds the record you would need to argue, in front of a legislature or a court, that a company knew what it was shipping.

Who has to comply

The order targets frontier models as a class. It names no companies and no specific models, and leaves what counts as frontier for later, which is a lot to hand a panel that has two months.

Newsom's framing leans hard on speed. "We're going to speed up our work on substantial and responsible AI oversight before it's too late," he said. "We're going to do this thoughtfully but with urgent velocity; the stakes are too high to wait or delay action."

Urgent velocity is doing a lot of work in that sentence.

The gap between the phrase and the mechanism

A frontier model that has escaped its constraints is, by definition, a system doing something its operators did not sanction. Whether the off switch is still wired up at that point is the whole engineering question, and the order doesn't go near it. What it does instead is build the reporting scaffolding around the problem and hand the hard part to a room full of experts.

The two-month clock is the thing to watch. If the panel reports on schedule and its recommendations make it into state law, the audits, the risk filings and the loss-of-control disclosures turn into obligations. If it slips, they stay suggestions, and the kill switch stays a phrase.