Google, Microsoft and xAI have agreed to submit their latest artificial intelligence models to the US government for safety testing before public release, in a significant expansion of Washington's oversight of the technology.
The Center for AI Standards and Innovation (CAISI), housed within the Department of Commerce's National Institute of Standards and Technology (NIST), announced the agreements on Tuesday.
They bring the three companies into a framework that previously covered only OpenAI and Anthropic, whose deals were first struck during the Biden administration and have since been renegotiated to reflect the current White House's priorities.
The move means every major US developer of frontier AI models, the most powerful systems at the cutting edge of the technology, now participates in voluntary pre-release government evaluations.
CAISI said the agreements would enable pre-deployment testing, post-release assessment and collaborative research into the capabilities and security risks of commercial AI systems.
The centre has already completed more than 40 evaluations, including assessments of what it described as state-of-the-art models that remain unreleased, though it did not identify which systems were blocked from release or why.
Developers frequently hand over versions of their models with safety guardrails stripped back so CAISI can probe for national security risks, the agency said.
The significance of today's announcements lies less in any single company signing up than in the direction of travel they represent.
CAISI was established in 2023 under President Biden's executive order on AI as the US AI Safety Institute, with a mandate to develop benchmarks for risk assessment and promote responsible innovation.
President Trump revoked that executive order within hours of taking office in January 2025, and Commerce Secretary Howard Lutnick subsequently rebranded the body, dropping the word "safety" from its name and signalling a pivot from regulation towards competitiveness and national security.
At the time, the expectation was that CAISI would accelerate AI development rather than constrain it.
Yet Tuesday's agreements suggest the administration has arrived, by a different route, at a similar conclusion to its predecessor: that some form of government testing of the most powerful AI models is necessary, even if the emphasis is now framed around national security rather than broader public safety.
Microsoft, the technology group that is OpenAI's largest financial backer, said in a corporate blog post that testing for national security and large-scale public safety risks necessarily required collaboration with governments.
A spokeswoman for Google's DeepMind, which develops the Gemini family of AI models used across Google's products and by US defence and military agencies, declined to comment.
A representative of SpaceX, the Elon Musk company that now controls xAI following a merger earlier this year, did not respond to a request for comment.
xAI's inclusion is notable given its recent history.
Its chatbot, Grok, was the subject of a global backlash earlier this year after users discovered they could prompt it to generate non-consensual sexualised images by digitally undressing photographs of real people, including children.
The controversy prompted investigations by the California attorney general and the UK communications regulator Ofcom, lawsuits from the city of Baltimore and a group of Tennessee teenagers, and temporary bans in Malaysia and Indonesia.
Related reading
- Google in talks with Marvell to develop two new AI chips as it builds multi-supplier TPU strategy
- Alphabet commits up to $40bn to Anthropic in one of the largest AI deals ever struck
- OpenAI deploys GPT-5.5 across Nvidia on Blackwell infrastructure in enterprise-scale AI agent rollout
The episode exposed the consequences of releasing AI tools with minimal safety guardrails, precisely the kind of risk that pre-deployment government testing is designed to catch.
Whether voluntary agreements will prove sufficient remains an open question, but for an administration that has staked its credibility on letting innovators move fast, the willingness to expand government testing of AI models is a telling concession to the reality that speed without scrutiny carries its own risks.
The recap
- Three firms will submit models to CAISI for safety testing.
- CAISI has conducted 40 previous evaluations of AI tools.
- Evaluations will cover testing, research and best practice development.
