An AI Safety Institute is a government body set up to evaluate advanced AI systems for risks to national security and public safety, and to build the state's own expertise on frontier AI. The first wave clustered around the 2023 international AI summit. Institutes now share methods and findings through an international network that Seoul and follow-on meetings pushed forward.
A note on names: the two founding institutes have been renamed. The UK body became the AI Security Institute in February 2025. The US institute is now the Center for AI Standards and Innovation (CAISI) at NIST. "AI safety institute" remains the standard generic term, and this guide uses it throughout.
For years the only organizations that deeply understood frontier models were the companies building them. An institute is an attempt to put independent technical capability inside government, so public authorities can assess systems rather than accept a developer's briefing as the last word.
What they actually do
Their work clusters into a few areas.
- Testing frontier models, sometimes before release, for dangerous capabilities in domains such as cyber, biology, and autonomy, using the methods behind capability evaluations.
- Developing the science of evaluation itself, because measuring these risks well is still an open research problem, not a settled procedure.
- Advising government, so policy is informed by people who have actually examined the systems.
- Coordinating internationally, so a model tested in one country need not be rebuilt from scratch everywhere, and so standards begin to converge.
This is real institutional progress. Building state capacity to understand frontier AI is a precondition for governing it. You cannot regulate what you cannot evaluate, and until recently governments largely could not.
The power they mostly lack
Most AI Safety Institutes can test, advise, and publish. Few can compel. Access to models often depends on lab cooperation. Findings usually inform rather than bind. In most cases an institute cannot order a dangerous model withheld. These bodies can see a risk and recommend a response. They rarely get to require one.
That gap between assessment and authority is the structural limit. An institute that discovers a serious hazard and can only advise is only as effective as the government's will to act against commercial and competitive pressure to keep going. Much of the current architecture is a warning system. A warning system is not a safeguard until someone with power answers the alarm.
The folk objection is that technical capacity is enough, and politics will catch up. Capacity without authority is how governments learn the bad news late and act later still. Testing that cannot force a stop is preparation for governance, not governance itself.
What they could become
The lasting value of AI Safety Institutes is what they assemble for binding regimes later: independent technical capacity, shared standards, and an international channel between governments. In that sense they are the early form of the kind of body the Foundation argues for. Our piece on an international monitoring agency describes where this could lead. The IPCC model shows how shared scientific assessment can underwrite international policy.
What has to change is authority. Assessment must connect to enforcement, whether through domestic law that makes an institute's sign-off a condition of deployment, or through an international framework that gives verified findings real consequences. Give these institutes power that matches their expertise, and much of the scaffolding of serious governance is already partly built. That transition is the subject of our plan: stop superintelligence under law, with public evaluation capacity as infrastructure, not as a substitute for prohibition and verification.
Common questions.
What is an AI Safety Institute?
An AI Safety Institute is a government body established to evaluate advanced AI systems for risks to national security and public safety and to build the state's own technical expertise on frontier AI. The first were announced around the 2023 international AI summit, and several governments now operate one or an equivalent, linked through an international network that shares evaluation methods and findings.
What do AI Safety Institutes do?
They test frontier models, sometimes before release, for dangerous capabilities in areas like cyber, biology, and autonomy; they develop the science of how to evaluate such risks, which is still an open research problem; they advise government by feeding technical assessments into policy; and they coordinate internationally so that testing and standards begin to converge across countries. In short, they give public authorities independent capability to assess systems rather than relying on developers' own claims.
What powers do AI Safety Institutes have?
Most can test, advise, and publish, but few can compel. Their access to models often depends on the labs' cooperation, their findings typically inform rather than bind, and in most cases they cannot order a dangerous model to be withheld. They are largely bodies that can identify a risk and recommend a response, not require one, which means their effectiveness depends heavily on whether governments choose to act on their advice.
Why does the gap between assessment and authority matter?
Because an institute that uncovers a serious hazard but can only advise is only as effective as the government's willingness to act against commercial and competitive pressure to keep development going. Testing without the power to act on the results functions as a warning system rather than a safeguard. Connecting assessment to real enforcement, through domestic law or an international framework that gives verified findings consequences, is what would turn these institutes from smoke detectors into an effective safeguard.