Open Source AI Weights Risk

In July 2025, Mark Zuckerberg published a letter on "personal superintelligence" that also warned about being careful with what Meta chooses to open source. The pairing is the whole policy problem in one breath: a major lab racing toward extreme capability while the software culture around it still treats weight releases like ordinary open-source virtue.

Open-source software built the modern internet's plumbing. That history earns respect. It does not automatically extend to publishing the weights of near-frontier general models that can be fine-tuned for cyber offense, biological assistance, autonomous fraud, or, eventually, components of uncontrollable agents. This guide separates the values people are defending from the irreversible security act they sometimes perform.

What "open source AI" actually refers to

The phrase is slippery. Sometimes it means open-source tooling: compilers, libraries, training frameworks. Sometimes it means open datasets. Sometimes it means open model weights with licenses that may or may not meet Open Source Initiative definitions. Sometimes it means a full release of code, data, and weights.

For risk analysis, the load-bearing object is the weight file plus enough information to run and adapt the model. Once capable weights are public, anyone with sufficient hardware, or rental access to hardware, can run variants. Downstream fine-tunes multiply. Takedowns fail. That is proliferation.

Narrow open models that classify images of machinery parts are not the subject here. The subject is general, high-capability models approaching the frontier of what well-funded labs can do, and the trajectory toward systems that could contribute to loss of control.

Why release is a one-way door

You can patch a SaaS model behind an API. You can revise terms. You can shut down an endpoint. You cannot unpublish a torrent.

Security people call this the difference between managed deployment and uncontrolled replication. Nuclear analogies are imperfect, but one piece fits: once a proliferation-relevant artifact is widely copied, your remaining tools are detection, denial of supporting infrastructure, and legal pressure. You no longer have a product recall.

That irreversibility is why open weights at the frontier is a strategic regime change in who can build on a capability, not a small product decision.

The genuine goods people are trying to protect

Arguments for openness are not all naive. List them fairly.

• Research reproducibility: outsiders can test claims, find flaws, and improve safety techniques only if they can run models.
• Competition and diffusion: open weights can reduce lock-in by a few API vendors.
• Security through scrutiny: more eyes can find failures faster, at least for some bug classes.
• Global access: researchers and firms outside rich-lab orbits can build applications.
• Democratic transparency: secret models in a few hands concentrate power.

A serious prevention agenda should try to salvage these goods where they do not require open-ending catastrophic capability. That may mean open tools, open evals, open narrow models, gated access for verified researchers, and transparency reports without full weight dumps.

The risk case, stated without melodrama

As models gain skill at coding, persuasion, biological reasoning, and autonomous tool use, the marginal user who should not have unrestricted local control becomes more numerous: criminals, paramilitaries, reckless startups, state proxies, and eventually automated pipelines that improve systems further.

Open weights also undercut lab-side safety techniques that depend on controlling the inference stack: usage monitoring, rapid patching, account bans, staged rollout. You can publish safety filters with the model. Users can remove them. That is a predictable response to friction, not a theoretical edge case.

At the extreme end of the trajectory, open-ending a system that can materially accelerate AI research itself compresses timelines for everyone else, including actors who will not invest in control. That is how open release feeds both misuse scenarios and multipolar race scenarios in the takeover catalog.

A short history of the open-weights debate in frontier AI

Earlier machine learning culture often released models as academic artifacts with limited dual-use bite. As models gained general competence, the stakes changed. Arguments that made sense for a sentiment classifier do not automatically make sense for a system that writes working exploits or assists with novel biological protocols.

Companies have oscillated. Some released weights to win developer mindshare. Some restricted access after safety concerns or political pressure. Meta's open-leaning strategy for certain large models became a focal point for both praise and criticism. The July 2025 note on personal superintelligence paired ambition with a caution about open sourcing choices, which captures the unresolved tension rather than resolving it.

Governments have begun to notice. Policy drafts increasingly separate low-risk open releases from high-risk frontier releases. The law is still catching up to the irreversibility problem.

Threat models specific to open weights

Cyber offense at scale

Local models can be fine-tuned and run without a provider watching prompts. That weakens abuse monitoring. As coding and exploitation skill rises, open release expands the set of actors who can automate intrusion campaigns.

Biological assistance

Models that lower the skill barrier for dangerous biological work create catastrophic misuse tails. Open weights make safety filters removable. Even partial assistance can matter when combined with other online resources and illicit lab access.

Autonomous fraud and influence

Unmonitored local agents can run scams, political persuasion pipelines, and social engineering at scale. API rate limits and account bans no longer apply.

Acceleration of rival capability

Open near-frontier weights let other organizations, including state-linked groups, skip costly training and invest only in fine-tuning and scaffolding. That can compress multipolar race dynamics.

Loss-of-control precursors

Open agent stacks plus strong weights create more independent experiments in long-running autonomy. Most will fail. Some will generate new techniques for evasion and self-preservation behaviors in weaker forms. The research commons cuts both ways.

Steelman of the open-source AI movement

Advocates argue that concentrated closed AI is itself a takeover risk: a few firms or states controlling cognitive infrastructure. They argue that open weights enable independent safety research that labs would not prioritize. They argue that security through obscurity fails, and that broad scrutiny finds failures faster. They argue that global south researchers should not be permanent API tenants of rich-country firms.

These points deserve answers, not sneers. A prevention-centered response looks like this: concentration risk is real, so public governance and narrow-open ecosystems matter; independent research can be enabled through structured access and trusted research environments without dumping every frontier weight onto the open internet; scrutiny helps most when the artifact is not already a catastrophe multiplier; global access can be expanded through compute sharing for low-risk systems and funded collaborations rather than through irreversible high-risk proliferation.

Why "the bad actors already have it" is usually premature

At any moment, someone will claim that restriction is pointless because weights leaked, or a rival already trained something similar. Sometimes that is true for a given model generation. It is often false for the next generation.

Policy is about rates and distributions, not about perfect denial. Export controls, training licenses, and release restrictions can slow diffusion, raise costs, and keep the most dangerous systems inside fewer, more inspectable contexts. Arms control never assumed zero leakage of knowledge. It assumed that frictions matter.

If a model has already been fully public for years, focus on the next thresholds. Do not use yesterday's leak as a universal excuse for tomorrow's reckless release.

Licenses are not containment

A restrictive license that forbids military use or warrants responsible behavior does not stop a malicious actor from clicking download. Licenses help against compliant firms and against some app-store ecosystems. They are not a technical control.

Treat license language as a complement to access control, not a substitute. If the file is public, assume adversarial fine-tuning.

Graded release as a serious policy middle path

Not every model is a frontier monster. A graded regime can look like this:

• Open by default: small and narrow models, tools, eval harnesses, safety datasets that do not themselves enable catastrophe.
• Registered access: mid-tier general models for verified researchers and firms with audit trails.
• Highly restricted: near-frontier general models with strong cyber, bio, or autonomy skills; staged deployment; independent evals.
• Prohibited to release publicly: systems at or beyond thresholds tied to catastrophic risk, including precursors to uncontrolled SI.

The political fight is over where to place the lines and who measures them. The structure itself is ordinary risk management.

Trusted research environments and structured access

If openness goals are reproducibility and external scrutiny, structured access can deliver much of the value. Researchers enter controlled environments where they can run experiments, red-team, and publish results without taking home weight files. Finance and medicine already use analogous controlled data rooms.

These systems must be designed carefully to avoid capture by the host lab. Independent governance, logging, and multi-party oversight matter. Done well, they undercut the false binary of "full open" versus "blind trust in the lab."

Open source tooling is still essential

None of this is an argument against open compilers, open kernels, open cryptographic libraries, or open safety evaluation tools. Those projects expand defense and scientific capacity. Conflating open tooling with open frontier weights is a category error that helps reckless releasers and confuses legislators.

If you work in open source, you can strengthen prevention by building monitoring tools, eval suites, interpretability instrumentation, and secure enclave workflows. That is open source as civilizational immune system rather than as proliferation engine.

The China factor and other geopolitical angles

Some U.S. voices argue that open weights are a way to set global standards and undercut adversaries' closed stacks. Some argue the opposite: open weights feed adversaries. Reality is conditional on what is released. Open-sourcing a near-frontier general model because a rival might train one anyway can accelerate the rival while also arming nonstate actors your rival cannot fully control either.

Geopolitical strategy that ignores nonstate proliferation is incomplete. Strategy that ignores rival states is also incomplete. The adult approach uses export controls, domestic law, and negotiated limits, not a single slogan about openness as soft power.

Developer culture and status incentives

Open releases confer status: stars on repositories, conference prestige, recruiter attention. Closed safety work is less legible. If culture only rewards the dump of weights, culture will produce dumps.

Shift prestige toward secure capability demonstrations under controlled conditions, toward verified safety findings, and toward engineering that makes monitoring possible. Funding bodies and conferences can change what counts as a win. So can senior engineers who refuse to treat catastrophic risk as an externality of personal branding.

What individuals should do regarding open-weights projects

If you maintain a small educational model, continue. If you are about to release weights near the public frontier of general capability, stop and seek independent dangerous-capability evaluation first. If a company asks you to ship a release that removes the last monitoring channel on a highly capable agent stack, treat that as a professional ethics moment, not only a product moment.

If you are a user, prefer API access with strong monitoring for high-stakes applications, and support political efforts that keep the most dangerous artifacts off public file lockers.

Legislative hooks

Lawmakers can require pre-release risk assessments for models above compute or capability thresholds, mandate reporting of intentional open releases, create liability for reckless public distribution of models with demonstrated catastrophic-use potential, and fund national repositories for safe research access.

They should avoid crude bans on all open-source AI, which would overbroadly attack beneficial software and collapse political support. Precision is legitimacy.

International coordination on releases

If one country allows free release of near-frontier weights, others will face pressure to match. That is a classic race-to-the-bottom structure. Coordinated standards among compute-rich states can set a floor. Treaty work on superintelligence prevention should include a chapter on weight proliferation, not only on training runs.

Verification is harder for downloads than for training clusters, which is why prevention wants to keep the most dangerous objects from being published at all.

Connection to the Foundation's core line

The Nakada Foundation exists to prevent uncontrolled superintelligence. Open-weight policy is instrumental to that goal. A world where anyone can iterate on near-SI systems in uncontrolled settings is a world where treaty limits on training can be bypassed through distributed fine-tuning and novel scaffolding.

You can love open source and still draw this line. Draw it clearly. Defend the goods of openness where they do not buy extinction-class risk. Refuse the one-way door where they do.

Practical decision tree for a lab considering release

Start with capability evaluation by an independent team with incentives to find problems. If cyber, bio, autonomy, or AI-research acceleration metrics exceed agreed thresholds, do not open-release. If metrics are low and the model is narrow or small, open release may be justified with standard security hygiene. If metrics are ambiguous, default to structured access rather than public weights.

Document the decision. Assume it will be read after an incident. If you would be ashamed to explain the release then, do not do it now.

Closing

Open source helped build a digital commons. Catastrophic-capability weights are not just another library. The difference is irreversibility plus scale of harm. Policy that respects both the commons and the species will be graded, verified, and boring in the way good safety engineering is boring.

For the broader prevention stack, read how to stop superintelligence, compute governance, and never build superintelligence. For takeover pathways that open proliferation feeds, read AI takeover scenarios.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Implementation will also fail if penalties are cosmetic. A fine that is smaller than the marketing value of an open release is a fee, not a deterrent. Pair civil liability with the possibility of injunctions and, in egregious cases, criminal exposure for willful reckless release against clear black-letter thresholds.

Finally, implementation needs international forums where states can share signals about upcoming releases and near-miss evaluations without dumping proprietary details into the open. Intelligence liaison channels already do analogous work in other domains. Use them.

Case study pattern: when a mid-tier release was still a bad idea

Not every harmful release is a record-breaking frontier model. A mid-tier general model with strong coding skill, easy fine-tuning, and no monitoring can still expand cybercrime capacity for thousands of actors who would never train from scratch. The right question is not only "is this the best model in the world?" The right question is "does public release materially expand catastrophic or large-scale criminal capability beyond what was already trivial?"

If yes, structured access beats a public file drop. If no, ordinary open-source norms can apply. Teams get into trouble when they answer with vibes about community goodwill instead of with dangerous-capability measurements.

Case study pattern: research benefit that did not need weights on a torrent

Some of the best external safety findings of the last few years came from API access, limited researcher programs, or replications on smaller models. That history undercuts the claim that full public weights are the only path to scrutiny. Scrutiny needs scientific access and freedom to publish uncomfortable results. It does not automatically need infinite anonymous copies.

Design researcher programs with independence: multi-year funding not controlled by the lab's PR cycle, clear publication rights, and technical ability to run fine-tunes inside a secure facility. If a lab refuses those terms while demanding credit for openness, the openness is a marketing channel.

Open weights and the biosecurity interface

Biological risk from AI is a misuse-heavy domain that becomes much harder when models are local and unmonitored. Filters help on hosted APIs. Local weights make filters optional. Even if today's open models are not yet catastrophic bio oracles, the trajectory of general scientific assistance points toward a future where open release policy is a biosecurity decision as much as a software decision.

Pair AI release policy with existing biosecurity institutions rather than inventing a parallel culture from pure ML norms. People who have worked pathogen oversight already understand one-way doors.

Open weights and automated AI research

If a model can accelerate algorithm design, releasing it publicly accelerates everyone, including groups optimizing only for capability. That shortens the calendar between "controllable" and "not obviously controllable." Closed extreme capability is already hard to govern. Open extreme capability multiplies the number of unsupervised improvement loops.

A practical rule of thumb: the closer a model is to automating AI research itself, the stronger the presumption against open release. That presumption should be written into lab policy before the marketing team names a launch date.

What "open" still should mean in a prevention world

Open standards for safety evaluations. Open tools for monitoring clusters. Open educational models that cannot meaningfully perform catastrophic tasks. Open documentation of failures. Open government reporting on large training runs above thresholds. Open academic access under controls for higher-risk systems.

That list is aggressively open where openness reduces extinction-class risk or builds public capacity. It is closed where openness primarily multiplies irreversible harm. Calling that list anti-open-source is like calling a bio lab anti-science for locking a freezer.

Investor and board questions that change release decisions

• What independent evals were run pre-release, and can the board read the unredacted findings?
• What is the estimated increase in actors who can run cyber or bio-relevant workflows after release?
• What is the rollback plan if a fine-tune demonstrates a dangerous jump next month?
• How does this release affect negotiations with governments on compute thresholds?
• Who personally owns the decision, and how are they insulated from launch-marketing incentives?

If board members cannot get straight answers, they are overseeing a strategic proliferation event with a product nickname, not a normal software ship.

Employee ethics when the launch train is moving

Individual contributors often see warning signs first: eval anomalies, weak fine-tune resistance of safety layers, internal arguments that minimize irreversible risk. Escalation paths should exist on paper and in culture. When they fail, coordinated disclosures through counsel and, where justified, protected whistleblowing become part of professional duty.

No article can resolve every personal risk calculation. Organizations can still lower the cost of internal dissent. Governments can still protect good-faith reporters. Readers who manage teams can start by rewarding the person who delays a release for evidence rather than punishing them for missed ceremony dates.

How open-release rhetoric maps onto race dynamics

One common script says that open release is how the democratic world keeps an edge. Another says open release is how the democratic world disarms itself. Both scripts can be true for different artifacts. The way out is artifact-specific analysis rather than civilizational poetry.

Race dynamics become worse when actors treat every restraint as unilateral surrender. Shared rules among compute-rich states can make restraint coordinated. Open-ending the most dangerous weights makes coordination harder because nonstate and small-state actors enter the game with less to lose from defection.

Metrics for a sane open ecosystem

Track the fraction of highly capable general models under monitored access versus public weights. Track time-to-dangerous-fine-tune for newly released models in controlled red-team settings. Track whether independent researchers can obtain structured access without months of dead email. Track enforcement actions after reckless releases.

Publish those metrics annually from a public authority. Culture fights love anecdotes. Policy needs series.

A concrete staged policy package

First, define compute and capability thresholds with a technical advisory body that includes independent skeptics. Second, require pre-registration of intended open releases above a mid threshold. Third, require independent evals with publication of summary risk findings. Fourth, prohibit open release above a high threshold tied to catastrophic-use potential and AI-research acceleration. Fifth, fund national or multilateral research clouds where verified scientists can work on restricted models. Sixth, attach real penalties and injunction powers.

That package leaves most ordinary open-source software untouched. It targets the thin slice of releases that change strategic risk. Political coalition building should keep saying that out loud, because opponents will try to reframe precision as a war on hobbyists.

Hobbyists, startups, and the fairness objection

Fairness matters. A world where only three firms hold frontier systems is a power problem. The answer is not to hand strategic-capability weights to every anonymous download. The answer is public-interest compute, regulated access, antitrust where appropriate, and a flourishing open ecosystem beneath catastrophic thresholds.

Startups can build enormous value on narrow models, tools, and licensed APIs. The myth that every startup deserves frontier weights as a human right is a myth about entitlement, not about innovation. Innovation under boundary conditions is normal in aviation, medicine, and banking software.

Communication guide for advocates

Lead with irreversibility. Show a graded chart. Affirm open tools. Name specific catastrophic pathways without gore for its own sake. Offer structured access as the positive alternative. Avoid insulting open-source identities. Many potential allies live in that culture and will move if the distinction is crisp.

When debates turn tribal, return to a physical analogy: publishing a high-risk pathogen protocol is not the same as publishing a cookbook. Cognitive tools now span that kind of range. Our norms must span it too.

What this Foundation will keep saying

We are pro-narrow-AI and pro-prevention of uncontrolled superintelligence. Open weights sit on a continuum. At the low end, openness is a gift. At the high end, openness is a proliferation failure. The line should be drawn with measurements, enforced with law, and explained without contempt.

If you take one action after this guide, support political and institutional work that keeps the most dangerous artifacts out of uncontrolled public distribution while expanding safe access for real science. Then read the enforcement guide on how to stop superintelligence and push the pieces that match your position in the world.

Keep pressure on definitions in every hearing and standards meeting. If a draft law says "open source AI" without thresholds, amend it. If a company blog equates a dangerous weight drop with community care, answer with the one-way door and the eval sheet. Precision is how this debate becomes governable.

Procurement and cloud marketplaces

Even when weights are not public, marketplace listings and one-click endpoints can approximate open access for paying customers with weak KYC. Cloud providers become chokepoints. Procurement rules for government and critical infrastructure should forbid unmanaged use of models above risk thresholds and require logging.

Providers can also require attested use cases for the highest tiers, similar to know-your-customer rules in finance. Perfect prevention is impossible. Raising the cost of anonymous large-scale abuse is still worth doing.

Academic incentives and tenure files

Universities still reward novel model releases as prestige objects. Promotion committees can reward safety evaluations, secure infrastructure engineering, and governance-relevant measurements instead. Funding agencies can make secure access plans a condition of grants that train large general systems.

Students should learn release ethics beside GPU kernel optimization. A generation that can only maximize leaderboards will keep walking into one-way doors.

Incident response after a reckless release

Once weights are out, response shifts to detection of downstream fine-tunes, takedowns on cooperative platforms, warnings to defenders, and diplomatic notification if state-level risks rise. None of that restores the pre-release world. It only reduces follow-on damage.

After-action reviews should feed threshold updates. If a model classed as mid-tier produced severe misuse, the threshold was wrong. Change it in public.

Coordination among allied regulators

Allied regulators can share evaluation signals under confidentiality agreements so that one country's refusal to allow a release is not undercut by another's silence. Forum design matters: technical experts, not only trade negotiators, need seats.

Where allies disagree, the floor should still be high for the most extreme systems. A race to rubber-stamp open releases among friends is a gift to the worst downstream users.

Closing the loop to prevention

Weight policy is one rung on a larger ladder that includes compute monitoring, domestic statutes, whistleblower protection, and treaty verification. Treat it as necessary and incomplete. A perfect open-release regime with unbounded secret training still fails. Unbounded open release with perfect training treaties also fails.

Stack the rungs. Keep ordinary open source healthy. Close the one-way door at the top. That is the whole argument, and it is enough to act on.

Comparative regimes: cryptography, biotech, and aviation software

Cryptography research is widely published, yet export controls and product rules still shape how some implementations travel. Biotech publishes methods while controlling select agents and toxins. Aviation software is heavily certified even when concepts are public. None of these regimes is a perfect analogy. All of them show societies already accept that knowledge policy can be layered.

AI weight releases sit closer to biotech dual-use and aviation certification than to a text editor plugin. The field's cultural preference for unconditional release is historically contingent, not a law of nature. Contingent norms can change when stakes change.

Model theft versus intentional release

Stolen weights are a security failure. Intentional public release is a policy choice. Both can produce similar downstream proliferation, but accountability differs. Labs that underinvest in insider threat controls and exfiltration defense are running an implicit open-release risk even when their public policy looks closed.

Prevention therefore includes ordinary high-security practices for near-frontier training: compartmentalization, staged access, monitoring of large downloads, and rapid revocation. Treating weight files like public README drafts is how theft becomes destiny.

Fine-tunes, merges, and the shadow ecosystem

Public communities already merge models, strip refusals, and specialize systems for roles the original developers disclaim. That ecosystem is creative and, at low capability, mostly fine. At high capability, it becomes a factory for removing friction from harmful use.

Eval regimes should test not only base models but also common fine-tune and de-refusal recipes. If a model is one cheap fine-tune away from severe misuse, the base release decision must price that in. Ignoring the shadow ecosystem is how official safety cards become theater.

Interoperability tools as dual-use infrastructure

Serving stacks, quantization tools, and agent frameworks increase the number of people who can run capable models locally. Those tools have legitimate uses and should generally remain open. The control point remains which weights are public, not whether efficient inference exists.

Trying to ban quantization would be both futile and misdirected. Trying to keep the most dangerous weights from becoming public is directed at the actual one-way step.

Public communication after a refused release

When a lab or regulator blocks an open release, expect backlash framed as censorship or monopoly protection. Preempt that story with transparent criteria published in advance, third-party eval summaries, and a structured access path for legitimate researchers. Silence looks like capture. Clear criteria look like safety engineering.

Advocates outside government should defend good refusals even when they dislike the company involved. Consistency builds a norm. Selective outrage based on brand politics destroys it.

Municipal and campus computing

Universities and cities will host clusters that can fine-tune mid-to-high models. Procurement and acceptable-use policies should include AI risk tiers, not only copyright and harassment rules. Campus IT is now part of national dual-use infrastructure in miniature.

Training local administrators to recognize risky projects is unglamorous work with outsized value. A single well-run university gateway can prevent a dozen careless public mirrors.

The long horizon: if control methods improve

If, someday, control of highly capable systems becomes scientifically demonstrated and independently verified, release policy could loosen under new evidence. That is a conditional sentence with a heavy burden of proof. It is not a promise that the burden will be met soon.

Until then, the default for near-frontier general weights should stay cautious. Optimism about future breakthroughs is not a present-tense license to proliferate systems we cannot steer.

Putting open weights back in the treaty picture

Diplomats drafting AI agreements sometimes focus on training runs because clusters are visible. They should also write articles on distribution: prohibitions on public release above thresholds, mutual legal assistance for takedowns, and shared evaluation standards. Otherwise a treaty on training becomes a sieve.

Verification of non-release is imperfect, yet intentional corporate releases are highly visible events. Start by policing the visible, then improve detection for leaks. Perfect is not the prerequisite of better.

Your next step

If you write code, refuse reckless release pressures and help build structured access tooling. If you write policy, insert graded thresholds into drafts that currently moralize about openness in the abstract. If you give money, fund verification and secure research clouds. If you only vote and talk, make the one-way door distinction common language in your circle.

Open source remains a pillar of digital life. Catastrophic-capability proliferation is not a pillar. Keep the first. Stop the second. Then continue with the wider prevention program on this site until enforcement, not rhetoric, is doing the work.

Appendix: language to use and language to drop

Use "near-frontier general weights," "structured access," "one-way door," "dangerous-capability evaluation," and "graded release." Drop "all open source is dangerous," "information wants to be free" as a policy closer, and "if we do not release, someone worse will" without evidence about the specific model class. The first set supports law. The second set supports slogan combat.

Journalists can help by asking which threshold a release crossed and who measured it. A story that only quotes two tribes yelling "freedom" and "doom" teaches the public nothing they can vote on.

Appendix: minimal internal release checklist

Before any intentional distribution of general model weights, a lab should complete independent cyber, bio, autonomy, and AI-research acceleration evals; document residual risk; check threshold tables in the current policy; choose public open, registered, or restricted research access; record the named decision owner; and schedule a post-release monitoring window with authority to warn downstream platforms. If any box is empty, the launch is not mature.

Checklists fail when they become paperwork theater. They work when empty boxes can stop a ceremony. Build that stop into the org chart before the marketing calendar fills.

Appendix: why this page is long

Short takes get captured by slogans. This subject needs room for steelman arguments, threat models, institutional design, and clear limits that protect ordinary open source. Length here is a feature for searchers who want a complete reference and for staffers who need more than a thread.

If you only needed one paragraph, take this one: keep open tools and narrow models flourishing; stop irreversible public release of near-frontier general systems that expand catastrophic misuse and undermine superintelligence prevention; replace the false binary with graded access and verification. Then go enforce it.

Share this distinction until it is boring. Boring norms are the ones institutions can enforce. Exciting slogans are the ones that start flame wars and end without inspectors, thresholds, or halt authority. Superintelligence prevention needs the boring version of openness: wide where it helps civilization, closed where a download cannot be undone.

When the next launch blog post waves the open-source flag over a near-frontier general model, read the evals first, then the license, then the threat model. If the evals are missing, treat the flag as decoration. Civilization does not owe any company a round of applause for unlocking a door that cannot be locked again.