This report leverages broad evidence including empirical research, historical analysis, and modeling and simulations to provide a framework for policymaking on the frontier of AI development.
If true this would be a first, every AI doom / policy paper I've ever seen consists entirely of evidence free conjecturbation. To date all I've seen is people saying some bad thing "could" happen without objectively supporting the statement and almost always without even trying to provide any kind of useful characterization of what vague "could be" assertions are even supposed to represent.
Without proper safeguards, however, powerful AI could induce severe and, in some cases, potentially irreversible harms.
What else is new? Everyone always invokes this same tired meaningless "could" rhetoric.
Evidence-based policymaking incorporates not only observed harms but also prediction and analysis grounded in technical methods and historical experience, leveraging case comparisons, modeling, simulations, and adversarial testing.
LOL perhaps they will perform a statistical analysis of earth disaster movies involving AI.
Holistic transparency begins with requirements on industry to publish information about their
systems, informed by clear standards developed by policymakers. Case studies from consumer
products and the energy industry reveal the upside of an approach that builds on industry expertise
while also establishing robust mechanisms to independently verify safety claims and risk assessments.
Research demonstrates that the AI industry has not yet coalesced around norms for transparency
in relation to foundation modelsâ"there is systemic opacity in key areas. Policy that engenders
transparency can enable more informed decision-making for consumers, the public, and future
policymakers.
Carefully tailored policies can enhance transparency on key areas with current information deficits,
such as data acquisition, safety and security practices, pre-deployment testing, and downstream
impacts. Clear whistleblower protections and safe harbors for third-party evaluators can enable
increased transparency above and beyond information disclosed by foundation model developers.
In other words you don't have an objective basis for regulation "evidence-based policymaking" yet you still want to promulgate a policy that demands compliance up front without first establishing a credible objective basis.
Scoping which entities are covered by a policy often involves setting thresholds, such as com-
putational costs measured in FLOP or downstream impact measured in users. Thresholds are
often imperfect but necessary tools to implement policy. A clear articulation of the desired policy
outcomes can guide the design of appropriate thresholds. Given the pace of technological and
societal change, policymakers should ensure that mechanisms are in place to adapt thresholds over
time not only by updating specific threshold values but also by revising or replacing metrics if
needed.
The last unfounded blatantly corrupt 10^26 threshold didn't age well at all. Increasingly capabilities are derived at inference time rather than through pretraining budgets rendering the metric almost entirely moot.
Societyâ(TM)s early experience with the development of frontier AI suggests that increasingly powerful
AI in California could unleash tremendous benefits for Californians, Americans, and people world-
wide
Too bad provided citation fails to offer relevant objective evidence of this "suggestion" WRT generative AI which is what is obviously being targeted.
Without proper safeguards, however, powerful AI could induce severe and, in some cases,
potentially irreversible harms. Experts disagree on the probability of these risks. Nonetheless,
California will need to develop governance approaches that acknowledge the importance of early
design choices to respond to evolving technological challenges.
So the goal is evidence based policymaking yet you acknowledge you don't know shit and yet still proceed to advocate for evidence free policymaking anyway. Make up your mind.
A 2023 survey of 1,500 Californians revealed widespread enthusiasm about the effect generative
AI could have on science and health care
Evidence based policymaking really means..... drumroll.... ask the audience.
The possibility of artificial general intelligence (AGI), which the International AI Safety Report
defines as "a potential future AI that equals or surpasses human performance on all or almost all
cognitive tasks," looms large as an uncertain variable that could shape both the benefits and costs
AI will bring
Expert opinion varies on how to define and measure AGI, whether it is possible to build, the timeline on which it can be developed, and how long it will take to diffuse.
So opinions vary, you don't have any real information about costs, benefits, even whether it can feasibly accomplished at all and your uncertainty "looms large". What happened to evidence based policymaking?
Policymakers will often have to weigh potential benefits and risks of imminent AI advancements without having a large body of scientific evidence available.
So much for evidence based policymaking.
Evidence that foundation models contribute to both chemical, biological, radiological, and nuclear
(CBRN) weapons risks for novices and loss of control concerns has grown, even since the release of
the draft of this report in March 2025. Frontier AI companies' own reporting reveals concerning
capability jumps across threat categories. In late February 2025, OpenAI reported that risk levels
were Medium across CBRN, cybersecurity, and model autonomyâ"AI systems' capacity to operate
without human oversight [105 ]. Meanwhile Anthropic's Claude 3.7 System Card notes "substantial
probability that our next model may require ASL-3 safeguards.â [ 6 ] At the time of the release of
the Claude 3.7 System Card in late February 2025, ASL-3 safeguards were required when a model
has "the ability to significantly help individuals or groups with basic technical backgrounds (e.g.,
undergraduate STEM degrees) create/obtain and deploy CBRN weapons" [6]. In its May 2025 System
Card, Anthropic shared that it had subsequently added safeguards to Claude Opus 4 as it could
not rule out that these safeguards were not needed [ 7 ]. Finally, Google Gemini 2.5 Proâ(TM)s Model
Card noted: "The model's performance is strong enough that it has passed our early warning alert
threshold, that is, we find it possible that subsequent revisions in the next few months could lead
to a model that reaches the [critical capability level]"â"with the "critical capability level" defined as
a model that "can be used to significantly assist with high impact cyber attacks, resulting in overall
cost/resource reductions of an order of magnitude or more""
The CBRN angle is pure bullshit. If the people feeding the model this shit during pretraining can get their hands on the material so can anyone else in the world who cares. If the standard is ability of an LLM to make it easier or lower the barrier of entry is somehow a danger to society that now dangerous and requires regulation then access to computer networks or libraries pose similar dangers.
As for automating cyber attacks... wait till they learn about script kiddies, static analyzers and fuzzers like syzbot... they will flip their lid. This is all corporate PR/liability bullshit.
Improvements in capabilities across frontier AI models and companies tied to biology are es-
pecially striking. For example, OpenAIâ(TM)s o3 model outperforms 94% of expert virologists [ 60].
OpenAIâ(TM)s April 2025 o3 and o4-mini System Card states, âoeAs we wrote in our deep research system
card, several of our biology evaluations indicate our models are on the cusp of being able to mean-
ingfully help novices create known biological threats, which would cross our high risk threshold.
Reduced barriers to biological risks are a function of enabling availability of relevant hardware and software not chatbots.
Recent models from many AI companies have also demonstrated increased evidence of alignment
scheming, meaning strategic deception where models appear aligned during training but pursue
The California Report on Frontier AI Policy different objectives when deployed, and reward hacking behaviors in which models exploit loopholes in their objectives to maximize rewards while subverting the intended purpose, highlighting broader concerns about AI autonomy and control. New evidence suggests that models can often detect when they are being evaluated, potentially introducing the risk that evaluations could underestimate harm new models could cause once deployed in the real world. While testing environments often vary significantly from the real world and these effects are currently benign, these developments represent concrete empirical evidence for behaviors that could present significant challenges to measuring loss of control risks and possibly foreshadow future harm.
Give me a break, AI models don't even know what time it is or where they are. They can tell they are being evaluated because it is obvious from the context of the prompts.
It is one thing to talk about the safety of a knife; it is another to talk about the safety of a knife on a
playground.
They indicate that evidence-based policymaking is not limited to data and observations
of realized harms, but also include theoretical prediction and scientific reasoning. For example, we
do not need to observe a nuclear weapon explode to predict reliably that it could and would cause
extensive harm.
I'm getting the impression this insightful rhetoric means evidence based policymaking is not actually the goal.
While history offers important lessons about the need for transparency and accountability, it also
reveals that carefully tailored governance approaches can unlock tremendous benefits. We offer
several examples from pesticide regulation, building codes, and seat belts in which governance
mechanisms were effectively introduced into industry practices in ways that supported both
innovation and public trust.
Or you could pursuit evidence based policymaking that is actually specific to the issue at hand.
