AIxBio Safeguards
Ensure that AI systems capable of substantially accelerating biological misuse incorporate robust safeguards.
Last updated: October 1, 2026
5-year budget target
Pillar
Five-Year Trajectory
2027
Secure separate biorisk evaluation commitments for frontier LLMs, biological AI models, and autonomous AI-lab workflows
2028
Make LLM, biological AI model, and autonomous-lab biorisk evaluation routine for relevant developers
2029
Mandate safeguards for high-risk LLMs, biological AI models, and autonomous AI-lab systems
2030
Achieve broad compliance with synthesis screening, KYC, AIxBio safeguards across distinct model tracks, and misuse monitoring
2031
Sustain AI-bio safeguards across frontier LLMs, biological AI models, and autonomous-lab systems
Current Status of the Field
LLM bio-risk evaluations and mitigations are emerging, but evaluations and mitigations for biological AI models, agentic bioscience workflows, and autonomous labs remain underdeveloped and fragmented.
Frontier AI commitments increasingly mention biological risk, but requirements rarely distinguish LLMs, specialized biological models, and AI systems linked to laboratory execution. Emerging legal requirements apply primarily to frontier labs, but not to the broader ecosystem of labs developing biological AI models and other scientific tooling.
A key bottleneck is creating effective incentive structures–including through regulations, liability, and financial incentives–for developers of LLMs, agentic systems, biological AI models, and autonomous labs to deploy robust evaluations and mitigations to guard against misuse. As part of this effort it will be important to clearly distinguish different sets of stakeholders who face different incentives and operational requirements, while preserving a common threshold for biological capabilities of concern and a common expectation of independent evaluation.
The Problem23,37,38
AI is rapidly lowering the expertise barrier for biological weapons design, and safeguards have not kept pace. Frontier LLMs, biological AI models, agentic systems, and autonomous-lab workflows share some risks but have different developers, evaluation tools, users, and failure modes. Not all frontier models undergo biological risk evaluation before deployment, bio-specific model evaluation tooling remains immature, and there are no widely adopted norms governing model release,39 high-risk biological training data,40 trusted and monitored access, or AI systems that can move from biological design toward physical creation.
The Solution
While further research is needed to develop more effective evaluations and mitigations, a meaningful set of technical and governance interventions already exists, including: evaluation standards, managed access, trusted user verification, LLM mitigation deployment, and independent red-teaming. The immediate bottleneck is that many AI developers do not conduct appropriate biorisk evaluation in the first place, and biological AI models often need different evaluation tooling and mitigations than general-purpose LLMs. Securing commitments for risk reduction actions for frontier LLMs and agents, biological AI models, and autonomous-labs is therefore the urgent first move. The longer-term goal is broader: consensus definitions of AIxBio capabilities of concern, full evaluation coverage and effective mitigations for relevant models, secure handling of high-risk biological training data, no open-source proliferation of AIxBio capabilities of concern, and carefully monitored managed access pathways for legitimate uses of especially powerful dual-use AIxBio capabilities.
Objectives
- ◆Consensus definitions of AIxBio capabilities of concern are adopted to inform evaluation and safeguards nationally and internationally.
- ◆High-risk biological training data are generated and shared only in secure environments.
- ◆Limit the ability of open-source models to proliferate the capability to design pandemic pathogens with the potential to cause a global biological catastrophe.
- ◆100% of frontier LLMs with biological capabilities of concern undergo pre-deployment biorisk evaluation and deploy effective and proportionate safeguards.
- ◆100% of biological AI models and other specialized biological AI systems with capabilities of concern undergo dedicated evaluation using bio-specific tooling, not only general-purpose LLM tests.
- ◆Robust mitigations41,42 are developed and deployed across AI systems43 to guard against misuse of AIxBio capabilities, including LLMs, agentic bioscience workflows, biological AI models,41 and autonomous labs.
- ◆Autonomous AI-lab workflows that connect design tools to physical creation pathways are evaluated and governed before they can operate at scale.
- ◆Defensive use is accelerated with secure early access to monitored models under a two-tier system: general frontier models with limited bio capabilities and safeguards to keep those limited, and monitored bio-helpful models with adequate KYC and oversight. This is similar to Project Glasswing as Anthropic’s Mythos model was released.44
- ◆Biological AI models and other specialized biological AI systems with especially powerful capabilities of concern–whether deployed by frontier labs, academic labs, startups, or other groups–are protected by trusted access frameworks45–50 that enable responsible scientists, who undergo know-your-customer vetting, to use those tools.
- ◆A Guardian AI system operates as a continuous, real-time monitoring layer across frontier AI deployments with biological capabilities — detecting anomalous query patterns, novel jailbreak attempts, and coordinated misuse campaigns, with the authority to throttle or suspend access autonomously when risk thresholds are exceeded
Urgent 2027 Milestone
Secure binding biorisk evaluation and safeguard commitments for both frontier LLMs and biological AI models, and launch a dedicated pilot for autonomous lab plus AI risk.
Long-term Targets
Evaluation coverage
Safeguard coverage
Data security coverage
Year-by-Year
Philanthropy
- •Convene international processes to define biological capabilities of concern for frontier AI and biological models
- •Fund separate evaluation standards for frontier LLM biological risk and biological AI model risk
- •Fund pilot evaluation of autonomous AI-lab workflows that connect biological design to physical creation pathways
- •Fund pilot safeguards, including managed access and monitoring, for agentic systems and high-risk biological AI models
- •Fund independent AI-bio red-teaming
Private Sector
- •Conduct pre-deployment biorisk evaluations for relevant frontier LLMs and submit them to independent red-teaming
- •Develop and test bio-specific evaluation tooling for biological AI models
- •Identify autonomous-lab integrations that require evaluation before broader deployment
- •Implement proven safeguards for models that exhibit capabilities of concern
- •Support information sharing and norm development among frontier developers
Government
- •Adopt or begin adopting policy definitions of biological capabilities of concern for AI systems
- •Support standards development for pre-deployment AI biorisk evaluation across separate LLM, biological AI model, and autonomous-lab tracks
- •Use formal agreements, procurement conditions, or other mechanisms to encourage safeguard adoption
- •Begin developing policy approaches for secure handling of high-risk biological training data and responsible model release
Philanthropy
- •Continue funding deployment of screening software and independent audit capacity across providers, including smaller and international firms
- •Support implementation-focused KYC tools and verification infrastructure
- •Continue funding separate AI-bio evaluation tracks for frontier LLMs, biological AI models, and autonomous-lab workflows
- •Support broader bio-audit adoption, institutional awareness, and development of publication-risk-management norms
- •Continue support for misuse-detection tooling and cross-provider threat-sharing systems
Private Sector
- •Implement standardized synthesis screening and customer verification at scale
- •Participate in split-order detection systems and submit to independent screening verification where possible
- •Expand KYC adoption across high-risk vendors and service providers
- •Make AI-bio evaluation and safeguard deployment routine across separate LLM, biological AI model, and autonomous-lab tracks
- •Expand participation in bio-audits, accident reporting, pre-publication review, and institutional adoption pilots
- •Routinely share threat signals and maintain misuse monitoring
Government
- •Move from legislative or rulemaking activity to initial compliance and oversight in synthesis screening and KYC
- •Support independent verification of AI biorisk evaluation and safeguard adoption through track-specific formal agreements or requirements
- •Expand implementation of accident-reporting requirements, research-risk oversight, and institutional participation pathways
- •Support law-enforcement pathways for escalating provider-generated threat signals
Philanthropy
- •Support more advanced evasion-resistant and function-based screening R&D
- •Support scaling of KYC tools and red-teaming across a broader set of high-risk service categories
- •Support secure data infrastructure for high-risk biological AI training data and continued safeguard research across LLM, bio-AI-model, and autonomous-lab tracks
- •Support global governance reform, institutional adoption, and continued capacity-building for bio-audits and risk-based research oversight
- •Support public communication efforts that reinforce deterrence posture and accountability
Private Sector
- •Submit to independent screening verification and deploy updated screening algorithms
- •Extend KYC and more extensive customer vetting to newly covered categories such as CROs, cloud labs, repositories, and AI/bio tools
- •Continue safeguard deployment for high-risk AI systems and participate in track-specific red-teaming
- •Expand compliance with institutional risk-governance requirements, adoption programs, and misuse monitoring mandates
Government
- •Enforce synthesis-screening compliance and audit requirements
- •Regulate the secondary market for benchtop synthesis devices and broaden KYC requirements to additional high-risk categories
- •Mandate safeguards for high-risk AI systems with biological capabilities across separate LLM, biological AI model, and autonomous-lab pathways
- •Institutionalize accountable public oversight for high-consequence biological research, including licensing, institutional participation, and bio-audit frameworks
- •Institute requirements for misuse monitoring and threat sharing for high-risk biological services
Philanthropy
- •Focus remaining philanthropic capital on gap-closing, deterrence signaling, and ensuring high-risk systems actually perform under stress
- •Support maturation of attribution capabilities and public communication around accountability
- •Continue targeted support for jurisdictions, sectors, or institutions lagging in compliance, AIxBio safeguards, or research-governance adoption
Private Sector
- •Achieve broad compliance with screening, KYC, safeguard, and misuse-monitoring requirements
- •Maintain regular red-teaming, audit participation, and rapid remediation processes
- •Support stronger bioforensic attribution capability through data-sharing and exercise participation
Government
- •Ensure broad compliance and meaningful enforcement across screening, KYC, AIxBio safeguards, and research governance
- •Verify that high-risk LLM, biological AI model, and autonomous-lab safeguards perform under stress
- •Establish or strengthen attribution and accountability doctrine for biological attacks
- •Exercise bioforensic and deterrence capabilities routinely
Philanthropy
- •Focus on closing residual gaps, supporting lagging geographies or sectors, and maintaining pressure for continuous improvement
- •Continue support for frontier-risk research, AI-bio track-specific evaluation, institutional research-governance adoption, residual-risk assessment, and measurement refinement
Private Sector
- •Maintain high compliance and continuous improvement in screening, KYC, safeguards, red-teaming, and misuse monitoring
Government
- •Sustain enforcement, update standards to keep pace with new technologies, and ensure risk-governance and AI-bio safeguard systems remain current
Who's Working on This
Funders
regranting
regranting
regranting
regranting
Implementers
field-building
field-building
field-building
field-building
field-building
ai
research
research
field-building
field-building
field-building
field-building
field-building