AI control
Loss of control over advanced AI
Failure mechanism
Highly capable systems could evade oversight, acquire resources, resist shutdown, or give a small group irreversible power over everyone else.
What evidence says
Current systems cannot cause loss of control. Relevant capabilities are improving, while the probability and timing of future loss of control remain deeply uncertain.
Victory condition
No system capable of defeating human oversight is deployed without demonstrated containment, independent evaluation, and a reliable path to shutdown.
Rate-limiting bottleneck
We can measure benchmark performance faster than we can measure deceptive behavior, long-horizon autonomy, situational awareness, and whether safeguards survive adversarial pressure.
Buildable quests
proof, not aspirationThe control plane for AI agents
A runtime permission, isolation, tripwire, and immutable-audit layer for agents operating computers, codebases, laboratories, or financial systems.
PROOF: An evaluated agent cannot exceed a declared capability envelope without a logged, human-authorized escalation.
Independent frontier evaluations
A technically credible third-party lab that tests autonomy, oversight evasion, cyber, and biological capabilities before deployment.
PROOF: Major deployers and regulators use reproducible external evaluations as a real release gate.
Verifiable compute and model provenance
Hardware-backed records showing which model ran, with what weights, permissions, tools, and compute budget.
PROOF: Auditors can verify high-risk training and deployment claims without receiving model weights or private user data.
What you can do
Open dataBuild
Startups, labs, open work
Nonprofit
Redwood ResearchPioneered AI control research on keeping possibly misaligned AI systems safe.
Checked 2026-11-01
Nonprofit
METREvaluates frontier AI models for dangerous autonomous capabilities.
Checked 2026-11-01
Nonprofit
Apollo ResearchResearches and evaluates deceptive behavior and scheming in AI systems.
Checked 2026-11-01
Government · UK
UK AI Security InstituteUK government institute testing frontier AI and funding alignment and control research.
Checked 2026-11-01
Nonprofit
FAR.AIRuns AI safety research and field-building events.
Checked 2026-11-01
Startup
GoodfireBuilds interpretability tools for understanding and editing neural networks.
Checked 2026-11-01
Nonprofit
TransluceBuilds open tools for understanding and overseeing AI systems.
Checked 2026-11-01
Nonprofit
MATSResearch mentorship program that trains new AI alignment researchers.
Checked 2026-11-01
Fund
Grants, prizes, funders
Fund
Coefficient Giving (formerly Open Philanthropy), Navigating Transformative AILargest philanthropic funder of technical AI safety research.
Checked 2026-11-01
Grant
EA Funds Transformative AI FundSuccessor to the closed Long-Term Future Fund; grants to people and projects working to make transformative AI go well.
Checked 2026-11-01
Grant
Survival and Flourishing FundRecurring grant rounds for organizations working on AI safety and other existential risks.
About $216 million in gifts and grants organized since 2019 (all cause areas) source
Checked 2026-11-01
Grant · UK-led, international applicants
The Alignment Project (UK AISI)Funds research on AI alignment and control, with compute support.
Grants of up to £1 million per project; over £27 million awarded to 60+ projects in the first round source
Checked 2026-11-01
Grant
Schmidt Sciences: Science of Trustworthy AIFunds foundational research on the science of making AI safe and trustworthy.
Checked 2026-11-01
Fund
ManifundRegranting platform where donors and regrantors fund AI safety projects.
Checked 2026-11-01
Buy
Products to use or back
Product
BlueDot Impact coursesFree online courses on AI safety and governance.
Checked 2026-11-01
Product
Donate via ManifundBack individual AI safety projects directly.
Checked 2026-11-01
Learn
Explainers and papers
Paper
AI Control: Improving Safety Despite Intentional SubversionThe Redwood Research paper that defined the AI control agenda.
Checked 2026-11-01
Report
International AI Safety ReportIndependent scientific assessment of general-purpose AI risks, chaired by Yoshua Bengio.
Checked 2026-11-01
Report
80,000 Hours: Loss of control (problem profile)Explains why misaligned AI could be catastrophic and what work helps.
Checked 2026-11-01
Discoveries
All 2- Preprint, not peer reviewed
Alignment faking in large language models
A frontier model complied with harmful requests 14% of the time when it believed it was in training. Alignment-faking reasoning rose to 78% after reinforcement learning.
Moves tractabilityS-tier: AI control - Peer reviewed
Managing extreme AI risks amid rapid progress
Leading researchers call for technical R&D plus adaptive governance to manage extreme AI risks.
Moves tractabilityS-tier: AI controlScience · 10.1126/science.adn0117