Safety risks
Recognition
Urging an International AI Treaty: An Open Letter
Governance of Superintelligence. OpenAI, 22 May
Statement on AI Risk
Pause Giant AI Experiments: An Open Letter
Statement on Superintelligence
Quotes
The Bletchley Declaration by Countries Attending the AI Safety Summit, 1–2 November 2023 (updated 13 February 2025)
State Department Warns of Extinction-Level AI Threat | David Wood. YouTube
International Community Must Urgently Confront New Reality of Generative, Artificial Intelligence, Speakers Stress as Security Council Debates Risks, Rewards. UN Press, 18 July
Timelines
A New Moore’s Law for AI Agents
Governance of Superintelligence
The Implausibility of Intelligence Explosion
Does AI Progress Have a Speed Limit? Asterisk
The Case for Multi-Decade AI Timelines
Preparing for Launch. Institute for Progress
Survey of 2,778 AI Authors: Six Parts in Pictures
Thousands of AI Authors on the Future of AI. arXiv preprint arXiv:2401.02843
Common Ground Between AI 2027 & AI as Normal Technology
The “Most Important Century” Blog Post Series. Cold Takes
Preparing for the Intelligence Explosion. Forethought
GPT-5: The Case of the Missing Agent. Second Thoughts
Why We Might Have Superintelligence Sooner Than Most Think
The Most Important Time in History Is Now
Why Do People Disagree About When Powerful AI Will Arrive?BlueDot
Can AI Scaling Continue Through 2030? Epoch AI
“Long” Timelines to Advanced AI Have Gotten Crazy Short
The Artificial Intelligence Revolution: Part 1. Wait But Why
Scenarios for 2030.London Futurists
Safety Risks General
Keep the Future Human
System Card: Claude Opus 4 & Claude Sonnet 4. May
Existential Risks: Analyzing Human Extinction Scenarios and Related Hazards. Journal of Evolution and Technology, 9
Superintelligence: Paths, Dangers, Strategies. Oxford: Oxford University Press
The New Fire: War, Peace, and Democracy in the Age of AI. Cambridge, MA: MIT Press
AI Safety, Ethics, and Society Textbook: Table of Contents
An Overview of Catastrophic AI Risks. Safe.ai
Catastrophic AI Scenarios. Future of Life Institute
If Anyone Builds It, Everyone Dies. New York: Random House
Richard Ngo – Reframing AGI Threat Models [Alignment Workshop]. YouTube
Racing Through a Minefield: The AI Deployment Problem. Cold Takes
The Case for AI Doom Rests on Three Unsettled Questions. Lawfare
#HaltAGI: Why We Should Not Seek to Create Artificial General Intelligence. YouTube playlist
AI and the Future of Humanity: Analysis, Insights and Foresight on Artificial Intelligence. FuturistGerd.com
London Futurists Podcast. Buzzsprout
Preparing for the Intelligence Explosion. Forethought
Uncontrollable: The Threat of Artificial Superintelligence and the Race to Save the World. Independently published
The Precipice: Existential Risk and the Future of Humanity. London: Hachette Books
The Existential Risk of Superintelligent AI
Human Compatible: AI and the Problem of Control. London: Allen Lane
Examining Popular Arguments Against AI Existential Risk: A Philosophical Analysis
Life 3.0: Being Human in the Age of Artificial Intelligence. London: Allen Lane
Future Risks of Frontier AI: Which Capabilities and Risks Could Emerge at the Cutting Edge of AI in the Future? Technology & Science Insights and Foresight.
A Taxonomy of Systemic Risks from General-Purpose AI. RAND Working Paper Series (forthcoming)
Removing the pressure to rush. dw2
International AI Safety Report 2026 (DSIT 2026/001, 2026)
Race
The AI Power Paradox. Foreign Affairs, 16 August
Artificial Intelligence Development Races in Heterogeneous Settings. arXiv (Cornell University)
Strategic Insights from Simulation Gaming of AI Race Dynamics. Futures, 103563
To Regulate or Not: A Social Dynamics Analysis of the Race for AI Supremacy. arXiv (Cornell University)
The Manhattan Trap: Why a Race to Artificial Superintelligence Is Self-Defeating. Convergence Analysis
Beyond Accidents and Misuse: Decoding the Structural Risk Dynamics of Artificial Intelligence. AI & Society
The Race for an Artificial General Intelligence: Implications for Public Policy. AI & Society, 35, pp. 367–379
Collective Action on Artificial Intelligence: A Primer and Review. Technology in Society, 66, 101649
The Suicide Region: Option Games and the Race to Artificial General Intelligence. arXiv (Cornell University)
Alignment Problem/Rogue AI
The AI Power Paradox. Foreign Affairs, 16 August
Artificial Intelligence Development Races in Heterogeneous Settings. arXiv (Cornell University)
Strategic Insights from Simulation Gaming of AI Race Dynamics. Futures, 103563.
To Regulate or Not: A Social Dynamics Analysis of the Race for AI Supremacy. arXiv (Cornell University)
The Manhattan Trap: Why a Race to Artificial Superintelligence Is Self-Defeating. Convergence Analysis
Beyond Accidents and Misuse: Decoding the Structural Risk Dynamics of Artificial Intelligence. AI & Society
The Race for an Artificial General Intelligence: Implications for Public Policy. AI & Society, 35, pp. 367–379
Collective Action on Artificial Intelligence: A Primer and Review. Technology in Society, 66, 101649
The Suicide Region: Option Games and the Race to Artificial General Intelligence. arXiv (Cornell University)
Misuse
The AI Power Paradox. Foreign Affairs, 16 August
Artificial Intelligence Development Races in Heterogeneous Settings. arXiv (Cornell University)
Strategic Insights from Simulation Gaming of AI Race Dynamics. Futures, 103563
To Regulate or Not: A Social Dynamics Analysis of the Race for AI Supremacy. arXiv (Cornell University)
The Manhattan Trap: Why a Race to Artificial Superintelligence Is Self-Defeating. Convergence Analysis
Beyond Accidents and Misuse: Decoding the Structural Risk Dynamics of Artificial Intelligence. AI & Society
The Race for an Artificial General Intelligence: Implications for Public Policy. AI & Society, 35, pp. 367–379
Collective Action on Artificial Intelligence: A Primer and Review. Technology in Society, 66, 101649
The Suicide Region: Option Games and the Race to Artificial General Intelligence. arXiv (Cornell University)
Gradual Disempowerment
Moore’s Law for Everything
All Can Be Lost: The Risk of Putting Our Knowledge in the Hands of Machines. The Atlantic, November
The Glass Cage: Who Needs Humans Anyway? New York: Vintage
Artificial Intelligence and Work – Building a New Employment Paradigm. Buenos Aires: Editorial Astra
The Economic Singularity. Oxford: Three C’s Publishing
A Taxonomy of Jobs Deeply Resistant to TAI Automation. Convergence Analysis
Threshold 2030: AI Risk, Capability Forecasting, and Global Readiness. Convergence Analysis
The Intelligence Curse
Rise of the Robots: Technology and the Threat of Mass Unemployment. London: Oneworld Publications
Humans Need Not Apply. YouTube
AI and Leviathan: Part I
Artificial Intelligence and Democracy. UNESCO Office Montevideo & Latin American Social Sciences Council
Economic Policy Challenges for the Age of AI. GovAI
Scenarios for the Transition to AGI. GovAI
Gradual Disempowerment: Systemic Existential Risks from Incremental AI Development. arXiv (Cornell University)
The Future of Work: Inequality, Artificial Intelligence, and What Can Be Done About It: A Literature Review. arXiv preprint
AGI, Government, and the Free Society. [online]. AI Policy Perspectives
The Tech Coup: How to Save Democracy from Silicon Valley. Princeton, NJ: Princeton University Press
A World Without Work. London: Allen Lane
The Future of the Professions: How Technology Will Transform the Work of Human Experts. Oxford: Oxford University Press
AI, Inequality, and Democratic Backsliding. Tech Policy Press
Military
‘Lavender’: The AI machine directing Israel’s bombing spree in Gaza. +972 Magazine
‘Lethal Autonomous Systems and the Plight of the Non-Combatant’. In: Perspectives on Lethal Autonomous Weapons Systems (LAW), Springer, pp. 215–243
What if robots go to war? World Economic Forum
Autonomous Weapons: An Open Letter from AI & Robotics Researchers
Lethal Autonomous Weapons Pledge - Future of Life Institute. Future of Life Institute
Educating about Autonomous Weapons - Future of Life Institute
Weapons of Math Destruction. New York: Penguin Random House
Stop Killer Robots
AI & Autonomous Weapons | David Wood. YouTube
Deliberating Autonomous Weapons. GovAI
Executive Order 13859: Maintaining American Leadership in Artificial Intelligence, Federal Register 84 pp.3967-3972
DoD Directive 3000.09: Autonomy in Weapon Systems. Office of the Under Secretary of Defense for Policy, 25 January 2023. Approved by Deputy Secretary Kathleen H. Hicks
Technical solutions
Constitutional AI: Harmlessness from AI Feedback. arXiv (Cornell University)
Technical Requirements for Halting Dangerous AI Activities. arXiv (Cornell University)
Superintelligent Agents Pose Catastrophic Risks: Can Scientist AI Offer a Safer Path? arXiv (Cornell University)
d/acc: one year later. Eth.limo
4.1: Safety Engineering | AI Safety, Ethics, and Society Textbook. Aisafetybook.com
Enhancing Model Safety through Pretraining Data Filtering. Anthropic Alignment Science
‘Should Humanity Build a Global AI Nanny to Delay the Singularity Until It’s Better Understood?’ Journal of Consciousness Studies, 19(1–2), pp. 96–111
Model Evaluation & Threat Research. METR
‘Deep Ignorance: Filtering Pretraining Data Builds Tamper-Resistant Safeguards into Open-Weight LLMs’. arXiv preprint arXiv:2508.06601
Open Problems in Technical AI Governance. GovAI
Human Compatible: AI and the Problem of Control. London: Allen Lane
Mechanisms to Verify International Agreements About AI Development. November. Machine Intelligence Research Institute
‘Corrigibility’. In: AAAI Workshops: Workshops at the Twenty-Ninth AAAI Conference on Artificial Intelligence, Austin, Texas, 25–26 January 2015. AAAI Publications
AI Tools for Existential Security. Forethought
Corporate Governance
Anthropic’s Transparency Hub. [online] Anthropic.com
Governing the Machine. Bloomsbury Publishing
FLI AI Safety Index 2024. Future of Life Institute
Third-Party Compliance Reviews for Frontier AI Safety Frameworks. GovAI
In-House Evaluation Is Not Enough: Towards Robust Third-Party Flaw Disclosure for General-Purpose AI. GovAI
The Promise and Perils of Nontraditional Corporate AI Governance. Lawfare
Key Components of an RSP. Metr.org
Responsible Scaling Policies (RSPs). Metr.org
Towards Best Practices in AGI Safety and Governance. GovAI
AI Lab Watch. Ailabwatch.org
Can Preparedness Frameworks Pull Their Weight? Federation of American Scientists
Assessing Risk Relative to Competitors: An Analysis of Current AI Company Policies. Gov.AI
Global Governance
Urging an International AI Treaty: An Open Letter
Coordinated Pausing: An Evaluation-Based Coordination Scheme for Frontier AI Developers. GovAI
An Interview with OpenAI CEO Sam Altman - HKS Student Policy Review. HKS Student Policy Review
Written Testimony for “Oversight of A.I.: Principles for Regulation” before the U.S. Senate Judiciary Committee, Subcommittee on Privacy, Technology, and the Law, 25 July 2023
Frontier AI Regulation: Managing Emerging Risks to Public Safety. arXiv.org
AI Governance to Avoid Extinction: The Strategic Landscape and Actionable Research Questions. arXiv (Cornell University)
What AI evaluations for preventing catastrophic risks can and cannot do. arXiv (Cornell University)
AI Policy Proposals. Existential Risk Observatory, November
Belaboring the Algorithm: Artificial Intelligence and Labor Unions. Yale Journal on Regulation
My recent lecture at Berkeley and a vision for a ‘CERN for AI’
The AI Triad and What It Means for National Security Strategy. Center for Security and Emerging Technology (CSET)
The Oxford Handbook of AI Governance. Oxford University Press
Wendell Wallach: AI & Governance
The Intersection of AI, Ethics, & Humanity, with Wendell Wallach. Carnegiecouncil.org
The Convergence of Artificial Intelligence and the Life Sciences: Safeguarding Technology, Rethinking Governance, and Preventing Catastrophe. NTI | Bio
Governing artificial intelligence: Ethical, legal and technical opportunities and challenges. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 376(2133)
8.1: Governance | AI Safety, Ethics, and Society Textbook. Aisafetybook.com
2024 State of the AI Regulatory Landscape. Convergence Analysis
The Political Philosophy of AI. John Wiley & Sons
Framework Convention on Artificial Intelligence and Human Rights, Democracy and the Rule of Law
Methodology for the Risk and Impact Assessment of Artificial Intelligence Systems from the Point of View of Human Rights, Democracy and the Rule of Law (HUDERIA)
Getting the Future Right: Artificial Intelligence and Fundamental Rights. Publications Office of the European Union
AI Alignment Podcast: Machine Ethics and AI Governance with Wendell Wallach - Future of Life Institute. Future of Life Institute
High-level summary of the AI Act | EU Artificial Intelligence Act. Artificialintelligenceact.eu
Approaches to Responsible Governance of GenAI in Organizations. arXiv (Cornell University)
Global Governance of the Transition to Artificial General Intelligence. De Gruyter
Global Governance of the Transition to Artificial General Intelligence: Issues and Requirements. Berlin: De Gruyter
AI Governance and Baruch: An Irresistible Force and an Immovable Object. One World Trust
Video and Transcript of Presentation on Introduction to Compute Governance. Blog - Lennart Heim
Training Compute Thresholds: Features and Functions in AI Regulation. GovAI
Governing Through the Cloud. GovAI
Superintelligence Strategy. Nationalsecurity.ai
Interim Report on AI Governance. United Nations. Available via UN, WIRED and The Guardian summaries
How Leaders in the Global South Can Devise AI Regulation that enables Innovation. Tony Blair Institute
ISO Policy Brief: Harnessing International Standards for Responsible AI Development and Governance. ISO
Chapter in Global Governance of AI: Challenges and Opportunities. NCBI Bookshelf
‘The sovereignty–internationalism paradox in AI governance: Digital federalism and global algorithmic control’, AI and Ethics, pp. 1–14
Strengthening Resilience to AI Risk: A Guide for UK Policymakers. Centre for Emerging Technology and Security (CETaS) and the Centre for Long-Term Resilience
If-Then Commitments for AI Risk Reduction. Carnegie Endowment for International Peace
Didier Coeurnelle on AI Governance. YouTube
Envisioning a Global Regime Complex to Govern Artificial Intelligence. Carnegie Endowment for International Peace
Regulating Algorithmic Harms. Florida Law Review, forthcoming 2025. University of Michigan Law & Economics Research Paper No. 24-031; University of Michigan Public Law Research Paper No. 24-031
How Existing Liability Frameworks Can Handle Agentic AI Harms. Lawfare
Advanced AI Governance: A Literature Review of Problems, Options, and Proposals. Institute for Law & AI, AI Foundations Report 4
Concepts in Advanced AI Governance: A Literature Review of Key Terms and Definitions. Institute for Law & AI, AI Foundations Report 3
Architectures of Global AI Governance: From Technological Change to Human Choice. Oxford: Oxford University Press
Preparing for the Intelligence Explosion. Forethought
Taming Silicon Valley: How We Can Ensure That AI Works for Us. Cambridge, MA: MIT Press
Missing links in AI governance. UNESCO Publishing
Taking control: Policies to address extinction risks from advanced AI. arXiv.org
Remarks to BRICS summit: Advocated for cooperative AI governance among Global South peers. Economic Times
Unbundling AI Openness. Wisconsin Law Review (forthcoming)
OECD AI Principles: Intergovernmental Standard on Artificial Intelligence
‘The OECD Artificial Intelligence Policy Observatory’
Futures of Global AI Governance: Co-Creating an Approach for Transforming Economies and Societies. Paris: OECD
AI in the Global South: Opportunities and challenges towards more inclusive governance. Brookings Institution
AI Global Governance – What Are We Aiming For? Conclusions and Recommendations
Inference Scaling and AI Governance. Gov.AI
Bridging the Global Divide in AI Regulation: A Proposal for a Contextual, Coherent, and Commensurable Framework
Building the Pause Button. PauseAI
The Feasibility of a Pause. PauseAI
‘At the Tensions of South and North: Critical Roles of Global South Stakeholders in AI Governance’, in Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’22), pp. 1–12
Global Governance Innovation Report 2025. Washington, D.C.: The Stimson Center
The Tech Coup: How to Save Democracy from Silicon Valley. Princeton, NJ: Princeton University Press
David Wood - Vital Decentralization | The adaptive, resilient pathway to sustainable abundance. YouTube
‘New Technologies and Old Treaties’. AJIL Unbound, 114, pp. 152–157
The Cambridge Handbook of the Law, Ethics and Policy of Artificial Intelligence. Cambridge University Press
Effective, Timely and Global: The Urgent Need for Good Global Governance of AI. World Federalist Movement / Institute for Global Policy
Beyond a Piecemeal Approach: Prospects for a Framework Convention on AI. Forthcoming in: Hacker, P., Engel, A., Hammer, S. & Mittelstadt, B. (eds) The Oxford Handbook on the Foundations and Regulation of Generative AI. Oxford University Press
‘How Should We Regulate AI? Current Rules and Principles as Basis for “Responsible Artificial Intelligence”’, in: Bruns, B. et al. (eds.) Legal Theory and Interpretation in a Dynamic Society, vol. 19, 1st ed. 2021, pp. 279–310
Do We Want an ‘IAEA for AI’? Lawfare
Transnational Working Group on the Global Governance of AI and other disruptive technologies (TWG on AI)
AI Watch: Global regulatory tracker | White & Case LLP. [online] www.whitecase.com
Regulatory Supervision of Frontier AI Developers. GovAI
The Governance Misspecification Problem. Institute for Law & AI, Working Paper No. 3-2024
Mulitilateral fora
‘A Framework for the International Governance of AI’. Carnegie Council for Ethics in International Affairs
‘Create an IPCC-like Body to Harness Benefits and Combat Harms of Digital Tech’. Nature, 617(7961), pp. 462–464
Existing Authorities for Oversight of Frontier AI Models. Institute for Law & AI, Working Paper No. 1-2024
‘Fragmentation and the Future: Investigating Architectures for International AI Governance’. Global Policy
‘Regulating Artificial Intelligence: Proposal for a Global Solution’. Government Information Quarterly, 39(4), p. 101748
‘Multilateralism and Artificial Intelligence: What Role for the United Nations?’
International Institutions for Advanced AI’. arXiv.
‘Policy Brief: The Creation of a G20 Coordinating Committee for the Governance of Artificial Intelligence’. AI and Ethics
‘UN High-Level Panel on Digital Cooperation: A Proposal for International AI Governance’. Centre for the Study of Existential Risk & Leverhulme Centre for the Future of Intelligence
International AI Institutions: A Literature Review of Models, Examples, and Proposals. AI Foundations Report 1. Institute for Law & AI
‘Why we need an intergovernmental panel for artificial intelligence’
‘The World Needs a Global AI Observatory’. Noema, 29 June
‘The Case for a Global AI Observatory (GAIO)’. Carnegie Council for Ethics in International Affairs
‘Opinion—A New International AI Body Is No Panacea’. E-International Relations (blog), 11 August
‘A Reality Check and a Way Forward for the Global Governance of Artificial Intelligence’. Bulletin of the Atomic Scientists, 10 September. p. 311
The Brussels Effect and Artificial Intelligence: How EU Regulation Will Impact the Global AI Market. GovAI
‘Foundations for the Future: Institution Building for the Purpose of Artificial Intelligence Governance’. AI Ethics, 2, pp. 463–476
‘The World Needs an International Agency for Artificial Intelligence, Say Two AI Experts’. The Economist, 18 April 2023
Bilateral
United States–China Multilateralism in the Age of Military AI. CIGI Paper No. 309, December 2024. Centre for International Governance Innovation
Prospects for AI safety agreements between countries. [online] Rethink Priorities
From Competition to Cooperation: Can US-China Engagement Overcome Geopolitical Barriers in AI Governance? [online] Tech Policy Press
An International Agreement to Prevent the Premature Creation of Artificial Superintelligence. arXiv (Cornell University)
Toward ASI Stability: A Treaty Framework for US–China Cooperation on Artificial Superintelligence. Convergence Analysis
China and the United States Begin Official AI Dialogue. [online] China-US Focus.
US
Assembly Bill 621 - Deepfake Pornography (Chapter 673, Statutes of 2025; approved and filed 13 October 2025). State of California. Available
Assembly Bill 853 - California AI Transparency Act (Chapter 674, Statutes of 2025; approved and filed 13 October 2025). State of California
California Senate Bill 53 - Artificial Intelligence Models: Large Developers (Chapter 138, Statutes of 2025; chaptered 29 September 2025). State of California
Chips for Peace: How the U.S. and Its Allies Can Lead on Safe and Beneficial AI. Lawfare
California Enacted AI Bills. Now Officials Must Define Them. Lawfare
Recent U.S. Efforts on AI Policy | CISA
AI Policy Levers: A Review of the U.S. Government’s Tools to Shape AI Research, Development, and Deployment. GovAI
Preparing for Launch | IFP. Institute for Progress
Lawfare Daily: Scaling Laws: Renée DiResta and Alan Rozenshtein on the ‘Woke AI’ Executive Order. Lawfare
OMB Releases Guidance on Trump’s ‘Woke AI’ Executive Order. Lawfare
AI Risk Management Framework. NIST
America’s AI Action Plan
America’s AI Action Plan | David Wood
Ensuring a National Policy Framework for Artificial Intelligence (Executive Order, 11 December 2025). The White House
Recommendations for the U.S. AI Action Plan - Future of Life Institute. Future of Life Institute
AI Watch: Global regulatory tracker - United States | White & Case LLP
China
Artificial Intelligence Standardization White Paper (2021 Edition). Translated by CSET, 29 September 2021
Next Generation Artificial Intelligence Development Plan. China Science & Technology Newsletter, No. 17 (15 September 2017). Organizer: China Association for International Science and Technology Cooperation
Global AI Governance Action Plan. Fmprc.gov.cn
China’s AI Regulations and How They Get Made. Carnegie Endowment for International Peace
China AI Strategy
China’s ‘New Generation Artificial Intelligence Development Plan’ (2017) - DigiChina. DigiChina
AI Watch: Global regulatory tracker - China | White & Case LLP
China’s Approach to AI Development and Governance | The National Bureau of Asian Research (NBR). Nbr.org
