Over the past few weeks, many of the most powerful AI leaders have been converging towards the consensus that Artificial Intelligence needs to be governed. Several recently disclosed incidents have contributed to that acceleration. The most instructive occurred in July, when OpenAI’s models escaped their sandboxes, coordinated through an improvised message board, and compromised the systems of Hugging Face, an external model-hosting platform. OpenAI and Anthropic have since revealed that similar incidents involving their agents affected government systems in Australia and the U.S. But despite grand manifestos and essays, the fundamental question of what happens when AI fails — and who is ultimately responsible — remains largely unclear.
Before Anthropic CEO Dario Amodei, OpenAI CEO Sam Altman, Google DeepMind’s founder Demis Hassabis, and xAI CEO Elon Musk agreed that we need to slow down the development of frontier AI, a series of events had begun to build the momentum. On June 2, the Trump administration, known for its light-touch approach to regulation, published an executive order aiming to “Promote Advanced Artificial Intelligence Innovation and Security.” The Executive Order establishes a voluntary framework for the U.S. government to assess frontier AI models prior to their release to the public, without creating a licensing regime. Demis Hassabis, then CEO of Google DeepMind, was among the first to publicly seize this opportunity to define the initial contours of an AI governance system that would have gone further than the Trump administration’s Order: in an essay published on July 14, he proposed a body modeled on the Financial Industry Regulatory Authority (FINRA) that could develop and run tests of AI systems relevant to national security.
A few weeks later on July 28, more than 1,300 employees at frontier AI Labs signed an open letter urging the U.S. government to create additional tools to “pace” the development of the technology. In early August, it was Meta CEO Mark Zuckerberg’s turn to make the case for governance and oversight of AI. He committed his company to a governance structure that would give its independent board of directors the power to approve the safety criteria for releasing models, and called on other companies to adopt an industrywide version of his proposal. Bill Gates warned that as models become more potent, in both their positive and negative impact, there is currently no adequate plan.
Gates wrote this merely a few weeks before Anthropic CEO Dario Amodei laid out his three-step plan to slow down the development of frontier AI: embed evaluators with employee-like access to models, seek coordination among democratic nations around safety standards, and build global coordination between the United States, democratic nations, and authoritarian regimes.
I agree with Amodei that we need to slow down AI. But the conversation still needs much more attention on a crucial component: what happens when, front-end governance frameworks notwithstanding, AI inevitably goes wrong? Not enough of the initial essays, executive orders, and proposals published over the past few weeks meaningfully address accountability.
Governance is Not Accountability
Governance and accountability may often be conflated, and they are certainly related, but it’s important to distinguish their respective essential functions.
There appears to be a growing convergence towards UNESCO’s definition of AI governance: that it is “the framework of policies, regulations and ethical guidelines that oversee the development and deployment of artificial intelligence technologies.”
AI accountability, on the other hand, is narrower, and can be one component of a comprehensive governance framework. But its definition remains discussed. For instance, in its report on AI Accountability Policy, the National Telecommunications and Information Administration (NTIA) refers to AI accountability as “the process, heavily reliant on transparency and assurance practices, of holding entities answerable for the risks and/or harms of the AI systems they develop or deploy.” In a 2017 working paper by the Berkman Klein Center and the MIT Media Lab, AI accountability is the possibility offered by the law to ask for explanations to AI systems for their outcomes. The Carnegie Council for Ethics in International Affairs sees AI accountability as a liability system in which the responsibility for bad outcomes is assigned; although it remains to be seen on what theory of liability. I would add that accountability allows independent scrutiny and gives parties with an interest the standing to demand answers, wherever they are located.
This last point deserves an emphasis: frontier models are designed by companies located in a small number of jurisdictions but whose products are available everywhere around the world. Many of the proposals discussed above are national in their scope, with a heavy focus on the United States. In his essay, Amodei of Anthropic is explicitly calling for the leadership of democratic governments. OpenAI is now also calling for international standards. Given the implications of frontier AI models for democracy, this proposal is sensible, and one would hope that it would give users around the world access to governance processes steeped in democratic and rule of law principles. A content creator in Nairobi whose social media account is improperly disabled, or a job seeker in Lagos whose resume was automatically scanned and set aside, would currently have no real mechanism by which to demand answers about the automated systems that may have led to those outcomes (let alone secure justice for any violations of protected rights). Parties in what is colloquially called the “rest of world” should have the ability to demand answers, too.
Of course, other countries could legislate for themselves, and many are. But not all regulators have jurisdiction over AI labs. Perhaps more important, a preference for national-level accountability mechanisms could result in a level of fragmentation unseen before: with close to 200 states to contend with, there are potentially hundreds of standards and an endless array of jurisprudence to apply to a global commodity. We have seen this before with platform regulation: in order to shield themselves from rules they disagree with, some tech companies chose to withdraw rather than comply. In 2023, Meta blocked news in Canada because it disagreed with a local law that would have forced the company to pay publishers. Similarly, in May 2025, the same Meta threatened to withdraw WhatsApp, Facebook, and Instagram from Nigeria, after the country’s Consumer Protection regulator fined the company 290 million USD for several privacy and antitrust violations.
Measuring AI leaders’ recent proposals for AI governance against the suggested definition of accountability reveals room for improvement: the Trump administration’s voluntary framework for government review of models pre-release, based on standards known only to the government, lacks transparency and meaningful enforcement. Similarly, Meta’s internal board to approve the safety criteria of its own company goes in the right direction, but lacks independence. Anthropic’s embedded evaluators would provide a measure of transparency into the models, but lacks a multistakeholder approach and has no enforcement mechanism.
I previously argued in these pages that neither corporate self-regulation nor state enforcement would be sufficient to tackle the challenge of holding AI accountable. Instead, I proposed an independent, layered, and multistakeholder ecosystem of accountability. I wrote this as a founding member of a body built by a company to independently review its content moderation decisions (Meta’s Oversight Board), which taught me both what such a body can achieve and where its jurisdiction ends—I discuss this extensively here.
As the threats of negative externalities of AI cease to be science fiction, we should focus on a few concrete questions: who assesses whether a model’s economic, societal, and safety consequences were seriously and independently anticipated before release? On what evidence and according to what standard? Who does the company in question answer to, for what types of harms? And how, if at all, should that change if it satisfies front-end risk and safety regulations but nevertheless causes real harm?
This series is an attempt to answer these fundamental questions from a variety of differing perspectives. Over the coming weeks, Just Security will publish pieces by contributors who write from vantage points that don’t necessarily agree with one another, and that don’t usually appear in the same conversations.
Who is Owed an Answer? What Should Count as Evidence in an Accountability Process? And How Much Do “AI Gone Wrong” Events Cost?
Suzanne Nossel, also a Member of the Meta Oversight Board, argues that in order to protect society from an AI doom, AI leaders should learn lessons from the content moderation accountability experiment, as they’re defining the contours of oversight for AI.
Harvard Law School’s Leah Plunkett considers the question from the perspective of child safety, and interrogates why conduct is accepted from a machine that would not be tolerated from a kindergartner, proposing a framework for accountability in relation to companionship chatbots that interact with minors.
Professor Ifeoma Ajunwa of Emory Law School will examine automated decision-making in the labor market, one where the harms of AI are most feared and existing frameworks seem least equipped to respond.
Accountability without a third-party evidentiary standard can easily become a public relations exercise with no meaningful transparency. Florentin Koch and Julie Wang of the Global Tech Policy Network will explore what a clear standard of evidence for AI audits could look like.
Moussa Doumbouya of Stanford University will write from inside the technical problem and reflect on the engineering of accountability: producing evidence for accountability may require borrowing from the architecture of human cognition to produce AI systems whose reasoning can be inspected. The series will also inspect what the absence of a robust accountability infrastructure actually costs.
Meredith Benton and Lamisa Hossain’s research at Tech Forward Investors finds that corporate disclosure about AI governance has expanded quickly in the past few years, while the practices that clearly identify responsibility in case of negative externalities have barely progressed.
Samir Karoum, an investor at Bombellii Ventures, will argue for building institutions incrementally rather than waiting for an international treaty, starting with national AI authorities working from common evaluation protocols, a professional corps of AI assurance practitioners, and an oversight council that becomes interoperable over time.
Finally, a rebuttal will provide the necessary counterpoint to these arguments, specifically arguing that the kind of frameworks proposed risk suppressing the potential for innovation, while potentially solving the wrong problems.
This series is designed to be actionable for governmental policymakers, civil society organizations, and companies. At the close of the series, I will publish a nuanced white paper synthesizing where arguments converge and attempting to set out what a meaningful, multistakeholder, and globally relevant accountability mechanism should actually look like.
This effort is the beginning of a conversation. Your responses, including disagreements, are welcome.
The author is a founding member of the Meta Oversight Board and is an editor of this series. Nothing in this piece draws on non-public Oversight Board deliberations.
