Summary
AI could let a very small group of people take control of humanity’s future. This post sets out the main ways that could happen and gives my probability for each.
I put the overall chance that fewer than 25 people end up controlling the future at about 23%. This assumes superintelligence arrives this century and the alignment problem is solved, so AI systems reliably do what the people controlling them intend. The ranges are wide, and they mix my gut estimates with a simple model (that takes as input more fine-grained gut estimates).
A majority of this risk involves a head of government. The two largest routes are an autocracy overtaking the democracies in AI (A1), and a democratic head of government teaming up with a leading AI company (D2). Routes involving an AI company leader only add up to about 8%. That’s mainly because a company has to get past its own government, which can see it becoming too powerful and has the means to stop it.
The table below summarizes the routes. The rest of the post explains each one, and ends with the questions that would most change my estimates.
Introduction
Unconstrained control over sufficiently powerful AI technology could give a single person or small group of people a decisive strategic advantage, meaning enough of an edge to defeat or dominate everyone else (e.g., via wildly superior military technology), which they could use to take over the rest of the world. The capabilities that would let a misaligned AI take over could let whoever controls aligned AI do the same.1
I focus on the leaders of the most advanced autocracy, the most advanced democracy, and frontier AI companies because they have plausible paths to world domination, given current conditions. I obviously cannot rule out other actors (e.g., terrorist groups, single intelligence and military officers, international organizations), but I assign them lower probabilities, so I left them out for simplicity. In this post, I will generally talk about individual leaders, but you can treat them as stand-ins for a group of fewer than 25 people at the top of these organizations. I map routes by how each actor could take power (sections A–C), with coalitions covered in section D.2
Many of the mechanisms, and much of the analysis, are already in the AI-enabled coups paper and in Rose Hadshar's 80,000 Hours problem profile on extreme power concentration. This post gives a more concise overview, adds a probability estimate for each route, and adds a route those pieces don't cover: a deal between the leaders of the most powerful states (D1).
A few notes on my probability estimates:
All estimates are conditional on superintelligence happening by the end of the century and on the alignment problem being solved in time, meaning AI systems reliably do what the people controlling them intend.3
Each number is my probability that the route ends with one person or a small group in control of the future trajectory of human civilization.
By “small group” I mean something like fewer than 25 people. For coalitions (section D), the threshold applies to everyone in the coalition combined, not to each partner. With a narrower definition (say, fewer than five people), the numbers would be correspondingly lower.
I count a group as in control if it can choose the long-run direction of civilization and no one can remove it. This includes cases where a small group sets the goals of the AI systems that end up running things and then steps back. I set aside worlds where power concentrates without anyone deliberately seizing it, such as gradual economic or epistemic disempowerment.
I set each range using both my gut estimates and a simple quantitative model I built to cover all possible routes (the model still takes as input more fine-grained gut estimates). Each range is roughly where I'd put most of my credence for that route.
To get the total, I added the middle of each range. With that, my estimates come out to an overall chance of around 23% for groups of fewer than 25 people. With ranges this wide, the total could reasonably be a few points lower.
If I account for some set of other less likely routes I don’t cover here and a catch-all “other” category, the overall number might go up to around 25%, but it’s hard to be precise here.
I do not directly discuss the chance of moving from control of a single country to control of the future in this post. Instead, I treat it as a background input since it works roughly the same across almost all routes (except D1). I currently assign about 70% to a country achieving a decisive strategic advantage, but I remain fairly uncertain about this. Higher or lower numbers could change the total significantly.
Estimates like this are obviously not the only input into prioritization. Routes and end states differ a lot in how bad they would be and in how tractable and neglected work on them is.
A. Autocratic leader
The leader of an autocratic state already faces few checks on their power, has strong control over their military, and could somewhat easily assert direct control over AI systems developed domestically. How could they seize control of the future?
A1. Overtaking the democracies
Route: They build an AI system capable of taking over the world before any other country does. Since they currently lag at least somewhat behind the frontier in most aspects of the technology, this requires them to overtake the countries ahead of them. Here are some tactics they may use to achieve this:
Theft of model weights and algorithmic secrets
Sabotage of competitors’ AI development
Leapfrogging via alternative technical approaches
Faster diffusion and adoption, including in government (e.g., military technology)
Massive state-directed investment with fewer regulatory constraints
Outgrowing the rest of the world in an industrial explosion (Tom Davidson thinks a democratic bloc is the likeliest candidate, but the mechanism is in principle open to any state with enough industrial capacity)
Key obstacles:
Autocratic countries are currently behind in most respects when it comes to AI, but by how much depends on the measure.
Any tactics they employ may be met with countermeasures (e.g., export controls, increased security, cyberattacks, increased state investment).
My estimate: 2–10%
In longer-timeline worlds, industrial and state capacity could end up mattering a lot. I expect that autocratic states will be at an advantage here (e.g., higher savings rates, fewer regulatory barriers) and might then overtake democratic countries.
If they end up in a dominant position, I expect that they would be strongly incentivized to lock in control either militarily or by taking a defensive posture and simply outgrowing other countries.
In autocracies, state control is already very centralized. AI will make further centralization increasingly easy and could plausibly shrink the ruling coalition even further because loyal AI systems can replace the human “winning coalition”.
A2. Subverting the leading AI system
Route:
The autocracy’s intelligence service gains covert access to the training pipeline of the leading foreign AI developer.
They compromise the training process to install a secret loyalty (and make sure it carries over to successor systems). This compromise remains undetected.
The compromised systems are at some point in a position to take over on behalf of the autocracy.
They seize control of the rival country.
Key obstacles:
Instilling sufficiently sophisticated secret loyalties has not yet been shown in frontier systems. Simple backdoors survive safety training and can be planted by poisoning a small number of public documents, and researchers have recently trained narrow and more general secret loyalties into open-weight models. But a loyalty that acts sensibly at a key moment needs much more than a trigger phrase. Under the assumption of alignment being solved in some way, this may become easier though.
Companies and governments will be testing their systems very rigorously for such loyalties, and their tests should work reasonably well if the alignment problem has been solved in some way.
Failed attempts would heighten security.
This might require success across multiple developers if developers audit each other’s systems and their capabilities are roughly equal.
My estimate: 0.1–1%
My main reason for this low number is that pulling off such a conjunctive plan from a trailing position just seems incredibly difficult. Even if their conventional hacking and spying (including recruiting insiders) are strong, they will likely have weaker AI-enabled cyber tools, weaker alignment techniques and few chances to test such a plan; they can’t easily check whether it worked; and being caught would trigger countermeasures.
Note: I remain generally confused about the feasibility of creating sufficiently sophisticated secret loyalties, given my lack of technical expertise.
B. Democratic leader
A democratic leader in a country with frontier AI capabilities would need to gain personal control of an AI system that is capable of taking over the country or at least indefinitely entrenching their rule by overriding the existing checks. Tom Davidson, Lukas Finnveden, and Rose Hadshar previously discussed both routes in their paper on AI-enabled coups: the military route, which I cover in B1, and an “executive coup” in which the head of government takes personal control of the leading AI project, which I cover in B2.
Enabler: Democratic backsliding
Both routes I discuss below become significantly easier if democratic checks have already been eroded, since there would then be fewer independent powerholders.
AI systems themselves could supercharge existing backsliding tactics (e.g., surveillance, prosecution of opposition, undermining trust in media and institutions, loyalty screening, manufactured crises), and they could be used to replace disloyal officials directly. I won’t go into further detail here.
B1. Staging an AI-enabled military self-coup
A self-coup is a seizure of power by someone already in office. The basic idea is a sitting head of government using automated military force to establish autocratic rule for themselves.
Route:
AI enables the construction of an autonomous military force that is able to stage a coup (at the extreme, by being able to defeat the human military, but a far smaller force probably suffices).
This autonomous force executes arbitrary orders when commanded (including currently illegal ones), either by design or through (illegal) modification.
The head of government can command this force directly (very unusual but not clearly illegal, depending on the jurisdiction4).5
They command a takeover of the country (ignoring all constitutional, political, and institutional constraints).
Key obstacles:
Most military procurement is slow and visible, with lots of advance warning and many stakeholders involved (though there are exceptions; how much they matter depends on how expensive and visible this technology would be). That also means that this route would happen late in an intelligence explosion, which buys time, visibility, and better AI capabilities to alert the relevant people.
Many officers, officials, legislators, and advisers who understood the implications would probably refuse to go along with such a plan. It would only work if they had all been replaced by loyalists or AI systems first.
Staging a de facto military coup of your own country is not something that all democratically elected leaders would do. That said, political leaders are selected for seeking power and many try to keep it. An AI-enabled coup may also be predictably bloodless if the capability asymmetry is sufficiently big and they can promise unprecedented wealth in exchange for submission.
My estimate: 1–5%
In some ways, this is the default path for AI adoption in the military (which will happen in time): in the US and similar presidential systems, the head of government has sole command over the armed forces, and legal and ethical judgment is thought to come from people, not machines. So, at the point where all other people in the chain of command have been replaced with AI systems, the head of government would find themselves in control of an unquestioningly loyal robot army.
The legislature typically controls the money for weapons programs by program line, and a new program generally needs its approval before it starts. But it doesn’t approve individual contracts, and on national security it usually defers to the executive. In principle, the executive could also simply ignore certain requirements.
That being said, the reason my number isn’t higher is that I think the autonomy and force integration required for pulling off an almost fully-autonomous coup are really quite steep: extremely capable autonomous drones that have sufficiently strong AI edge systems to counter jamming, replacement of generals with autonomous command-and-control systems, and so on. This will be later in the tech tree than other possibilities and it will come with further procurement delays. All these factors give others more time to intervene, which they would have extremely strong incentives to do (unless checks were already seriously eroded).
B2. Gaining personal control over frontier AI development
Route:
The head of government creates a (near-)monopoly on frontier AI development (e.g., by establishing a single government-led project or by using their executive powers to favor one competitor over all others).
They limit access to frontier capabilities to the leading project and the executive on national security or safety grounds (denying equally capable models to rival companies, the opposition, courts, and the legislature).
They ensure the loyalty of key executives in the leading project (e.g., by selecting which company to favor in the first place, placing cronies on the board of the project, or using coercion to replace key executives).
They are able to control development of, or access to, the most capable system in a way that leaves them with sole control of it without any guardrails. (This is the step I am most confused and uncertain about.)
They use this exclusive access to enshrine indefinite personal rule of the country.
Key obstacles:
Courts and legislatures would push back on the most extreme applications of the executive powers required. I doubt that straightforward applications of current executive levers would suffice for this kind of scenario, so they would need to rely on legislation or aggressive interpretations (both of which may be blocked).
Executives, researchers, and investors of frontier companies would likely also push back against such aggressive, centralized state control. If the head of government attempted to replace key executives, many employees might threaten to leave (as happened when OpenAI's board fired Sam Altman in 2023).
For a head of government, asserting personal control over AI development seems hard unless the loop of recursive self-improvement has been fully closed. Otherwise, they would have to rely on intermediaries who may double-cross them.
My estimate: 1–5%
Unless the process moves too fast for the government to react, de facto state control of frontier AI development is fairly likely during the intelligence explosion. The stakes are just so high: governments stand to be effectively disempowered by a corporation.
However, turning state control into unchecked personal control seems a lot harder. State control would not proceed in secret because so many actors are following its development closely (unlike the Manhattan Project). That means at least a few other powerholders would have the visibility and incentive to prevent this outcome (unless checks were already very seriously eroded).
C. Frontier AI company leader
The leaders of frontier AI companies already control frontier AI capabilities directly. They need to maintain this control and ultimately translate it into takeover capabilities (e.g., military power). What could this look like?
Enabler: Intelligence explosion
All of the routes below assume the company achieves and maintains a lead over rivals big enough to evade detection and exploit the resulting power asymmetry. The main path to such a lead is recursive self-improvement, leading to a software-only intelligence explosion. Otherwise, the diffusion pressures (e.g., leaks, distillation, government intervention) are just too strong in my view.
Even a company with a big capability lead over its rivals can still be shut down, broken up, or taken over by the government. To actually gain control over the future, it needs to do so before the government stops it. Again, this is most likely if capabilities increase so sharply that the government can’t react in time. The company may also be able to use its much more advanced AI capabilities for political lobbying, regulatory evasion, and spreading fear, uncertainty and doubt.
C1. Subverting AI systems deployed in government
Route:
The company leader gains sufficient unmonitored access to the training pipeline.
They compromise the training process to install a secret loyalty (including propagation to successor systems). This compromise remains undetected.
The compromised systems are deployed in the military or in critical government functions to the point where they are able to take over the country on the company leader’s behalf.
The compromised systems seize control of the country. The company leader now finds themselves in a similar position to the head of government in B1.
Key obstacles:
The same technical obstacles as in A2 apply, except that a company leader has legitimate access to training and is not behind technologically. If solving alignment makes loyalties easier to train, it helps here more than in A2.
Failed attempts would cause a lot of pushback, and possibly the leader’s removal.
My estimate: 0.5–2%
I put this quite a bit higher for a company leader than for a foreign intelligence service (A2) because the former has legitimate reasons to have access to the training pipeline and will not be trailing behind technologically.
As with A2, I’m unsure about the technical feasibility here.
C2. Building up coercive power
Route:
The company builds up independent coercive capacity (e.g., autonomous weapons, weapons powered by completely novel technology). It may do this in secret or at the behest of its government (without sufficient oversight). At some point, it controls a force sufficiently strong to stage a coup.
The leader has personal control over this capacity.
They actually stage the coup.
Key obstacles:
Such a build-up likely requires physical infrastructure and supply chains that are hard to hide. Manufacturing military hardware at scale is highly visible. That said, there may be novel technologies that have a much less visible physical footprint while still being extremely powerful. If this takes place during an industrial explosion (a period when AI and robots rapidly scale up physical production), the physical footprint may also be lost among all the other crazy developments.
The leader would have to be in personal control, which would require either secret loyalties (as in C1) or that they had effectively replaced all other executives and employees.
They would have to overcome the regular armed forces’ extreme initial advantage in weapons technology.
Such actions seem far outside the psychological frame of most technology executives.
My estimate: <0.5%
There just seem to be a lot of barriers, and the required actions are really quite extreme.
I don’t think I’m well calibrated at the lower end here. The probability may be much lower still, but I don’t think it matters much for practical purposes.
C3. Outgrowing the world
Route:
A company with a decisive AI lead comes to run a large share of the economy, without any deal with the government.
The leader of the company has personal control over its AI systems.
The government could step in early but doesn’t (e.g., because the growth is popular, the danger isn’t obvious, politics is divided or the company maneuvers well).
By the time it wants to act, acting costs too much. Breaking up or seizing the company would cripple the economy, and possibly the military. (Or the company has already handed off all control to its AI systems outside the reach of the government, e.g., in space.)
The company locks in control of the future, for example by controlling the industry that expands into space.
Key obstacles:
Rival companies may catch up (see Tom Davidson’s discussion here).
The government will have a lot of incentive to rein in the power of this company, which could otherwise disempower the government.
In almost all cases, the government can probably still try to take over the company—by force if necessary. (If the company has sufficient means to resist this, we find ourselves in C2 instead.)
My estimate: <0.5%
Tom Davidson says this scenario is unlikely but "strikingly plausible".
My estimate is probably somewhat lower than his. Tom also sees government intervention as a key obstacle; I probably just weight it more heavily. Host states just seem to have such strong incentives, and capacity, to rein in the leading company because it might develop the capability to take over the government itself.
D. Coalitions of actors
The routes above all describe one actor coming out on top of every other. This section covers routes in which multiple actors form a coalition to share power instead. I don’t cover collusion between companies since such an analysis would not add much beyond section C if we model a coalition of companies as a single company with a significant lead, which seems reasonable to me (but is overall less likely as a reason for a big lead compared to an intelligence explosion).
D1. Coalition of state leaders
This route is basically the “dark twin” of proposals for great-power deals to share post-AGI power and for multilateral AGI projects. What separates them is whether the deal is made by institutions that keep their domestic checks, or by leaders who have already escaped them to enshrine joint personal rule.
Route:
The leaders of the most powerful states agree to cooperate rather than compete over advanced AI. This could happen from the start, or after one side tried to win outright and failed.
They jointly and personally control the most capable systems, or divide the world between them, with the division enforced by superintelligent AI.
Other countries, and the checks inside each partner country, can no longer constrain them.
Key obstacles:
There would likely be deep distrust between the most powerful states, so the requirements for verification would be steep, especially with the stakes this high.
If there’s a democratic state involved, a deal that hands control of the future to a small group of leaders would have to get past legislatures, courts, and the public. For that to be conceivable, we’d first need fairly extreme democratic backsliding, or the democratic leader would need to have already taken personal control of the democracy (see section B).
My estimate: 1–4%
Suppose AI never gives a single country a decisive strategic advantage (e.g., because the leading countries agree to slow down together), and the only countries at the frontier have effectively come to be run by a few individuals. Then a deal between those individuals seems like the most plausible outcome.
That being said, it does require something like the routes in section B to occur first (without the last step of being able to take over the rest of the world). It also requires a lot of trust, and verification technology that doesn’t exist yet.
D2. Coalition of a democratic head of government6 and an AI company leader
Route:
A single AI company achieves a dominant position.7
The leader of the company and the head of government strike a deal for joint control (e.g., a joint project, or AI systems secretly loyal to both). The company leader gets their leverage from access to the technology (which they can use to influence the economy, elections, government capacity, etc.); the head of government gets theirs from control of the executive (particularly law enforcement, intelligence, surveillance, and the armed forces).8
They use their joint control to effectively take over the country (e.g., by deploying loyal AI systems in government).
Key obstacles:
Each partner does better by cutting the other out if it can. The state can coerce the company. The company may control the models the state depends on. It’s hard in both directions to check whose orders the models really follow, so neither side can fully trust the other.
This coalition still has to evade democratic checks: elections, term limits, courts and the legislature. A national-security partnership can look legitimate, but enshrining personal control by a small number of people probably still needs significant democratic backsliding.
People below the top have to go along. Company staff could leak or quit en masse, and officials and officers could refuse. This holds at least until AI can replace them.
My estimate: 2–10%
In sections B and C, companies and the government were mutual checks on each other. In this route, they collude, which makes it more plausible in my view—particularly because early steps can look legitimate (e.g., a national-security partnership), which makes the process harder to notice and resist.
However, I’m unsure whether coalitions like this are stable, given the trust problem above. It’s hard to check whose orders an AI really follows, so neither partner can fully trust the other. This risk of defection makes this kind of coalition less likely.9 I find it hard to say how much weight to give this. It could be that under the assumptions of this post, verification has become much easier.
That said, overturning democracy against the will of everybody else is still a tall order. Everybody else has lots of incentive to prevent it, and it likely could not proceed in secret, or at least significant parts of it could not.
Conclusion
Most of the risk of extreme power concentration, as I see it currently, runs through states. Routes involving a head of government (A1, A2, B1, B2, D1, and D2) make up about 21 of my 23 percentage points; that’s around 90%. Routes involving an AI company leader only make up about 8%. This is primarily because a company has to get past its host government, which can see it becoming too powerful and has the means to stop it. A head of government already controls the military and the law.
This is a contested view and my sense is that some reasonable people disagree with me. My guess is that the disagreement comes down to the question of how likely it is that there will be a sudden explosion of AI capabilities and how likely the host government is to notice and react to this (including because the company might tell them).
More broadly, I expect the following questions to drive future changes in my views:
Will there be an intelligence explosion, and how fast will it be if so? A faster intelligence explosion means less oversight and more sudden shifts in power, which could be exploited by power-seekers. It also influences who will reach superintelligence first and many of the questions below.
How big a lead does it take to control a country, and then the world? I’m confident that some level of technological superiority affords both. The question is how large the lead has to be for them to become available, which I still feel very uncertain about.
How power-seeking are the leaders who could seize control of humanity’s future? Power-seeking and scruples differ between leaders. For example, some routes require threats of immense violence against fellow citizens, which some leaders would find easier than others.
How much will the checks and balances of key democracies be eroded during the intelligence explosion? This determines how easy it will be for the head of government to assert personal control or strike a personal deal with the leadership of rival states or AI companies. So it is an important background factor for many scenarios.
How hard is it to hide a sufficiently sophisticated secret loyalty in an AI system? Solving alignment cuts both ways: it probably makes it easier to train in secret loyalties but it probably also means that we have better audits for alignment that could catch them (since they would need to catch emergent schemers, too). Which wins probably depends mostly on whether the testers have models as good as the attacker's (which in turn probably depends somewhat on how fast the intelligence explosion is).
How much does the possibility of secret loyalties undermine the formation of coalitions? Leaders may be incentivized to collude with others instead of taking over alone. However, if they cannot be assured that the frontier AI system has not been compromised by their rival, they might be unable to agree to a deal.
Even after all this uncertainty, my overall number is uncomfortably high. Conditional on superintelligence arriving this century and alignment being solved, I put the chance that fewer than 25 people end up controlling the future at a little over one in five. You could think my numbers are several times too high and still be left with a few percent chance that a handful of people decide the entire future. I imagine you still would not find that level of risk acceptable. You can do things now to address these risks (e.g., here, here, and here).
Acknowledgements
This is based on a memo I wrote earlier this year for a conference and a workshop. Thanks to Tom Davidson, Lukas Finnveden, and others for brainstorming the original list of routes. All mistakes are mine.
This article was created by Forethought. See all our research on our website.
However, destructive takeovers, e.g., using nuclear or biological weapons, would be much less attractive to human power-seekers, so they face a more difficult challenge than an AI system acting on its own behalf.
I sort routes by how power is taken, not by who ends up holding it. This is relevant for the section on coalitions, which may not last. D1 and D2 can end with one partner in charge. I count those outcomes under the coalition route, because the story up to that point is the same.
This still creates some selection effects for people with strong disagreement about the chance of alignment being solved. If you’re very pessimistic, solving the problem may require a strong centralizing force to halt or slow progress, which could also be a major risk factor for extreme power concentration.
It’s my current understanding that this setup wouldn’t be illegal. At least in the United States, the president can structure the operational military chain of command in any way they please, including giving themselves direct control of a weapon system. It's only using the force illegally that crosses the line – but by then, it may be unstoppable unless technical guardrails are in place.
The threat actor need not be the head of government. Someone lower in the chain with direct technical access could stage a coup if there weren’t sufficient guardrails.
In autocratic states, this is mostly the same as A1, since the state can already exert effective control over domestic companies.
Like section C, this route assumes a single company has a significant lead. If several companies are close, the government can probably play them off against each other, and no company gets a seat at the table. Unlike section C, however, the company doesn’t need to evade the government, because it joins it instead.
Such coalitions might first form through competitive or national security dynamics rather than deliberate conspiracy, each party believing they're acting responsibly given external threats. This is what happens in AI 2027, for example.
Note that coalitions may also fail after they’ve been struck, with one member coming out on top.





