Comparing Possible Plans
Eli Lifland
Introduction
In this supplement I’ll estimate how well each plan does on various intermediate metrics, and its chance of leading to a great future. I discuss quantitative metrics for the sake of concreteness, however I’m very much not confident in my precise estimates. I am confident in the conclusion that Plan A is substantially better than Plans B-D. For plans that I think could be competitive with Plan A, see this table.
I estimate how good each plan would be conditional on the government and/or leading companies attempting to execute it. The below aren’t meant to be an exhaustive taxonomy; I am considering what would happen if roughly the described plan is attempted, rather than bucketing every possible future into the plan it is closest to. For more detail on how I classify trajectories into plans, see below.
The plans I consider are:
Plan A (as implemented specifically in the scenario): Verified slowdown + total research transparency. A verified international slowdown deal coupled with complete transparency of AI research. AI capabilities are scaled up while maintaining high confidence in safety but balancing risks from deal decline and covert projects.
Plan A: Verified slowdown + substantial transparency. To be more generally classified as Plan A, the implementation must involve a substantial verified slowdown which involves continued AI scaling, as opposed to an indefinite halt. It must involve transparency at least as strong as embedded auditors from foreign governments which publish reports to governments (and for which redacted versions are made public), as described in our filtered transparency proposal. Other aspects of the plan, such as mutually assured compute destruction, are not required.
Plan B-kinetic: Sabotage China kinetically and burn some lead. Sabotaging China in order to get more of a lead to burn on safety, with the willingness to escalate to large-scale kinetic attacks such as by drones or conventional missile strikes. To be classified as Plan B the US must have the intention to use their lead over China for safety purposes, and must in practice slow takeoff by at least 3 months.
Plan B-cyber: Sabotage China non-kinetically and burn some lead. The same as Plan B-kinetic, but without willingness to escalate to large-scale kinetic attacks. This could include cyber and supply chain attacks.
Plan C+: Domestic regulation and slow China without sabotage. The US buys substantial time via domestic regulation, without sabotaging China. They instead aim to slow down China via non-sabotage measures such as export controls and giving green cards to Chinese talent. Requires at least a 2 month slowdown.
Plan C: Leading project burns some lead. The leading AI project burns some of its lead on safety, and potentially coordinates with other frontier projects for a further slowdown without domestic regulation. Requires at least a 1 month slowdown.
Plan D: Race. Frontier AI projects race through the intelligence explosion at nearly max speed, devoting a nonzero but small percentage of resources to safety (at least 1%; lower would count as Plan E).
There are also other plans that the government could attempt to implement. Some of these are discussed below.
For simplicity, I assume that:
The minimum-length takeoff from Automated Coder (AC) to Top-Expert-Dominating-AI (TED-AI) is slightly less than a year, from the start of 2030 to the end of 2030. This matches the default Plan A AI Futures Model trajectory. The minimum-length takeoff is what happens if the leading AI projects go at maximum speed.
All plans can begin implementation in early 2029 as in the Plan A scenario, though for Plans C and D this doesn’t matter as the plan involves doing nothing until takeoff is underway. This still might involve some safety work, if doing so is useful for advancing capabilities.
I remove these conditionals and present other authors’ views below.
Plan assessment table
These estimates are all rough best guesses, and the specifics of them aren’t load-bearing to our recommendation of Plan A (though the high-level takeaways are).
| Metric ↓ / Plan → | Plan A | Plan B | Plan C+ | Plan C | Plan D | |
|---|---|---|---|---|---|---|
| Kinetic | Cyber | |||||
| Objective metrics | ||||||
| Takeoff length (AC to takeover-capable AI) | 6 yrs | 3 yrs | 2 yrs | 1.5 yrs | 1.13 yrs | 1.02 yrs |
| Training compute safety tax (OOMs) | 2.8 | 1 | 0.75 | 0.46 | 0.13 | 0.02 |
| Safety compute (H100e-yrs) | 100B | 500M | 100M | 20M | 4M | 500K |
| Safety-researcher-years | 1M | 2,000 | 1,000 | 1,500 | 500 | 200 |
| Subjective metrics (1–5) | ||||||
| Epistemic culture | 3 | 1 | 2 | 2 | 1 | |
| Public transparency | 4 | 1 | 2 | 2 | 1 | |
| Distribution of power | 5 | 1 | 2 | 2 | 1 | |
| Outcomes (medians across implementations) | ||||||
| p(alignment); i.e., 1 - p(misaligned takeover) | 72% | 50% | 45% | 40% | 25% | |
| p(great future | alignment) | 58% | 50% | 55% | 50% | 40% | |
| p(great future) | 42% | 25% | 25% | 20% | 10% | |
Plans are alphabetically ranked in terms of their objective metrics, with a big jump between Plan A and Plan B. However, plans C+ and C beat out Plan B on some subjective metrics. Plan B’s focus on security and its wartime vibe could degrade epistemics, and its likely implementation of nationalization or pseudo-nationalization would decrease public transparency and concentrate power.
Comparing authors’ views on plan likelihood and outcomes
These estimates aggregate over uncertainties about the default capability trajectory and when the plans are implemented, as well as how competently the plans are executed. The numbers are in this sheet, and the plan classification is as described below.
Likelihood
Probability each plan happens across authors
| Eli | p(govt does something at least as intense as Plan B): 35%; seems like govt wakeup has been increasing and also the importance of ASI will get more and more obvious. Though note that Plan B requires actually slowing down by at least 3 months. |
|---|---|
| Ryan | I'm very uncertain about what should count. I think Plan C+ is mostly worlds with dumb regulation that slows down AI. I think Plan B would be higher if not for the intent to spend the lead on safety. |
Outcomes
We focus on p(alignment) here, i.e. 1 - p(misaligned takeover). We ignore p(great future | alignment) in this section for simplicity.
Probability of avoiding misaligned takeover by plan
| Eli | My probabilities in the plan assessment above were conditional on: My median takeoff is ~2x longer than the one described in (a), but I have significantly probability on faster takeoffs. This means that my p(alignment) estimates should do a combination of increase and regress a bit toward 50%. Also, Plan A isn't as sensitive to default takeoff length as other plans, and from a covert project perspective it's actually unclear in which way the effect size points because larger effective compute gaps between the start and end of takeoff require more software progress. The trajectory timeline to AC / start of takeoff is pretty similar to my median so for that I regress abit toward 50%. For (b), my median is that the plan gets implemented later than this, but I think it doesn't matter much for all the plans besides A because in the other plans it's best to slow down later anyway. I do lower the value of Plan A due to this consideration, though not hugely because I think covert projects would likely still be difficult to do without detection. Also, the plan assessment numbers were based on the median implementation of the plan, rather than aggregating the expected outcome over all implementations. My unconfident quick guess is that switching to expected outcome should also regress my numbers toward 50%, but not sure so this adjustmnet will be small. With the above in mind, I make the following adjustments: For Other, I think a bunch of this is random pretty good plans that aren't covered, including Plan S. E.g. there's ~40% that this is a non-Plan-A deal. Though it also includes Plan E. Overall I rate it the same as B, 55. |
|---|---|
| Ryan | These numbers are very sensitive to how you classify plans. E.g., I think Plan A as described in the scenario has maybe 3/5 as much risk as Plan A as classifed by the supplement. An actually good version of Plan C done by the leading AI company is pretty close to the numbers I give for Plan B (but worse than a great Plan B). A large fraction of "other" is worlds with not enough effort on safety to classify for Plan D. |
Other plans
There are many other possible plans that the government could follow. Below we catalog some of the ones that we find to be most promising and/or likely.
Plans that might be competitive with Plan A
Summary | Advantages (relative to A) | Disadvantages (relative to A) |
Plan S: Indefinite halt. A halt on all frontier AI capabilities progress, intended to last at least a few years. Different variants of Plan S have different conditions for resumption of AI progress; for example it could be alignment progress, lie detectors, human uploads, or intelligence enhancement. | A longer expected slowdown and more margin for error regarding scaling too fast. Potentially simpler. | Scaling to controllable AIs within the human range is helpful for accelerating alignment/control, epistemics, verification, and general understanding of AI. |
Domestic-first Plan A. Regulate AI domestically enough to reduce AI takeover risk to acceptable levels, which will require a long slowdown. It’s possible other countries will also regulate domestically and something like Plan A won’t be needed; otherwise, transition to Plan A later. | Initial steps are achievable by the US and very helpful even if the international stage isn’t viable because it massively extends the timeline. Proactively builds trust with China. | Worse for covert projects and setting up verification than negotiating Plan A at the same time. May not be politically feasible due to race dynamics. |
GPU arms control. International agreements for countries to limit their GPU stock or flow, analogous to historical arms reduction agreements. | Much simpler to enforce, and more historical precedent. Can achieve substantial slowdown. | Less slowdown, less ability to pay safety taxes, ongoing race dynamics mean no ability to coordinate towards safer paths through the tech tree. |
CERN for AI. An international project to develop frontier AI, with all other projects regulated to be substantially behind in capabilities. | Easier to defend against algorithmic leakage and distillation to covert projects. Less actors at the frontier might make it easier to enforce. | Worse decisions (including worse decisions on technical safety) due to less transparency and less broad deployment. More concentration of power risk. |
Plans that we think are clearly worse than Plan A
Repurpose the GPUs: Like GPU arms control, except they are now required to be used for existentially useful applications rather than being destroyed. See this comment for a more specific proposal in which GPUs are required to be used for giving inference capacity to external safety organizations or to do publicly transparent safety research. In theory this plan should be better than GPU arms reduction, but it’s less reversible and harder to enforce.
Plan B-but-don’t-slowdown, aka D-war: Like Plan B except that you don’t slow down, i.e. you don’t burn ~any of the lead that you increased by sabotaging China. (Currently classified as a variant of Plan D when making the above estimates.)
Weaker deals with China. For example, the US could offer: "You get access to more chips for AI inference, and a non-binding or non-majority say in our safety requirements, if you give us full transparency into what you're up to.”
Plan E: Like Plan D but even less safety investment, i.e. less than 1% of resources.
Classifying trajectories into plans
Define the implementation window as the period between:
Start: 1 year before Automated Coder (AC) would be reached if scaling at maximum speed. An AC is a collective of AIs that fully automates an AGI project’s coding work, autonomously replacing the project’s entire coding staff. That is, you’d rather fire all of the humans than stop using AIs.
End: Takeover-capable AI. Takeover-capable AI is reached when it is practically very unlikely that humans could stop AI takeover, conditional on the AIs being adversarially misaligned. It could still be in theory possible for them to shut down the AIs by taking extremely drastic actions like shutting down the internet, but it’s unlikely that they would do so, e.g. because AIs might be heavily relied upon and/or highly persuasive.
If the plan being attempted is consistent throughout this window, then the trajectory is classified into that plan. Even if multiple plans are attempted, each trajectory can count as at most one plan; this is a non-obvious choice and it seems also reasonable to count trajectories as multiple plans.
If multiple plans are attempted during the implementation window, the trajectory is classified as the most intense plan that was attempted. This is a bit vague to use as a general classification, but for concreteness among Plans S/A/B/C+/C/D trajectories would be classified as the attempted plan that is earliest in that list. The reasoning behind this is that it seems intuitively like a trajectory should count primarily as Plan A or S even if the deal dissolves quickly, and also should count as Plan A or S if it starts as a less intense plan but then transitions to A/S within the implementation window.
The following selection effect is a consequence of this classification: In more intense plans, people are selected for being more competent. Furthermore, less intense plans are selected for people not later transitioning to a more intense plan even if concerning evidence arises.
One selection effect that I explicitly remove is the effect of background variables such as alignment difficulty on what plan a trajectory counts as. I.e., I assume when rating plans that the distribution of alignment difficulty is the same no matter what plan is attempted; in particular my overall guess as to the distribution of alignment difficulty. As mentioned above, I don’t do this for things that humans control such as their competence.
Appendix
Training compute safety tax definition
The training compute safety tax is the difference between the amount of compute actually used on a training run to reach a given capability level, and the minimum amount of compute required to reach that capability. Examples of ways to productively use that unnecessary compute might be to train in a more safe but less compute-efficient paradigm, or to spend lots of compute on AI-driven scalable oversight to improve incentives during the training process.
Measuring the safety tax by this method is an imperfect proxy for what we actually care about. There is likely large variance among improvements which provide the same amount of capability in how useful it is for safety to forego that improvement.
Other potential differences from Plan A, though it depends on which variant of Plan S is implemented: slower compute buildout, less transparency.
A special case of this is to Pause the GPUs: Require that a fixed fraction of the GPUs be either (i) turned off or (ii) used to mine cryptocurrency (or be used for some verifiable, non-AI R&D purpose). This proposal has the upside and downside of making it easier to unpause the GPUs.
But maybe you can mitigate this via small public deployments and selective transparency.
Potentially I should allow for plan implementation to begin earlier, especially if the plan is very intense. For example, probably Plan S should count if it begins 1.5 years before the default AC time.