AI Tools for Existential Security

Rapid AI progress is the greatest driver of existential risk in the world today. But — if handled correctly — it could also empower humanity to face these challenges.

Full post: AI Tools for Existential Security (summary on X)

Executive summary

1. Some AI applications will be powerful tools for navigating existential risks
Three clusters of applications are especially promising:

• Epistemic applications to help us anticipate and plan for emerging challenges

  • e.g. high-quality AI assistants could prevent catastrophic decisions by helping us make sense of rapidly evolving situations

• Coordination-enabling applications to help diverse groups work together towards shared goals

  • e.g. automated negotiation could help labs and nations to find and commit to mutually desirable alternatives to racing

• Risk-targeted applications to address specific challenges

  • e.g. automating alignment research could make the difference between “It’s functionally impossible to bring alignment up to the requisite standard in time” and “this is just an issue of devoting enough compute to it”

2. We can accelerate these tools instead of waiting for them to emerge

• While broad AI progress will drive the development of many applications, we have some flexibility in the timing of specific applications — and even small speed-ups could be crucial (e.g. by switching the order of risk-generating capabilities and risk-reducing ones)

• We could use a variety of strategies to accelerate beneficial applications:

  • Data pipelines & scaffolding: by curating datasets or scaffolding for key capabilities, or laying the groundwork to automate this, we could enable those capabilities as soon as underlying AI progress supports them

  • Complementary tech & removing other barriers to adoption: by building out the UI or other complementary technology, and ensuring that people are eager to use the applications, we could enable the applications to see use as soon as the underlying capabilities are there, rather than accept delays to adoption

  • Shaping compute allocation: by building support among key decision-makers who might allocate compute, we could ensure that crucial applications are among the earliest to see large amounts of automated research

• Accelerating beneficial applications can often be done unilaterally (in contrast to delaying dangerous capabilities, which may need consensus)

Implications

These opportunities seem undervalued in existential risk work. We think a lot more people should work on this — and the broader “differential AI development” space. Our recommendations:

1. Shift towards accelerating important AI applications

   • e.g. curate datasets for automating alignment research; or build AI forecasting systems

2. Plan for a world with abundant cognition

   • Some new approaches will come online, and some current work may be obsoleted

   • e.g. it could make sense to build tools that process rich information to provide bespoke infectious disease exposure advice in contact tracing apps

3. Get ready to help with automation

   • e.g. build relevant expertise, or work towards institutional buy-in

---

See also a thread that summarizes the post, and includes some extra visuals. Preview of select images:

There’s meaningful room to accelerate some applications — and even minor speed-ups could be extremely significant

We have promising strategies that focus on almost all of the major inputs in the development of an AI application.

As AI capabilities rise, AI systems will be responsible for a growing fraction of relevant work. Shaping what AI systems (can) do is among the best ways to steer towards existential security.