AI Tools for Existential Security
Rapid AI progress is the greatest driver of existential risk in the world today. But — if handled correctly — it could also empower humanity to face these challenges.
Full post: AI Tools for Existential Security (summary on X)
Executive summary
1. Some AI applications will be powerful tools for navigating existential risks
Three clusters of applications are especially promising:
Epistemic applications to help us anticipate and plan for emerging challenges
e.g. high-quality AI assistants could prevent catastrophic decisions by helping us make sense of rapidly evolving situations
Coordination-enabling applications to help diverse groups work together towards shared goals
e.g. automated negotiation could help labs and nations to find and commit to mutually desirable alternatives to racing
Risk-targeted applications to address specific challenges
e.g. automating alignment research could make the difference between “It’s functionally impossible to bring alignment up to the requisite standard in time” and “this is just an issue of devoting enough compute to it”
2. We can accelerate these tools instead of waiting for them to emerge
While broad AI progress will drive the development of many applications, we have some flexibility in the timing of specific applications — and even small speed-ups could be crucial (e.g. by switching the order of risk-generating capabilities and risk-reducing ones)
We could use a variety of strategies to accelerate beneficial applications:
Data pipelines & scaffolding: by curating datasets or scaffolding for key capabilities, or laying the groundwork to automate this, we could enable those capabilities as soon as underlying AI progress supports them
Complementary tech & removing other barriers to adoption: by building out the UI or other complementary technology, and ensuring that people are eager to use the applications, we could enable the applications to see use as soon as the underlying capabilities are there, rather than accept delays to adoption
Shaping compute allocation: by building support among key decision-makers who might allocate compute, we could ensure that crucial applications are among the earliest to see large amounts of automated research
Accelerating beneficial applications can often be done unilaterally (in contrast to delaying dangerous capabilities, which may need consensus)
Implications
These opportunities seem undervalued in existential risk work. We think a lot more people should work on this — and the broader “differential AI development” space. Our recommendations:
Shift towards accelerating important AI applications
e.g. curate datasets for automating alignment research; or build AI forecasting systems
Plan for a world with abundant cognition
Some new approaches will come online, and some current work may be obsoleted
e.g. it could make sense to build tools that process rich information to provide bespoke infectious disease exposure advice in contact tracing apps
Get ready to help with automation
e.g. build relevant expertise, or work towards institutional buy-in
See also a thread that summarizes the post, and includes some extra visuals. Preview of select images:
