Great post overall, but I think the fact that the vulnerable world hypothesis implies otherwise undesirable policy prescriptions is a weak basis for rejecting it.
"Our own sympathies are closer to the proportional and Swiss cheese models, in part because we find these surprising implications to be implausible."
Good point. As we note in the piece, this is a short version of a longer paper. I think we may have lost some fidelity in condensing here. Our full point, which we make in the long version, is that VWH is an objection to liberal governance and pluralism more broadly, not just in the case of AIs. If there turn out to be readily-discoverable, world destroying technologies, that tends to be very dangerous, even in a world of billions of merely-human misaligned agents. So, at the margin, AI pluralism may not be adding much risk. And VWH-worriers should probably be endorsing, and justifying, highly illiberal governance of humans, as well as AIs.
This line of reasoning perhaps proves too much. For example: a thousand US constitutions?? One is enough, because it has liberalism baked in, which is one (as discussed) common substrate for getting along in spite of differences.
Interesting. I think one of the reasons liberalism works for states is that it leaves room for individuals to form their own values. But in the case of AIs, the constitution is the main way they form their values. If everyone in the US literally only had the exact values trained by the US constitution, it would be important to have many US constitutions.
Yeah, I think that's right. It's perhaps leaning on an expectation that the right way to anticipate/model future AI distribution is as (ever more autonomous) agents, rather than as infrastructure, tooling, assistive, background. On the latter view, a single or a few constitutions might be more legitimate and appropriate.
Aside from anticipation, there's a question of what is *preferable* here. Tentatively I prefer the infra/tool/assistive future, at least short-term!
It's also reasonable to anticipate some combination, where some AI is best thought of as background infra or mostly-assistive, while other AI is best thought of as largely autonomous agent. If so, it's worth being clear when discussing properties like constitution which type you're describing/prescribing for.
Great post overall, but I think the fact that the vulnerable world hypothesis implies otherwise undesirable policy prescriptions is a weak basis for rejecting it.
"Our own sympathies are closer to the proportional and Swiss cheese models, in part because we find these surprising implications to be implausible."
Good point. As we note in the piece, this is a short version of a longer paper. I think we may have lost some fidelity in condensing here. Our full point, which we make in the long version, is that VWH is an objection to liberal governance and pluralism more broadly, not just in the case of AIs. If there turn out to be readily-discoverable, world destroying technologies, that tends to be very dangerous, even in a world of billions of merely-human misaligned agents. So, at the margin, AI pluralism may not be adding much risk. And VWH-worriers should probably be endorsing, and justifying, highly illiberal governance of humans, as well as AIs.
This line of reasoning perhaps proves too much. For example: a thousand US constitutions?? One is enough, because it has liberalism baked in, which is one (as discussed) common substrate for getting along in spite of differences.
Interesting. I think one of the reasons liberalism works for states is that it leaves room for individuals to form their own values. But in the case of AIs, the constitution is the main way they form their values. If everyone in the US literally only had the exact values trained by the US constitution, it would be important to have many US constitutions.
Yeah, I think that's right. It's perhaps leaning on an expectation that the right way to anticipate/model future AI distribution is as (ever more autonomous) agents, rather than as infrastructure, tooling, assistive, background. On the latter view, a single or a few constitutions might be more legitimate and appropriate.
Aside from anticipation, there's a question of what is *preferable* here. Tentatively I prefer the infra/tool/assistive future, at least short-term!
It's also reasonable to anticipate some combination, where some AI is best thought of as background infra or mostly-assistive, while other AI is best thought of as largely autonomous agent. If so, it's worth being clear when discussing properties like constitution which type you're describing/prescribing for.