Thanks. Yep! Perhaps we could call that the 'sociotechnical roadmap': mobilising attention, effort, money... and it interrelates with the distribution/adoption challenge. All important pieces needed to make this go well.
Thanks for sharing! I checked out the presentations and have a few questions;
-are there more fellowships planned or is the focus on scaling solutions from the 2025 one?
-what GenAI-models are generally used? Are open-source ones up to the task yet? I feel like a major crux for some of these is whether people trust the models not to be subtly biased
We're considering more fellowships, currently uncertain. There are several 2025 cohort projects which are being pursued further, in several cases with FLF's support.
Yes, trust in models is important! I think we'll partly find out as we go, and be able to patch (either by updating models or by prompt crafting etc.). Some applications may be bottlenecked on this still. One cluster of tools/procedures that would help here are 'epistemic virtue evals'; we discuss briefly here: https://newsletter.forethought.org/p/ai-tools-for-collective-epistemics. As well as being directly important, it's potentially foundational to some kinds of *further* epistemic tech, in the obvious way.
Foundational indeed, the "rigorously tested to avoid bias, sycophancy, and manipulation" is important, but my main concern is that 50% of people we want to mobilize see it as leftwing-biased and the other 50% as right-wing biased, even if it was 100% fair and unbiased. I would seriously consider prioritizing developing a standalone LLM for all these projects. One with full transparency of all pretraining and fine-tuning data. That seems like the only way everyone can trust it not to be biased in subtle ways
Yeah, polarisation like that is just pernicious. I think it can be addressed partly in at least two ways: one at the structural level by gradually deradicalising people into better faith dialogue, and one at the local level by exposing sources more legibly and directing attention to the state of *discourse and range of perspectives*, rather than taking object level stances where contested.
I agree, but my fear is that even if the output is a perfectly balanced overview of the range of perspectives, the "brand" of the underlying model can become a heuristic for dismissal.
Another approach might be ensemble methods; using multiple models and making that transparent. If the tool cross-references outputs from several LLMs and surfaces where they agree and disagree, it becomes less about trusting any single model.
Re: "Scenario planning on tap so that people can explore the likely implications of possible courses of actions and get analysis of the implications of different hypotheticals"
Great piece. How do we make this happen? And is there a way to get involved?
Thanks. Yep! Perhaps we could call that the 'sociotechnical roadmap': mobilising attention, effort, money... and it interrelates with the distribution/adoption challenge. All important pieces needed to make this go well.
FLF and other incubators are putting some effort in here, e.g. https://aiforhumanreasoning.com/
Thanks for sharing! I checked out the presentations and have a few questions;
-are there more fellowships planned or is the focus on scaling solutions from the 2025 one?
-what GenAI-models are generally used? Are open-source ones up to the task yet? I feel like a major crux for some of these is whether people trust the models not to be subtly biased
We're considering more fellowships, currently uncertain. There are several 2025 cohort projects which are being pursued further, in several cases with FLF's support.
Yes, trust in models is important! I think we'll partly find out as we go, and be able to patch (either by updating models or by prompt crafting etc.). Some applications may be bottlenecked on this still. One cluster of tools/procedures that would help here are 'epistemic virtue evals'; we discuss briefly here: https://newsletter.forethought.org/p/ai-tools-for-collective-epistemics. As well as being directly important, it's potentially foundational to some kinds of *further* epistemic tech, in the obvious way.
Foundational indeed, the "rigorously tested to avoid bias, sycophancy, and manipulation" is important, but my main concern is that 50% of people we want to mobilize see it as leftwing-biased and the other 50% as right-wing biased, even if it was 100% fair and unbiased. I would seriously consider prioritizing developing a standalone LLM for all these projects. One with full transparency of all pretraining and fine-tuning data. That seems like the only way everyone can trust it not to be biased in subtle ways
Yeah, polarisation like that is just pernicious. I think it can be addressed partly in at least two ways: one at the structural level by gradually deradicalising people into better faith dialogue, and one at the local level by exposing sources more legibly and directing attention to the state of *discourse and range of perspectives*, rather than taking object level stances where contested.
I agree, but my fear is that even if the output is a perfectly balanced overview of the range of perspectives, the "brand" of the underlying model can become a heuristic for dismissal.
Another approach might be ensemble methods; using multiple models and making that transparent. If the tool cross-references outputs from several LLMs and surfaces where they agree and disagree, it becomes less about trusting any single model.
Great, thank you. Yes, it's interesting to think about what the necessary coalition is to mobilise attention, effort, money etc.
Re: "Scenario planning on tap so that people can explore the likely implications of possible courses of actions and get analysis of the implications of different hypotheticals"
Any concrete discussion/project wrt this yet?
Take a look at https://deepfuture.now/, for example