Get the latest tech news
AI existential risk probabilities are too unreliable to inform policy
How speculation gets laundered through pseudo-quantification
But regardless of which method is used, when it comes to existential risk, there are many barriers to assessing forecast skill for subjective probabilities: the lack of a reference class, the low base rate, and the long time horizon. A paper by our Princeton CITP colleagues led by Shazeda Ahmed explains the epistemic culture of AI safety, which consists of “cohesive, interwoven social structures of knowledge-production and community-building”. This series of essays is based on an upcoming paper that benefited from feedback from many people, including Seth Lazar and members of the MINT lab at Australian National University, students in the Limits to Prediction course at Princeton, and Shazeda Ahmed.
Or read this on Hacker News