Futurology

His theory will be increasingly relevant in the coming years, I think. It won’t look the same, and will be more of a period/epoch than a discrete event, but the kernel was true, to the extent of statistical inevitability.

Unfortunately, the way singularity theory is handled by pop culture, and honestly most of the community, especially the those of the EA group, is terribly flawed. In particular, the fear of so-called AGI, murderous robots etc, is mostly unfounded, while the real dangers and their current relevance remain mostly unaddressed.

I’ll write up an explanation if enough people are interested, but in summary:

1. Runaway AGI is highly improbable due to (a) conditional probability and (b) thermodynamics, and it has nothing to do with the heat wall.
2. Estimating the arrival of the singularity is silly, because it has already begun.
3. The critical technological safeguard against negative outcomes isn’t redundant kill switches and blackbox barriers. It’s just machine ethics, a nascent research space that currently has little profit incentive, and thus has barely made it off the ground.