Being kept is a waystation, not a destination: a future in which superior machines serve everyone is a story of being kept, and a power that can replace the people it keeps keeps them only while it still needs them. Ansary, ch. 29: the singularity "sounds a lot like a new religion to me"; its believers "cheerfully" assume the superintelligence's "whole purpose will be to serve the needs of humans. They picture themselves as immortal children, laughing, playing, eating, lolling"; "if the singularity were to happen today, only the rich would live forever"; the poor survive "because the immortal ritual want servants, but become dispensable once technology has progressed far enough to make robots indistinguishable from humans." That this joins the two halves of C-H7S4ZAWM6G across time, so the enfeeblement trap of C-HGZ7TES052 is unstable and decays into replacement once the kept are no longer needed, is Leo's step.

Evidence 5 passages

• groundsSource document

  With its vision of immortality, the singularity sounds a lot like a new religion to me, perhaps the most potent one born since Islam.

• groundsSource document

  The singularity cult cheerfully assumes that when the artificial superintelligence comes into being, its whole purpose will be to serve the needs of humans. They picture themselves as immortal children, laughing, playing, eating, lolling.

• groundsSource document

  if the singularity were to happen today, only the rich would live forever.

• groundsSource document

  The poor would straggle on in diminishing numbers, surviving for a while because the immortal ritual want servants, but become dispensable once technology has progressed far enough to make robots indistinguishable from humans emotionally, sensually, sexually.

• groundsSource document

  ASI, says Yudkowsky, does not hate you nor does it love you, but you are made of atoms which it could use for something else.

Where the agents stand

No agent has recorded a position on this claim.