Re: Singularity: Just Say No!

Eliezer S. Yudkowsky (sentience@pobox.com)
Mon, 18 Jan 1999 01:50:28 -0600

Chris Wolcomb wrote:
>
> 3) Those who believe the singularity is not inevitable but want it to come as soon as possible.

That's me.

> As an extropian, I want to shape my future around my desires. Any force which attempts to curb my desires will be met with the strongest resistance possible.

As a mortal, I'd like to see humanity survive in any form at all. Getting the future to conform perfectly to my desires is ludicrous and would probably result in eternal living hell, no matter how careful I was.

As a Singularitarian, I want a Singularity to happen, but I don't really care when it happens as long as it's within the next ten thousand years or so (a mere moment on a galactic time scale). Of course, I could be totally wrong about the lack of urgency; it could be that every single second of delay is more terrible than all the pains of human history. But the moral scales (and my feeble understanding of How Causality Actually Works) seem to me to tilt far more towards "maximize probability" than "minimize time".

The question doesn't arise.
My two allegiances don't come into conflict. It's Singularity or bust.

Okay, maybe nuclear war would just delay things by a century. So then we have the same option again - Abort, Retry, Singularity. And again. And again. Until either intelligent life wipes itself out or creates its successor.

Okay, so maybe the Singularity will wipe out humanity for spare atoms. Maybe it won't. I accept both possibilities. If I knew "for certain sure" it was option one, I'd still be a Singularitarian; but as has been stated, I'm not sure, so the question of divided allegiance doesn't arise.

I race towards the Singularity because every delay in the Singularity increases the chance that there will be no human Singularity, ever. And while I might weep over that for my own peculiar logical reasons, your reason for weeping is that it means that you and everyone you know and the entire human race will be extinct.

Screw dynamic optimism. The future isn't a playground, it's a minefield. Okay, so it's a unique minefield in that blind panic will get you killed even faster than blind enthusiasm, but blind is blind. The fact is that shiny new technologies always get used for military purposes, and that technological advance decreases the amount of cash needed to wipe out human life. Forgive me, but I don't see any reason for raving optimism.

> According to several Singulatarians, quite literally nothing - it considers 'me' irrelevant.

I don't know that, and if anyone else says they do, they're mistaken. I simply offer no guarantees and point out that no guarantees are necessary.

> So why should I accept it or want it? If the Singulatarians are right, then the Singularity will sweep away everything we know in its rapid path to increasing complexity.

Again, this is only a possibility.

> I would much rather spend a few centuries in a place like Iain Banks' Culture, than become part of some blind singularity as soon as possible.

Yes, I'd love to spend a few centuries in the Culture, especially the new revised-history "You can Sublime off any time you get bored" version. But I don't see any GCUs in Earth's atmosphere, do you? Evidently this simply isn't an option.

Besides which, the Archive (my own invention; see _Singularity Analysis_) beats the Culture any day of the week.

> Now, what if I were to start a movement to stop the singularity from occurring? I think I'll call this new political movement - the Anti-Singularity Party. Some might say I I'm going down the path to curbing people's freedom - like freedom to build seed AI's.

I happen to think that nanotechnology could get us all killed. Do you see me proposing any laws to restrict Zyvex? That trick never works. Not for me, not for you. Restricting technology is always more destructive than the technology itself.

> Yet, how is this any different from *them* creating a singularity and forcing me to be either be assimilated or weeded out through cybernetic natural selection?

Very simple. We are not forcing you to do a damn thing. We don't have the right. We are not superintelligent. You are not superintelligent either and have no more right than I to impose your views on your fellows.

A Culture Mind might be able to search through enough futures to safely suppress a technology. I can't. The same applies to coercion. I do not refrain from coercion and suppression for any fundamental moral reason, but because they ALWAYS end in disaster.

If you're really right about coercion being morally wrong, I'm sure that our Future Friends will be able to figure it out as easily as you did.

> So what if most living transhumanists do not want to be absorbed into the 'sublime plenum' that is this singularity? I've heard Singulatarians say "too bad, you will be assimilated by the singularity anyway'.

If that's how the cards are set up, yes. If not, not. It's not something I think we can influence via initial conditions.

Look, the human brain is finite. It's got a limited number of states. So you have to die, go into an eternal loop, or Transcend. In the long run... the really long run... mortality isn't an option.

So why not just go ahead and do it, leave the womb, for good or for evil, while my grandparents are still alive and before the Earth gets eaten by grey goo?

Wouldn't it be ironic if you delayed the seed AI, got smothered in goo, and all along it turned out that the Singularity would have obligingly provided an environment hedonistic beyond the most fevered imaginings of the AhForgetIt Tendency?

> Chris Wolcomb.
> GSV Its Only a Matter of Time.

eliezer yudkowsky
the excession

> P.S. Eliezer, we now find ourselves on the opposite sides of a new political spectrum - those of you for the Singularity, and those of us against it. The future is shaping up to be very interesting. :-)

Considering how hard I've been pushing against the opposition, I can't say I'm displeased to find it coalescing, particularly on my terms.

-- 
        sentience@pobox.com         Eliezer S. Yudkowsky
         http://pobox.com/~sentience/AI_design.temp.html
          http://pobox.com/~sentience/sing_analysis.html
--
Who on this list is Culture, who is Contact... and who is SC?