Spike Jones wrote:
>
> Eliezer S. Yudkowsky wrote:
>
> > I, incidentally, am a fanatic Singularitarian. I want something smarter and
> > more powerful and more ethical than I am to exist; I don't care how, or what
> > happens to the human race or myself afterwards...
>
> eliezer, you are not alone. i suspect that desire for something greater is
> hardwired into our brains by way of genetics.
Spike, you're talking to someone with Web pages about evolutionary psychology. I appreciate the thought. However, I have done my best to filter politicosocial imperatives out of it. What I have set out to do has a significant probability of annihilating the human race, and I know it. You'd better believe I'm not in it from some vague desire to be part of something big. I grew up knowing I was a genius and knowing that I could almost certainly be part of something big if I played my cards right. I am not and have not ever been acting on an insignificance complex. I'll admit that it was pride and the desire for acknowledgement that figured in my fantasies, back when I thought I could have my cake and eat it too. But now...
Now, I happen to think that even from humanity's perspective, a rapid Singularity is the best way to go, because I don't see a feasible alternative. At this point, the only way to prevent a Singularity is blowing civilization back to the Stone Age or wiping humanity out entirely via grey goo. In which case, everyone now living will die in any case. The only way that any of us can "not die" is through a friendly Singularity. If that's impossible, well, at least our deaths will be the ethically correct thing to do. Sooner or later human civilization will perish or go through a Singularity. This I guarantee. What can you possibly accomplish, for yourself or for anyone, by delaying it?
But that could all be rationalization. It's not the reasoning I use. I wish to do the right thing, which is a question that is resolved by intelligence, and thus I am required to create a higher intelligence to accept orders from. My allegiance is to the Singularity first, humanity second, and I'm tired of pretending otherwise to myself. If you want safeguards, if you want to negotiate using humanity's ability to prevent a Singularity by exterminating itself, if you would like humanity to be a game-theoretical partner whose interests must be considered, then get a non-hypothetical negotiating position and say so.
--
sentience@pobox.com Eliezer S. Yudkowsky
http://pobox.com/~sentience/AI_design.temp.html
http://pobox.com/~sentience/sing_analysis.html
Disclaimer: Unless otherwise specified, I'm not telling you
everything I think I know.