Spike Jones wrote:
> Eliezer S. Yudkowsky wrote:
> > I, incidentally, am a fanatic Singularitarian. I want something smarter and
> > more powerful and more ethical than I am to exist; I don't care how, or what
> > happens to the human race or myself afterwards...
> eliezer, you are not alone. i suspect that desire for something greater is
> hardwired into our brains by way of genetics.
Spike, you're talking to someone with Web pages about evolutionary psychology. I appreciate the thought. However, I have done my best to filter politicosocial imperatives out of it. What I have set out to do has a significant probability of annihilating the human race, and I know it. You'd better believe I'm not in it from some vague desire to be part of something big. I grew up knowing I was a genius and knowing that I could almost certainly be part of something big if I played my cards right. I am not and have not ever been acting on an insignificance complex. I'll admit that it was pride and the desire for acknowledgement that figured in my fantasies, back when I thought I could have my cake and eat it too. But now...
I am not acting on wishes. I do not at this time project that the Singularity will result in the gratification of any of the desires that initially motivated me, not pride, not the acknowledgement of greatness, not the fun, and probably not even the knowledge of success. The personal drama that once captivated me is irrelevant. I am acting on logic.
Now, I happen to think that even from humanity's perspective, a rapid Singularity is the best way to go, because I don't see a feasible alternative. At this point, the only way to prevent a Singularity is blowing civilization back to the Stone Age or wiping humanity out entirely via grey goo. In which case, everyone now living will die in any case. The only way that any of us can "not die" is through a friendly Singularity. If that's impossible, well, at least our deaths will be the ethically correct thing to do. Sooner or later human civilization will perish or go through a Singularity. This I guarantee. What can you possibly accomplish, for yourself or for anyone, by delaying it?
But that could all be rationalization. It's not the reasoning I use. I wish to do the right thing, which is a question that is resolved by intelligence, and thus I am required to create a higher intelligence to accept orders from. My allegiance is to the Singularity first, humanity second, and I'm tired of pretending otherwise to myself. If you want safeguards, if you want to negotiate using humanity's ability to prevent a Singularity by exterminating itself, if you would like humanity to be a game-theoretical partner whose interests must be considered, then get a non-hypothetical negotiating position and say so.
-- email@example.com Eliezer S. Yudkowsky http://pobox.com/~sentience/AI_design.temp.html http://pobox.com/~sentience/sing_analysis.html Disclaimer: Unless otherwise specified, I'm not telling you everything I think I know.