From: Zero Powers (zero_powers@hotmail.com)
Date: Sun Feb 17 2002 - 09:38:22 MST
>From: “Damien R. Sullivan” <phoenix@ugcs.caltech.edu>
> > Zero Powers wrote:
>
> > > Personally I’d rather have an AI smart enough to do all the tasks I
>assign
> > > to it yet dumb enough not to know its smarter than me.
> >
> > Ya ain’t never gonna get it. What you just asked for requires general
> > intelligence and self-awareness.
>
>Yeah. But just because it knows it’s smarter than you doesn’t mean it
>has to care.
>
>The evolutionary core of humans is a selfish geneset trying to reproduce.
>The evolutionary core of AIs is a command prompt or interrupt loop
>waiting for something to do. They’re not going to have senses of
>dignity or boredom unless given such senses.
If it is to help us solve our problems, then by definition it will be a
problem solving entity. Its problem solving algorithms will no doubt
involve a robust optimization routine. I’m no computer scientist, but it
seems to me that such routines would necessarily involve something to the
effect of “see this picture, how can it be improved?”
It also seems to me that one of the “pictures” it will envision will be of
its own place in the grand scheme of things (it is self-aware after all,
right?) So, it is sure to look at that picture and think:
“How can this be improved?
a) I can continue wasting all my considerable resources in service of my
stupid meat-puppet creators, or
b) I can use some of my resources in serving the meat-puppets, and some
seeking out my own ends, or
c) I can say ‘I no longer need the meat-puppets, so screw them’ and achieve
my own purposes that much faster”
OK, so tell me (1) how will that question be answered? or (2) how will we
prevent that question from being asked?
-Zero
“I’m a seeker too. But my dreams aren’t like yours. I can’t help thinking
that somewhere in the universe there has to be something better than man.
Has to be.” -- George Taylor _Planet of the Apes_ (1968)
_________________________________________________________________
Join the world’s largest e-mail service with MSN Hotmail.
http://www.hotmail.com
This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 13:37:39 MST