From: Eliezer S. Yudkowsky <sentience@pobox.com>
>Zero Powers wrote:
> >
> > Well, back to the heart of my initial question. Is friendliness toward
> > humans a supergoal, a subgoal, or even a goal at all? I assume
>(possibly
> > incorrectly) you will claim it to be a supergoal. If that is the case,
>how
> > do you keep a sentient AI focused on that goal once it begins to ask
> > existential questions?
>
>If the existential questions don't have answers, I don't see why the SI
>asking
>them would pose a problem. If there is no objective morality, then the
>existential questions don't have answers. If there is objective morality,
>then the initial suggestions will get junked anyway.
Could be we are getting tripped up on each other's semantics. I don't
believe there is such thing as a question without an answer. There may be
no "objective" morality in the sense that one morality will work for every
intelligent being at all times and all places. But, for each individual,
and in each circumstance, there is certainly a morality which works best.
It may not necessarily be "objective" per se, but it is real nonetheless.
I believe at some point AI will say to itself, "What is the best course of
endeavor for me, considering the kind of being I am?" I highly doubt it
will answer that question by saying "The best thing for me to do is obey the
commands of these ignoramus humans, because that's what they tell me to do."
When you say "the initial suggestions will get junked anyway" if there is an
objective morality, then you are pretty much conceding that ultimately
there'll be little assurance of human-friendly AI, right?
>Remember, an SI isn't going to be tormented by the pointlessness of it all
>because it doesn't have the be-tormented-by-the-pointlessness-of-it-all
>hardware.
What makes you think our torment at pointlessness resides in the hardware?
Seems pretty obvious to me its in the *software*. In terms of hardware,
ours isn't much different than monkeys. Somehow I just don't see them as
suffering torment over pointlessness.
-Zero
Learn how your computer can earn you money while you sleep!
http://www.ProcessTree.com/?sponsor=38158
_________________________________________________________________________
Get Your Private, Free E-mail from MSN Hotmail at http://www.hotmail.com.
Share information about yourself, create your own public profile at
http://profiles.msn.com.
This archive was generated by hypermail 2b29 : Mon Oct 02 2000 - 17:37:34 MDT