From: Zero Powers (zero_powers@hotmail.com)
Date: Mon Feb 18 2002 - 23:18:10 MST
>From: "Eliezer S. Yudkowsky" <sentience@pobox.com>
>You know, I hadn't intended to ask, but I just don't get it. Why are there
>no
>traces at all of Friendly AI concepts in this discussion? I don't just
>mean
>the answers, I mean the underlying systemic concepts used to untangle the
>questions. Nobody is distinguishing between supergoals and subgoals.
>Nobody
>is distinguishing between the goal system, the goal system's external
>referents, and the AI's reflective design model of its own goal system.
>People are asking the same questions they were asking in 1999. I could
>understand if there'd been progress and yet not progress toward Friendly
>AI,
>but this is just stasis. Why?
Well, as the guy who started this thread, I suppose it falls to me to answer
that one. My answer is that, so far as I comprehend it, your
supergoal/subgoal explanation does not resolve the problem. My
understanding of your theory is that, so long as the AI's sub-goals are all
in service of friendly supergoals, we have nothing to fear. But, I'm making
a few assumptions, and here they are:
1. The AI will be curious (either as a sub- or supergoal);
2. The AI will have "general intelligence" (such that it can evaluate a huge
decision tree and choose which among the myriad branches will best meet its
goals);
3. It will be self-aware (such that it has an accurate picture of the world,
including its own place in that picture).
If any of those assumptions is wrong, then you can forget everything I've
said before. But, if each of those assumptions is correct, then they
inherently conflict with your concept of a friendly AI. If you do not see
this conflict let me know and I'll paint the picture in full detail (if
you're still interested by now.)
-Zero
"I'm a seeker too. But my dreams aren't like yours. I can't help thinking
that somewhere in the universe there has to be something better than man.
Has to be." -- George Taylor _Planet of the Apes_ (1968)
_________________________________________________________________
MSN Photos is the easiest way to share and print your photos:
http://photos.msn.com/support/worldwide.aspx
This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 13:37:40 MST