Andrew Lias wrote:
> I've been following the debates regarding the possibilities of friendly vs.
> unfriendly AI and I have a question. It seems that we are presuming that a
> friendly AI would be friendly towards us in a manner that we would recognize
> as friendly. Indeed, what, precisely, do we mean by friendly?
I think this is a good point (and welcome to the list!). It's not so
much a problem in what we build into the AI, it's that "friendly" is
a very loose concept. It's so loose that I doubt that it is really
meaningful as a top-level goal. Such a goal would have to have a
relatively precise definition, otherwise the AI will not be able to
rigorously derive subgoals from it.
What do we really want an AI to do? We need to clarify our own thoughts
and ideas on the matter to have any hope of telling it what we want.
One problem is that there is going to be a tradeoff between challenge
and happiness. The AI can make people happy by putting nanobots in their
heads. He can give them challenges that they are rigged always to win,
but that cheats them out of any meaningful success. He can give them true
challenges where they may fail, but then some people will be unhappy.
The word "friendly" fails to give any insight to the AI about where he
should aim for on this scale (and on other similar tradeoffs). Humans
will disagree about what a "friendly" AI should do. Satisfying some
people will disappoint others. Even a single person may have mixed
feelings about what is best.
I don't believe there is an answer to the question of what the AI should
do for humans. It's not that we're not smart enough to see the answer,
it's that the question itself is not well posed. That word "should"
keeps creeping in.
Ultimately we're back to Absolute Morality again. We want the AI to be
God, to do what is Good, what is Best. Capital letters are everywhere.
This concept just doesn't wash. There are no absolutes when it comes
to these issues, IMO.
Hal
This archive was generated by hypermail 2b29 : Mon Oct 02 2000 - 17:39:30 MDT