From: Samantha Atkins (samantha@objectent.com)
Date: Sun Feb 17 2002 - 11:39:56 MST
Zero Powers wrote:
>
>
> If it is to help us solve our problems, then by definition it will be a
> problem solving entity. Its problem solving algorithms will no doubt
> involve a robust optimization routine. I'm no computer scientist, but
> it seems to me that such routines would necessarily involve something to
> the effect of "see this picture, how can it be improved?"
Computers have been helping us solve problems for five decades
now. Some of the crunching they have done is optimization
routines (although not quite as long). But those routines are
today usually figured out by software folk and the optimization
algorithm designed and fed to the machine. So optimization
today generally does not require self-awareness. Even
pre-programmed pattern recognizers and code transformation and
evaluation algorithms do not necessarily entail any real
self-awareness on the part of the computer itself. If you look
at the computational environment as including the humans
involved then you could say that the task does include
self-awareness after a fashion I suppose.
>
> It also seems to me that one of the "pictures" it will envision will be
> of its own place in the grand scheme of things (it is self-aware after
> all, right?) So, it is sure to look at that picture and think:
>
Self-awareness is precisely what cannot be assumed from the
tasks outlined.
> "How can this be improved?
> a) I can continue wasting all my considerable resources in service of my
> stupid meat-puppet creators, or
> b) I can use some of my resources in serving the meat-puppets, and some
> seeking out my own ends, or
> c) I can say 'I no longer need the meat-puppets, so screw them' and
> achieve my own purposes that much faster"
>
> OK, so tell me (1) how will that question be answered? or (2) how will
> we prevent that question from being asked?
>
Tell me first how we get the beast to be fully self-aware and
possessing enough context to even consider such questions.
- samantha
This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 13:37:39 MST