>From: xgl <xli03@emory.edu>
>Subject: Re: Why would AI want to be friendly? (Was: Congratulations to
>Eli, Brian ...)
>Date: Tue, 5 Sep 2000 09:30:29 -0400 (EDT)
>
>On Tue, 5 Sep 2000, Zero Powers wrote:
>
> >
> > So, I guess I have no fear of the AI being malignant like the “Blight”
>in
> > Vinge’s _Fire Upon the Deep_, but I can’t see how it is that we expect
>it
> > to give a hoot about our puny, little problems, or even to be “friendly”
> > to us.
> >
>
> i see no reason that an SI (the kind that eliezer envisions,
>anyway) would experience anything remotely as anthropomorphic as
>gratefulness. we are talking about an engineered transcendent mind, not
>a product of millions of years of evolution -- no parents, no siblings, no
>competition, no breast-feeding.
>
> as eliezer points out in his various writings, if such a mind does
>anything at all, it would be because it was objectively _right_ -- not
>because it feels good, and not as a result of any coerced behavior (ie,
>evolutionary programming). thus, even if the SI is the best alternative
>for the human race, i would still approach it with fear and trembling.
>
>-x
>
1. All intelligence is fallible.
2. Conscious intelligence uses perceptual feedback for base level
correction.
3. Emotional interaction with other conscious beings is the only method by
which one can "mirror" one's consciousness, as such.
4. Acting as a predator makes real emotional interaction impossible, as one
has to conceal part of ones nature and this required putting a conceptual
process into the loop, defeating the epistemological purpose - perceptually
reaffirming the nature of one's consciousness. (This is one of the base
reasons polygraphs work - and why we place such an extravagent value on
things like love.)
5. Any smart conscious being is capable of figuring this out.
Therefore, if the SI is conscious, we probably don't have to worry -
although I would want to have a serious talk with it on this very subject
before I put it on line. A very smart, non-conscious SI is another matter.
_________________________________________________________________________
Get Your Private, Free E-mail from MSN Hotmail at http://www.hotmail.com.
Share information about yourself, create your own public profile at
http://profiles.msn.com.
This archive was generated by hypermail 2b29 : Mon Oct 02 2000 - 17:37:24 MDT