Re: Why would AI want to be friendly?

From: Scott Badger (w_scott_badger@yahoo.com)
Date: Sat Sep 09 2000 - 11:54:58 MDT


--- hal@finney.org wrote:
> Robin writes:
> > Humans do have an abstract reasoning module, and
> that module often tries
> > to give the impression that it is in fact in such
> control over the rest
> > of the mind. But in fact I think our conscious
> minds are more like the
> > PR department of our minds - they try to put a
> good spin on the decisions
> > that are made, but are not actually in much
> control over those decisions.

> I suspect that most of what we perceive about our
> own consciousness is wrong. It is like a visual /
> perceptual illusion, brought inward and made
> pervasive. The entire mind is an optical illusion.

{snip}
> Dennett has a model along these lines, in which
> consciousness is essentially an illusion, a lie
> created by our minds. We can combine this with
> Minsky's "society of mind" and get the following
> picture (blue-sky speculation):

{snip: wonderful description of illusory nature of
consciousness]

This model seems likely to me as well. I think of the
"I" as the part of the mind that watches the actions
of the various societal agents and immediately spins a
narrative ("The Story of Me") to explain it all "as
if" the "I" were a unitary causal entity. This is old
ground but, I like the analogy of the rainbow, an
emergent phenomenon brough about by and dependent on
particular physical processes. "I" am the rainbow that
thinks it controls the light and the rain. I have
conceptualized transhumanism in the past as the
rainbow's attempt to become that which it dreamed it
was ... to move from the insubstantive to the
substantive.

But now I wonder.

If I seek transcendance through intelligence
enhancement, uploading, or something else, can't I
expect to lose this illusion I call my identity? What
need would there be for such an illusion? Those I've
discussed this with claim that it makes little sense
for "me" to want to transhumanize if the "me" is lost
in the process. There would only be a memory of the
"me" illusion ... like remembering how the highway
looks like water on a hot day.

It's odd because we cryonicists are all about
preserving our identities/personas/personalities/etc.
and we have long threads on the issue from time to
time, but are we simply hoping to preserve something
that is little more than a vestigial aspect of the the
really important parts of our total architecture?

It's interesting to think that "I" am not planning on
transcending at all. The sub-conscious agents of my
mental architecture are making those plans while "I"
take the credit.

I'm looking forward to looking backward and finally
understanding.

So can we expect AI to have a architecture similar to
huumans ... multiple agents acting in a coordinated
fashion but without a central component analogous to
our notion of a self?

--Scott

__________________________________________________
Do You Yahoo!?
Yahoo! Mail - Free email you can access from anywhere!
http://mail.yahoo.com/



This archive was generated by hypermail 2b29 : Mon Oct 02 2000 - 17:37:34 MDT