Re: Why would AI want to be friendly?

From: Samantha Atkins (
Date: Sun Oct 01 2000 - 16:45:02 MDT

Barbara Lamar wrote:
> On Sun, 1 Oct 2000 02:10:17 -0700 (PDT) Eugene Leitl
> <> writes:
> > Eliezer S. Yudkowsky writes:
> >
> >
> > Friendly is not observer-invariant, goddamit. If you want to impose
> > your particular concept on the rest of us via (thankfully
> > hypothetical) AI proxy, then please call a spade a spade. Call it
> > Eliezer-Friendly, or something.
> I brought this same point up towards the beginning of this thread,
> addressed specifically to Eliezer, and there was no response.
> Two days ago I posted a URL for a recent paper by a couple of people who
> are currently working in the AI field--"Cooperative Coevolution: An
> Architecture for Evolving Coadapted Subcomponents" by Mitchell A. Potter
> (Navy Center for Applied Research in Artificial Intelligence) and
> Kenneth A. DeJong (Computer Science Dept. George Mason University).
> Published in *Evolutionary Computation* 8(1) (2000)

> Again there was no response. As an outsider to the AI field, I would
> nevertheless like to have a better understanding of it. Has anyone else
> read Potter's and DeJong's paper?

I have not finished it but I believe it is very applicable both to the
creation/evolution of an SI and to human/AI coevolution. While I
believe it is a very important work and highly germane to this dicussion
(and others) I am letting its implications and repurcussions bounce
around a little before I say much more.

> 1. What difference would cooperative coevolution make with respect to the
> relationship between humans and highly evolved AI?
> 2. Are the capabilities of AI severely limited in the absence of
> algorithms for cooperative coevolution?
> 3. Potter's and DeJong's EA's call for destroying unsuccessful
> "species" and maintaining "genetic" diversity by introducing new species
> whenever stagnation is detected in the system. Stagnation is detected by
> monitoring the quality of interspeices collaborations via checking each
> collaboration for the improvement it provides in the functioning of the
> "ecosystem" as a whole. Is this methodology troubling with respect to
> possible future interaction between humans and AI? Why or why not? What
> are some other methods of maintaining diversity?

It is quite troublesome and probably unnecessary since species can be
modified inline to have expanded/corrected abilities, including
ourselves. Not many seem to yet realize that the rules have changed
this radically. It is not only an AI that can self-modify. Evolution
does not require destruction of sentient entities. Without this
possibility our hopes for immortality are empty.

A 10 billion dollar question is what is the "ecosystem"? What are we
attempting to create? If we don't know that can we judge whether we are
getting closer or further away?

> 4. Do the "species" in this research correspond more closely to
> biological genes or to biological species?

Not sure that question is important. Both, Neither.

- samantha

This archive was generated by hypermail 2b30 : Mon May 28 2001 - 09:50:14 MDT