Re: Why would AI want to be friendly?

From: Eugene Leitl (
Date: Mon Oct 02 2000 - 04:51:02 MDT

Barbara Lamar writes:

> I brought this same point up towards the beginning of this thread,
> addressed specifically to Eliezer, and there was no response.
I'm not surprised. He rarely responds to technical points, rendering
the whole interaction a rather sad parody of discussion.
> Again there was no response. As an outsider to the AI field, I would
> nevertheless like to have a better understanding of it. Has anyone else
> read Potter's and DeJong's paper?
Welcome to the wonderful world of extropians@, where resident AI
researchers don't have to read papers, because, well, they don't have
to, because they're so damn smart.
> Some questions I have include:
> 1. What difference would cooperative coevolution make with respect to the
> relationship between humans and highly evolved AI?
The AI can't derive any benefits from a transaction with the human
player(s). Apart from eating you, of course, since it has good use for
all the atoms in your body.

> 2. Are the capabilities of AI severely limited in the absence of
> algorithms for cooperative coevolution?

I don't understand that one.

> 3. Potter's and DeJong's EA's call for destroying unsuccessful
> "species" and maintaining "genetic" diversity by introducing new species
> whenever stagnation is detected in the system. Stagnation is detected by

Did they show that the resulting complex system is more efficient?
(Guess what? I haven't read it either. Must have missed that URI).

> monitoring the quality of interspeices collaborations via checking each
> collaboration for the improvement it provides in the functioning of the
> "ecosystem" as a whole. Is this methodology troubling with respect to
> possible future interaction between humans and AI? Why or why not? What
> are some other methods of maintaining diversity?
In comparison to mechanosynthesis-based life we're extremely
time-space-energy inefficient. Our relative fitness is so low we don't
have the chance of a snowball in hell.
> 4. Do the "species" in this research correspond more closely to
> biological genes or to biological species?
Huh? The genotype (a bunch of bits) encodes a phenotype ("behaviour")
when expressed in the system context. The phenotype gets evaluated for
fitness, not the genotype.

This archive was generated by hypermail 2b30 : Mon May 28 2001 - 09:50:14 MDT