Re: Why would AI want to be friendly?

From: Eugene Leitl (eugene.leitl@lrz.uni-muenchen.de)
Date: Wed Sep 06 2000 - 08:45:20 MDT


James Rogers writes:

> I don't think this is an accurate characterization. The rules don't
> change at all; there is still fierce competition among intelligent,
> self-directed entities. All that happens is that the competition moves
> out of the grossly physical domain for the most part.
 
We're still pretty far from an equilibrium, still being in a
spontaneous expansion process of civilisation into the wilderness. The
grossly physical part may well come back with a vengeance (see the
emergence of neoplagues and pests for a shade of things potentially to
come), when we're nearing the more sustainable/equilibrium part of the
development. Assuming the constraints of finite concentration of
matterenergy in spacetime must always hold, things must eventually
plateau. Before things might pass through a sequence of bottlenecks,
e.g. if we can't expand freely into space, after having covered the
planet with a thick crust of manmade artefacts and people.
 
> The capability to project outcomes of actions as a result of increased
> intelligence and knowledge is most likely responsible for this shift.
> The ability to accurately forecast the costs/benefits for a broad range of

Of course, by expressing the planned behaviour, we're collectively
changing the state of the system, and hence introduce an additional
uncertainty. The others constitute a major part of the fitness
function, which has to move when their strategies move. For instance,
all the fat dotcom fishes in a tiny pond create a lot of ruckus, and
muddy the waters. Mutually making planning more difficult. Time for
some dynamite fishing ;)

> potential actions would encourage more subtle and less costly
> manipulations than brute force to achieve the same effective results in a
> competitive environment. It also allows one to recognize losing

All that assuming smart agents. They are not the only one on the
stage. Dumb agents still pretty much brute-force.

> positions and to cede them at the lowest possible cost. Maneuver is less
> costly than attrition and not as grossly destructive, but it requires
> superior situational intelligence to accomplish.

If only we were so rich as we're smart... ;P



This archive was generated by hypermail 2b29 : Mon Oct 02 2000 - 17:37:20 MDT