Re: Why would AI want to be friendly?

From: Samantha Atkins (
Date: Mon Oct 02 2000 - 12:20:25 MDT

Eugene Leitl wrote:

> > Some questions I have include:
> >
> > 1. What difference would cooperative coevolution make with respect to the
> > relationship between humans and highly evolved AI?
> The AI can't derive any benefits from a transaction with the human
> player(s). Apart from eating you, of course, since it has good use for
> all the atoms in your body.

If this is truly the case then every human being should work their butt
off to insure that such a creation is never built. Hopefully we can
come up with better scenarios that allow us to have our SI and not be
eaten too. But if the above is the end of the discussion then what sort
of person would actually try to build this? Wouldn't such a person be
doing technology as a sacred Holy Grail even if it would for certain
destroy all humanity? Sorry, but this vision certainly won't get me out
of bed every morning.

We are assuming a perfect AI with not only switching capacity 10^6
better than us but an equal or greater degree of parellism (or something
as good or better) happens pretty much overnight and it thus has zero
use for humans (as if "use for" is the end and be-all all interactions
anyway). We assume in our technological fever dream that it will almost
at once think oh so much faster and better than us little meatballs that
it will see us as less than nothing and dispose of us as a lite snack.
We assume ethics is bunk and only survival and growth will drive this
super-intelligence. And we will largely cheer it on and help build it.
Then we wonder why there are technophobes in the world.
If I believed this is the best we can possibly come up with then I would
be a technophobe.


> In comparison to mechanosynthesis-based life we're extremely
> time-space-energy inefficient. Our relative fitness is so low we don't
> have the chance of a snowball in hell.

Sure. If you assume Darwinian evolution and a rather single-minded
survival of the fittest model is the end and be-all of what intelligent
and super-intelligent beings can come up with. I am not willing to
believe that as of yet.

- samantha

This archive was generated by hypermail 2b30 : Mon May 28 2001 - 09:50:14 MDT