Re: Attacks (was Re: Why would AI want to be friendly?)

From: Emlyn (
Date: Wed Oct 04 2000 - 00:59:29 MDT

Samantha wrote:
> Simply this, if it is all just "Doom", then there is no reality, no real
> context or pushback. Once we get uploaded I think it is going to be
> increasingly important to define realities a bit more strictly. Would
> you like to spend the next umpteen millenia in environments where other
> entities will wipe out your being in that environment just for the heck
> of it because after all you are fully backed up and so on? It doesn't
> sound like good fun to me.

So, I wander off into another environment, big deal. I think Eliezer's point
was that there is such abundance for all, that no-one can harm anyone else
in any real fashion.

I do dissagree with that, in the end. For instance, in the above scenario,
maybe I'm attached to the identity I've created in environment X, maybe I've
built reputation, maybe going elsewhere is not what I want. But then, maybe
I can be unfraggable (or whatever), depending on the rules of the

The "rules of the environment" is an interesting idea. You could build an
environment (most likely many and varied) to enforce certain rules, like
unkillability, or karma, or killer hamster attacks, or whatever. Whatever
you like. If people want in, in they come.

However, each member of the environment either has "admin" privellages, or
not. If not, then the environment surely a prison. If the person has admin,
on the other hand, then the "rules" are toothless tigers, shadows, only
applicable whilst the person wishes them to be.

This leads to the conclusion that these environments must either have some
top level rules larger than and beyond the reach of the contained entities
(thus prisons), or that the environments could hold no real interest or sway
over members for more than trivial time, unless those contained entities are
vapid escapists who choose that the rules bind them indefinitely, rather
than deal with the rest of the universe.

The only environment which doesn't hold to this analysis is the "real
world", the physical universe, and only given that it's laws are not
dictated by a sentient (otherwise it classes as a prison, as above).

> For the crims and a special hypothetical environment that is just a big
> "Doom", well, the problem is there is zero opportunity to learn to live
> in a less violent environment. There are no repercussions and no
> incentives (beyond hopefully eventual boredom). There may well be such
> environments but I doubt they would be useful for anything but certain
> types of "entertainment". This may mark me as really odd but personally
> I don't have any use for environments where people frag other people for
> fun. I realize that some people do. To me that makes them odd. But
> heh, it takes all kinds.

Why should anyone learn if they don't want to? The only reason to force
learning on others is to help ensure they respect a shared environment
better. The context in which the "doom" environment was proposed
(non-seriously) was one in which you don't have to share if you don't feel
like it (unlikely, to be sure). In such an environment, prisons make no
sense, except if your goal is power over others.


This archive was generated by hypermail 2b30 : Mon May 28 2001 - 09:50:15 MDT