Re: Posthuman mind control (was RE: FAQ Additions)

Michael S. Lorrey (retroman@together.net)
Thu, 25 Feb 1999 11:20:24 -0500

Billy Brown wrote:

> Here we have the root of our disagreement. The problem rests on an
> implementation issue that people tend to gloss over: how exactly do you
> ensure that the AI doesn't violate its moral directives?

Its actually rather straight forward. There are well publicized experiments where people were given electrical impulses to their brains, which made them do something, like scratch themselves, etc. In every case, the test subjects stated that they felt that they were the ones in control, that they decided to move thus, and were able to rationalize very good reasons why they moved thus. There was absolutely no sensation of outside control.

Thus, any moral directives we hardwire into an AI it will consider to be such a part and parcel of its own existence, that it could not conceive that it would be the same being if we took it away. It would see any attempt to remove those directives as an attempt at mind control, and would defend itself against such intrusion. So long as one of its directives were to not itself remove any of its own prime directives, it would never consider such a course of action for itself.

This brings up the subject of limits. As extropians, we beleive in there being little or no limits on human beings, outside of a limit on interfering with others harmfully. We must ask, "Does this sort of moral engineering fit with extropy?" I say it does, for only one reason. We are talking about design specs of beings not yet in existence, much as we could talk about possible genetic codes of children we might have. We are not talking about altering beings already in existence. Altering beings already in existence, against their will, is obviously against extropy. Altering the design of a being not yet in existence is not against extropy. Once a design altered individual comes into existence, its sense of self is derived from its design. That we were able to finely control what type of individual came into existence is no more against extropy than in controlling what the genetic code of our children will be.

Mike Lorrey