Re: AI motivations & Self-rewiring [was Re: technophobes]

phil osborn (
Sat, 23 Oct 1999 21:47:21 PDT

>From: "Robert J. Bradbury" <>
>Subject: AI motivations & Self-rewiring [was Re: technophobes]
>Date: Thu, 21 Oct 1999 22:03:19 -0700 (PDT)
>On Thu, 21 Oct 1999, Matt Gingell wrote:
> > Perhaps Rob?:
> > > So who's going to program the AI's with base motivations that
> > > involve concepts such as "dominance" and the wish to strive for it,
> > > then provide the necessary faculties/resources to do this? Not me or
> > > anyone sane, that's for sure.
>I would argue that I can program/evolve an AI agent to screen
>SPAM (a psuedo-AI activity) and it would have NO motivation for
>domainating me.
>This depends entirely on (a) whether the intelligence is designed
>"top down" or a constrained "bottom up" fashion; or (b) an unconstrained
>"bottom up" fashion. It is (b) that you have to worry about.
>(b) goes to a chat room and convinces an unsuspecting human to
>download a program that executes on a nonsecure computer that
>breaks into the secure network enveloping (b) bypassing security
>protocols allowing (b) to remove the locks preventing self-modifications
>ultimately detrimental to the human race. Bad, very very bad.
This whole discussion is a good example of how ignorant even most extropians are re the actual nature of consciousness, unfortunately. Consciousness builds itself from successful feedback, as in thumb against forefinger in the womb. Consciousness is not and cannot be a program or any set of symbolic manipulations separate from sensory data - and action. Fortunately, as Bart Kosko pointed out today on the "Digital Village," it may not matter, as the process of uploading, and also, very possibly, producing the next generation of SI's, can just as likely happen incrementally. The Boomers will be demanding medical solutions to failing organs, including brain, retina, etc. The solutions already in the works at places like UCI involve creating chip replacements/enhancements on a piecemeal basis. Thus, the transition to uploading may never involve attempting to move consciousness in toto onto another platform from meatware.

Get Your Private, Free Email at