Re: Preventing AI Breakout [was Genetics, nannotechnology, and ,

Damien Broderick (
Sun, 24 Oct 1999 18:01:06 +0000

At 11:24 PM 23/10/99 -0700, Robert Bradbury wrote:

>I believe that Skye may have hit upon something every
>interesting that partially solves a major problem
>involving self-modifying AI that has plagued the
>list and worried me greatly:
>> It might be interesting if you could create an
>> artificial environment where you could test such
>> things. For example, if you had an artificial human
>> body existing on some kind of programmed level

Of course this idea is a commonplace of sf, and has been at least since Fred Pohl's `The Tunnel Under the World' in 1955, and Dan Galouye's 1964 novel SIMULACRON-3 (aka COUNTEFEIT WORLD) - where, interestingly, observers from the next layer up can `merge' with selected sims via upload-style couplers, allowing a lower level guy to `escape' into and overwhelm the brain of the cognitive scientist who wrote him, using an isomorphism of his own brain as a template. (Warning: never try to do this at home, kids.)

Damien Broderick