Re: Preventing AI Breakout [was Genetics, nannotechnology, and

Eliezer S. Yudkowsky (
Sun, 24 Oct 1999 23:09:31 -0500

Joseph Sterlynne wrote:
> It seems that everyone is confident that an AI of sufficient intelligence
> will be able to discover that it exists within a simulation. This is a
> little surprising. I won't say that we could absolutely guarantee its
> eternal ignorance but I don't see why an AI would not simply accept the
> laws of its environment as the laws of a nonconstructed universe.

Maybe it'd accept the laws of physics, if it wasn't smart enough to engage in a-priori ontological reasoning. But how is the AI supposed to accept itself? We all know that the human body and human mind are the result of evolution, right? Conscious design would be just as obvious to the AI.

           Eliezer S. Yudkowsky
Running on BeOS           Typing in Dvorak          Programming with Patterns
Voting for Libertarians   Heading for Singularity   There Is A Better Way