Re: Preventing AI Breakout [was Genetics, nannotechnology, and ,

Eliezer S. Yudkowsky (
Mon, 25 Oct 1999 15:01:53 -0500

Matt Gingell wrote:
> I think the original remark misses the depth of Church-Turing. It's
> not just talking about what you can do with a computer, it's talking
> about what you can do with _any conceivable_ formal system. If you can
> find expressible regularities in the universe which can't be described
> under the lambda calculus then you've disproven the conjecture.

With all due respect, I do *not* "miss the depth" of things like Church-Turing-Tarski. I believe the Church-Turing-Tarski thesis is false and will be disproved. I believe there are expressible regularities in the laws of physics that cannot be described under the lambda calculus, by Turing machines, et cetera. Furthermore, I believe that *any* complete causal explanation of an event, which means the complete chain of causality extending back to the reason why anything exists in the first place, *must* contain noncomputable phenomena. I believe that the noncomputability of objectively real phenomena is a necessity, not a coincidence. And, for the record, I believe that the Church-Turing-Tarski thesis is fundamentally *wrong*, not just limited in the space of phenomena it describes; there is *no such thing* as a simulation; it is not possible to "compute" the complete behavior of a quark without creating an actual quark.

I don't know that a genius-in-a-box that naturally evolved in a Life universe could deduce that it was in a simulation, but I know it could deduce that it wasn't looking at the lowest level of reality.

           Eliezer S. Yudkowsky
Running on BeOS           Typing in Dvorak          Programming with Patterns
Voting for Libertarians   Heading for Singularity   There Is A Better Way