Re: Failure of AI a prediction of Neal Stephenson's "The Diamond Age"

Eliezer S. Yudkowsky (sentience@pobox.com)
Mon, 21 Dec 1998 13:32:22 -0600

Anders Sandberg wrote:
>
> Suppose somebody actually
> manages to come up with a convincing experimental proof for a
> non-material soul (I would really like to see the methods and
> equipment section of that paper! :-)

Well, let's say that tomorrow physics discovers a set of non-computable interactions called weirdodynamics, which can perform useful information-processing, and have the interesting property that any weirdodynamic interaction continues to interact forever. If a neuron can be demonstrated to use weirdodynamic interactions, and weirdoscanners used to find and retrieve a dead person's visual memories and display them onscreen, then I think any scientist would have to admit that there was a soul. Dunno about "non-material"; I've never heard a good explanation of what "non-material" could possibly mean.

> This is important to remember for us transhumanists, since we put so
> much faith in some technologies that might actually never be done. If
> we stop being openminded about being wrong, even about our most
> cherished beliefs, we will become just like any other religion, cult
> or ideology.

I've seen the word "extropian" used in science fiction on three occasions: "Slant" by Greg Bear, "Voice of the Whirlwind" by Walter John Williams, and "Wang's Carpets" by Greg Egan. In the last two cases, the "Extropians" are a cult. In the first book, the "Extropians" are mentioned as justification by a member of a cult.

I have great difficulty visualizing, say, Anders Sandberg or Max More or Mitchell Porter as members of a cult, but I still agree with the SF in question. Once Extropianism goes mainstream, there won't be real Extropians any more; just Singularitarians and AIers and IAers and nanotechnologists... and one cult group, composed of all the clueless people, continuing to call themselves "Extropians".

> BTW; the baby analog is spurious. It just shows that intelligence is
> possible, just as biology shows that physical law allows nanotech. It
> does not say we can necessarily build it.

The baby analog does show we can build intelligence, by doing the same thing as DNA (unless routine divine intervention is involved), but says nothing about whether we can create intelligence using only transistors, or whether neurons exploit weirdodynamic effects. 

-- 
        sentience@pobox.com         Eliezer S. Yudkowsky
         http://pobox.com/~sentience/AI_design.temp.html
          http://pobox.com/~sentience/sing_analysis.html
Disclaimer:  Unless otherwise specified, I'm not telling you
everything I think I know.