hal@finney.org wrote:
>
> Eliezer S. Yudkowsky, <sentience@pobox.com>, writes:
> > Frankly, I think a fairly large percentage of you *are* naive
> > technophiles. You think you can take controlled sips from a tidal wave.
>
> However, many us don't subscribe to the runaway AI/singularity scenario.
I was mostly referring to the nanoSanta tendency. I've changed philosophies quite a bit over the last three years, but one thing that's remained constant since the day I posted "Staring Into the Singularity" is that I despise childhood fantasies of omnipotence; it shows a lack of both imagination and maturity. Yes, nanotech is powerful enough to build you a luxury Mercedes, or heal your hand if it gets run over by a car. But it's way, way, way more powerful than that. Even leaving out all IA, AI, and military implications, it would change the fabric of society past all recognition. Accounts of twentieth-century American culture unaltered by nanotech are as foolish as accounts of a computer-assisted hunter-gatherer society using guided missiles to hunt down deer.
But no, they don't think that. They think in Santa syllogisms. They want a Mercedes. Nanotech is big and powerful. Nanotech will give them a Mercedes. QED.
What *else* will it give them? Supercomputers easily capable of running AIs? Uploading? Vingean headbands? A mass diaspora from Earth? They don't know. They don't care. All they want is a Mercedes. What if I try to point out the military implications? They don't want military implications. Repeat the words "active shields". This is a magic cure.
> As Robin Hanson pointed out, we don't know how quickly the difficulty of
> increasing intelligence grows as you become smarter. Is it easier for
> an IQ of 400 to go to an IQ of 800 than for an IQ of 200 to go to 400?
> We have no evidence one way or the other. It is uncharted territory.
Those two statements are entirely unequivalent. Yes, it's uncharted territory. That doesn't mean we have no grounds for believing some statements over others. I don't see any grounds for believing in a "difficulty" that will prevent a nanocomputer with a million times the raw computing power of a human from being at least as much smarter than humans as humans are from chimpanzees, or in a difficulty that prevents an AI running over the Internet from being intelligent enough to use rapid infrastructure to recompile the planet's mass and upload the population. Whether difficulties occur after that is something of a moot point, don't you think?
--
sentience@pobox.com Eliezer S. Yudkowsky
http://pobox.com/~sentience/tmol-faq/meaningoflife.html
Running on BeOS Typing in Dvorak Programming with Patterns
Voting for Libertarians Heading for Singularity There Is A Better Way