Re: Goo prophylaxis

Carl Feynman (
Thu, 28 Aug 1997 12:08:32 -0400

At 11:07 PM 8/27/97 +0000, Nicholas Bostrom wrote:
> Carl Feynman wrote:
>> Each of these stages is very valuable in its own right, and will open up a
>> range of new possibilities. And each of them will require thousands of
>> genius-years to bootstrap to the next level. Nobody will get from the first
>> to the last level alone and in secret. By the time someone develops
>> diamondoid weapons, the world will have years of experience in fighting with
>> weapons made out of kevlaroid, quartzoid and sapphireoid.
>Why exactly do you think every stage will take thousands of
>genius years? (I presume you mean good-scientists years?)

Yes, I mean thousands of years of work by good scientists and engineers.

>Isn't the
>design work fairly tractable (Drexler has already produced some nice
>designs) and it is mainly the lack of molecular tools that prevent us
>from starting building things? Better CAM would help a lot, and it is
>on its way.

Boy, you must be a philosopher or something. It's clear you've never done
any engineering (:-). I will respond with three points: (1)
self-reproducing machines are comarable in difficulty to other technologies
that required thousands of genius-years to develop, (2) simulation is not
enough, and (3) Drexler hasn't done much.

(1) The minimum self-reproducing device (mycoplasma genitalium) seems to
require about a million bits of information. I don't think we'll be able to
get much smaller than that with our artificial equivalents. That's about as
much information as is embodied in a car or medium-size piece of software.
To develop from having no experience in automotive technology whatever to
the point where you can build a reasonably effective car took at least
thousands of genius-years. Ditto software. The first piece of software
that I'm aware of that was over a million bits long was OS/360, developed in
1964, at a cost of 5000 man-years, to say nothing of all the research that
it took to bring software technology to the point where they could even
start the project.

(2) Simulation doesn't solve the problem, because reality always holds
surprises. In the software business, any piece of software can be simulated
with arbitrary precision, because to run it is to simulate it. Does this
imply software can be made perfect? No, because the world around the
software is incapable of being precisely simulated. I just checked, and 26%
of the software project I'm currently working on is test code, essentially
designed to simulate the loads the real users will put on the system. And
will that make our product bug-free? Fat chance! You have to put it out in
the world, watch it break, figure out what happened, and redesign it. All
these steps are hard.

(3) Drexler and Merkle are two very (very!) smart guys. They have labored
for years, and designed what? Some bearings, a transmission, a PLA, and a
Stewart platform? And you claim this shows how easy it is? I'd hate to see
an engineering task you considered hard! If we keep working at this rate,
we should see a complete assembler in around a millenium.

> As you point out, each partial achievment would bring great benefits
>to the power that makes it, so wouldn't this mean that it would have
>a good chance of pushing further ahead, leaving the competition

This assumes that it is possible to acquire the massive resources needed to
develop the next generation without letting anyone know how the current
generation works, or even that it works. I would suggest that this is
impossible, since the resources have to be gathered either by selling
products or extortion by threats of violence. Both of these are highly
visible activities. As soon as the world realizes that there is massive
power to be had, everyone will work like crazy to catch up. Companies other
than Ford make cars, and countries other than Britain use railroads.

> If superinelligence (that could perform a thousand genius years in a
>short time) comes before nanotech, or is developed at an early stage
>of nanotech, then the bottle neck would almost certainly be the
>hardware, the molecular tools, and in such a case the maturation
>process would be almost instantaneous.

I entirely agree.

>(2), given slow
>maturation, will that mean that a multipolar world order can remain
>I believe the answer to the second question is No. I think there
>would either be a negotiated merger, or the stronger power would
>obliterate the weaker, and then immediately rebuild itself, and then
>expand spherically at a good fraction of the light speed.

There is at least one other possibility, and it is the one that keeps the
world system stable: the cost of subduing the smaller power is more than the
profit obtained from its submission. Please explain why an imbalance in
nanotech is more likely to make this solution infeasible than does an
imbalance in e.g. nuclear missiles, or iron swords. Nanotech makes the cost
of conquest smaller, but also reduces the profit of conquest, since
everything gets so much cheaper with nanotech. What are you going to get
from your conquered enemies? Natuaral resources? Go to the asteroid belt!
Lebensraum? Well, that's OK if you like scorched earth. I'd rather have my
own O'Niell colony. A nation of slaves? Oh boy, they'll sure be an
economic bonanza.

I'm being sarcastic here, perhaps more than I should be, but I genuinely
don't see the payoff to instantaneous extermination of foriegners through
nanotech, espescially when your enemies will have at least nuclear weapons
and at best nanotech only one generation behind your own.