Re: Goo prophylaxis

Hal Finney (hal@rain.org)
Tue, 2 Sep 1997 11:57:15 -0700


Most people are pretty reasonable, IMO. They're not perfect, but they're
not raving lunatics or frothing sociopaths. Most of them, if you gave
them the power to destroy the world, would refrain.

This is not true of everyone. If everyone had the power to destroy the
world, it would be destroyed. This is one of the things we fear about
widespread nanotech.

But initially it won't be widespread. Initially it will be controlled
only by one or a few groups. What will they do with it?

Are they likely to launch a pre-emptive strike to destroy the competition
and seal their own advantage? Will they create a goo which eats everything
in the world, turning it into resources under the control of the initiating
group?

I don't think these are likely scenarios. The game theory argument is
going to be unpersuasive to most people. Yes, by doing so they could
increase the odds of their future success, they could arguably put
themselves in a position to control the entire universe. However this
inducement will not be enough to make most people destroy innocent
lives all over the world, people whose only crime is to be potential
future competitors. This is not consistent with most people's ethical
sense.

It is perhaps more likely that one or more of the initial nano powers will
try to preemptively take protective action. Eliezer proposes dispersing
the entire population into space, cities in flight, where distance and
vacuum will hopefull provide a moat of security around each enclave.
Nicholas suggests involuntarily uploading everyone into a VR where the
laws of physics are such that they can't hurt each other.

I doubt that even these supposedly benevolant schemes will be attractive
to the first groups with nanotech. The engineering problems to be solved
are huge, so unless you believe in a strongly peaked Singularity, it is
doubtful that either of these ideas is practical. They also require an
unusual level of confidence and arrogance, reaching out to drastically
change the entire structure of human society. Here on this list we can
toy with ideas, but few people would be willing to implement these plans.

What I suspect will happen is less dramatic. I would expect the labs to
begin working almost immediately on countermeasures for gray goo and
similar nanotech based destructive devices. Unfortunately, this will
probably require the creation in the lab of destructive goo. The labs
will undoubtedly be built around a principle of defense in depth, to
prevent the escape of destructive nanotech.

Drexler talks about active shields, technologies which can defend against
harmful goo. We've had a lot of discussion here recently on how this
might work. It seems that one of the issues is which side gets an
initial advantage. Hopefully, by deploying anti-goo technologies ahead of
time, it will give people a fighting chance by the time the really nasty
nanotech attacks begin. This is Drexler's picture of initial nanotech
development from over ten years ago, and it still seems more plausible
to me than many of the extreme scenarios we've been discussing.

BTW, the recent issue of the Foresight newsletter (www.foresight.org)
suggests that they will be moving into a "phase 2" in terms of their
purpose and objectives. Up until now they've mostly been working to
convince people that nanotech was plausible and would come into existence
some time in the next few decades.

Now they feel that this has largely been accomplished, and they want
to begin working in earnest on what was always their ultimate goal.
They want to have the kinds of discussions we've been having here,
to talk about the dangers of nanotech, and what kinds of approaches
to nano would leave us with the best chance for success and survival.

Obviously this is difficult, on several levels; we may disagree on even
basic goals, with the extropians' ideal future being the rifkinites'
dystopia, and vice versa (as was pointed out here). I suspect that
a consensus would be for a much more moderate and controlled future,
with less access to technology, than we would like. Still I think
people deserve a voice in the future they are going to live in, and
they at least need to understand what the options and alternatives are.
It will be interesting to see how Foresight goes about its new task.

Hal