Re: Goo prophylaxis (was: Hanson antiproliferation method?)

Hal Finney (
Mon, 25 Aug 1997 17:23:58 -0700

Eliezer Yudkowsky writes:
> I want to repeat this, because it's important. Our immune systems are the
> closest analogue to proposed nano-immunities. The mismatch in available power
> and sophistication is enormous. Our immune systems learn from experience, use
> controlled and directed evolution, have memories... everything but the ability
> to consciously design things. And yet the viruses waltz casually through our
> bodies, because it's so much easier to destroy than create. Sometimes a virus
> which has deliberately evolved not to kill the host will be expelled from the
> body after a few weeks. Lethal viruses who've taken the gloves off thrive for
> years under continuous assualt by our immune systems and the designed
> pharmaceuticals of science. And really lethal viruses, the truly destructive
> ones, kill us directly.

This ignores one thing: viruses do not destroy, they create. A virus is
primarily interested in reproduction, not destruction. Any destruction
which occurs is largely a side effect, bursting cells to free the newborn
virus particles. This does play some role in helping the spread of
the virus, but probably it would be even better for the virus if the
converted cells were able to keep cranking out new virii indefinately,
and there were some mechanism to get them out of the cell. In some cases
it is the body's own immune system which destroys virus-producing cells,
not the virus itself.

The creation-vs-destruction model is a misleading picture of what happens
in viral infections.

Applying this to the gray goo scenario, the thing to remember is that the
gray goo must replicate in order to be effective. A single gray goo
disassembler will not cause much damage. And whatever mechanisms it uses
to replicate will be vulnerable to attack just as much as the systems which
it is trying to "eat".

In nature, we have a distinction between predators and prey. Large numbers
of animals have pretty much given up on trying to defend themselves,
relying instead on avoiding predation, and fast enough reproduction to
overcome the loss due to predators. This leads to a model where we think
of gray goo replicators as the predators, and ordinary nanotech as prey.

But this is not the right model. Think instead of interactions among
predators, each seeking to be as aggressive as possible against the
others. Think of lions attacking bears, badgers against wolverines.
Think of armies against armies, or spy vs spy.

Barring nuclear destruction, it is not clear that gray goo will win
the battle. Gray goo is not interested solely in destruction. Rather,
it is a replicator like everyone else; it seeks to preserve its own
structure and function, it seeks to reproduce, it seeks to protect itself.
It must do these things in order to survive.

My prediction would be a band of "war zones", where the battle rages,
with surges as one side or the other gets a local advantage. Between
these zones would be relatively stable regions, dominated by cooperating
replicators. But the border shifts, and occasionally a stable region
is overcome.

Each side seeks to destroy the other, but that is an over-simplification.
Actually there are many factions, some seeking to destroy all others,
others cooperating. Can game theory give us cause to believe that the
nice (in the technical sense) replicators will win? Will the hackers
and madmen cooperate in their efforts to destroy the world, or isn't it
more likely that each attempts to destroy all others? As the good-guy
nanotech cooperates while bad-guy factions kill each other, it may well
be that we can expect the good guys to have an advantage.