Re: Goo prophylaxis

Nicholas Bostrom (
Fri, 29 Aug 1997 17:27:42 +0000


Eliezer S. Yudkowsky wrote:

> Anders Sandberg wrote:
> > I never said I would sketch an
> > *invulnerable* system, just a sufficiently strong system. For any immense
> > shielding I can come up with you could always invoke an even greater
> > cosmological disaster ("But your immune system can't stand a supernova!").
> > This exercise is trying to look at defenses against gray goo, which
> > is the real problem, not macro-level warfare.
> But you *do* need an invulnerable system, or at least one that is invulnerable
> to goo. I can reasonably invoke any forces modern technology is capable of
> wielding, up to and including nuclear weapons. If black goo reduces your city
> to radioactive ash, you lose! It doesn't matter how sophisticated your
> defenses are! I am darn well entitled to demand that your immune system stand
> up against nuclear weapons, because in practice, that's what going to be used!

That's right.

> Not impossible. Not at all impossible. If you have a layered defense system,
> a diamond shell, and Fog seat belts, the city might be perfectly capable of
> withstanding a nuclear attack. It would lose a layer of defense, but might
> well be capable of rebuilding it before more black goo crossed the radioactive
> zone. In addition, as I pointed out, the city might gain more than it lost.

Does this mean that yoo are no longer confident in you "destruction
by induction" argument?

> Even so, it's entirely possible that, on any planet, the black goo wins. All
> the time. Every time. The Universe is under no obligation to make things
> easy for us. Hence the proposed Directive of Evacuation: "Get everyone off
> the Earth, into partitioned space colonies, before releasing nanotechnology to
> the world."

Yes, it is remarable how many people are stuck with the idea that an
effective defense *must* be possible. That's blind faith. And even if
effective defense is possible, there is still the genesis problem;
but by another leap of faith, that problem *must* be soluble too,
even within an unregulated mulipolar world order. I suspect there is
some ideological prejudice at the bottom of this.

> My mental picture of these conflicts is partially drawn from Conway's Game of
> Life, in which a single particle can destroy an enormous, complex structure.
> Things on the molecular level will almost certainly be different. Even so, I
> know of no better image.

An interesting way of looking at it.

> Try this tactic. First, the goo hits the city with a vaporize-one-layer
> attack, even though this also vaporizes a layer of goo. Then it detonates a
> nuclear weapon next to the city. This pushes the city into the goo,

Do you literally mean "push"? That would seem like fantasy.

> The gray goo problem is overrated; it's black goo we need to worry about.

I agree. As I think is clear from my postings, goo designed to
destruct is what I meant by grey goo, but from now on I will call it
black goo.

> > You assume that the destroyed cell will no
> > longer be a problem. But what if it turned into a cube of inert diamondoid?
> > Then it would also be a hinder, and give me even more time to develop
> > countermeasures.
> If it is inert diamondoid, is that more of a hindrance than the outer shell?
> Besides, this whole discussion is looking obsolete anyway. Any rigid outer
> defense is toast. It has to be surrounded by explosive-equipped soft 'mune.
> At most, you can have a rigid outer defense as a pretty shell.

I agree.

> > Note that they would be produced faster than the goo since the goo would
> > have to both reproduce, breach security and defend itself while the
> > antibodies and macrophages would just be produced (although a goo-like
> > macrophage is an interesting and dangerous concept) and sent on their way.

>if the goo has so much as a cubic millimeter to
> call its own, it can keep its queen reproducers in the center while
> surrounding itself with warrior goo, just like the city.
> Why can't the goo use antibodies and macrophages against the 'mune?

> The tactical symmetry remains unbroken.

Soo it seems.

> > > The goo simply makes repeated
> > > attacks, and each time, the city shrinks a little. We won't even speak of
> > > such horrors as cutting off that city's solar power.
> >
> > An isolated immune system in a world where nobody else has immune systems
> > is weaker than a system in a world where nanodefenses are common. The ultimate
> > defense would of course be to have immune systems everywhere, defending
> > not just the transhumans but the himalayas, squirrels and grass. A bit like
> > the current state in biology, really.
> If nanotechnology was released on Earth, the 'munes would have to be
> EVERYWHERE. The air. The water. Earth's molten core. Otherwise, the goo
> could gain a foothold.

Right. An infallible global immune system would be reguired. A local
immune system wouldn't stand a chance against black goo.

> > > Nanosystems are always faced with "destruction by induction". That is, you
> > > can always destroy one cell; therefore you can destroy the whole thing. To
> > > defend against this, it is required that the system expand faster than the
> > > destruction OR that it be impossible to destroy one cell.
> >
> > Or that you can make the loss of a finite number of cells bearable. If their
> > loss removes the threat (for example by forming a nanoscar), then the
> > defense side will win.
> I disagree with the whole concept of a nanoscar. If there is such a thing,
> you turn it into your first line of defense, not wait until after you've been invaded.


Nicholas Bostrom
London School of Economics
Department of Philosphy, Logic and Scientifc Method