Re: System Security

Eugene Leitl (eugene.leitl@lrz.uni-muenchen.de)
Fri, 20 Nov 1998 20:42:11 +0100

Anders Sandberg writes:
> "T.R. Fullhart" <trf2020@ksu.edu> writes:
> > These
> > sound like natural phenomena to me. The have proven to be valuable resources
> > for change and survival in nature. Why should computers be any different?

Thank you for supporting my point. Before one can embark upon enhancing a system, one obviously should have a deep level of understanding about its modus operandi. Human designers can't create systems nonbrittle in respect to modifications, robust, self-healing or gracefully failing, very complex, autohomeostating &c&c systems. It's time we learn the trick. Hubris and ignorance don't mix well, at least not on the long run.

> > I suggest that we follow a very Taoist attitude and instead of trying to
> > fight problems, we adopt a more observational attitude and apply techniques
> > that have worked in natural systems to systems that we develope. If it
> > worked for mother nature, I'm sure it will work for us. Let us not reinvent
> > the wheel.
>
> I agree with you, but also remember that nature didn't invent the
> wheel. Some solutions are distinctly non-natural but work well
> too. One shouldn't be dogmatic about one's non-dogmaticisim :-)

I disagree. I think at the heart of the malaise is the human hubris of believing we came up with the best metadesign scheme ever and pushing it onward despite of an impressive track of unfavourable evidence. Structures break, systems crash, people die. For instance, from my anecdotal experience, system design as software engineering is one of the most irrational human engineering endeavours. (How many unmet-schedule-all-nighter-pulling developers with a strong maestro streak do you know?) So far we've been at some level adaptive to systemic damage and occasional casualties, fixing the most glaring holes -- but shielded from the worst of it that failures didn't autoamplify as of yet, and by the fact that (human-designed) brittleware doesn't evolve well. But this doesn't mean we will be immune forever. If our designs are really distinctly suboptimal, and someone comes up with a substrate less brittle in respect to evolutionary optimization, or allows the mutation function to mutate as well we might be in for an eye-opening experience. A smart enough mutation function can do wonders to the field of GP. With GP, you won't have provably secure practical systems (which are as real as unicorns), but at least they will be robust.

Of course, this doesn't mean nature's ways are the best. We can do better than nature, and indeed we will one day. But please don't ignore tricks and metatricks nature came up with. It will be difficult, particularly considering analysis of evolved fit, but opaque solutions and creation of hybrid systems, but it should be well worthwhile.

Oh, and of course nature did invent the wheel. The flagella rotor bacteria use for propulsion is of course a wheel, and driven by a nearly 100% efficient nanomotor at that. The reasons why macroscopic organisms don't use wheels are more difficult. Too large valley between two local optima peaks (rotor tissue invasculation trouble? rotors can pump as well) might be one thing, unsuitability of wheeled animals in natural rough terrain habitats might be another.

ciao,
'gene