RE: Challenge of Design Complexity

Robin Hanson (
Tue, 15 Dec 1998 10:03:54 -0800

Billy Brown wrote:
>> Similar issues have been considered. For example, here is an exerpt from
>Yes, I've read the Singularity debate. My argument is a rather different
>one that your comment about IQ enhancement, and it leads to different
>If you make an AI that increases its own IQ by 1%, that tells you nothing
>about how difficult the next 1% improvement will be. It could be 10%
>harder, or 0.5% harder, or not any harder at all. The only way to find out
>is to try it and see what happens.

This is what I was saying, for example, in saying "they assume this `make myself 1% smarter' task stays equally hard, as the engineer becomes smarter."

>Now, this does not mean that the whole thing is a game of diminishing
>returns. Recent history demonstrates that if you research enough different
>things, you can create a situation in which the ability of your society to
>make technological advances increases faster than the difficulty of taking
>the next step. A reasonable extrapolation of the trend would predict a
>century or two of steadily-accelerating progress before things begin to
>change so fast that an unenhanced human can't cope.

What is the "recent history" that shows a "trend" of "steadily-accelerating progress"? No such acceleration shows up in economic trends over the last half century.

>Intelligence enhancement (IE) of any kind would, however, add a new
>dimension to this saga. Roughly speaking, the our rate of progress is
>determined by:
> R * P * I
> Progress / unit of time = -----------
> T * C
>Where R represents the resources available to each researcher, P is the
>population of researchers, I is the average intelligence of the researchers,
>T is our current level of technological sophistication, and C is a measure
>of the time and effort required for researchers to communicate.

I think you mean T to be an inverse of tech sophistication.

You are assuming that progress comes mainly from researchers. This seriously undercounts contributions to progress from everyone else. And you are not allowing for the possibility that we solve the easier problems first, so that problems get harder with time.

>Most of the
>increasing rate of change in recent times comes from a slow geometric
>increase in both R and P, and a steady drop in C. Since the changes in R
>and C are both due to technology, the whole process tends to feed on itself.
>Meaningful IE would make I increase in roughly the same fashion as R. Not
>only would this dramatically speed up our rate of advance, it would also
>increase the rate at which our rate of advance speeds up.

You might see if these claims can be illustrated in the context of a specific mathematical model of these processes.

Robin Hanson   
RWJF Health Policy Scholar             FAX: 510-643-8614 
140 Warren Hall, UC Berkeley, CA 94720-7360 510-643-1884