Re: AI big wins

Eliezer S. Yudkowsky (
Thu, 24 Sep 1998 19:41:07 -0500

Robin Hanson wrote:
> "I think a more realistic model is" is not a sufficiently detailed argument.
> for t' = a*e^(b*(t+c)). And even accepting this form, substantial growth
> could still take centuries, depending on the values of a,b,c.
> Your argument here seems awfully close to the claim that "the doubling time
> of computer hardware efficiency is proportional to the computer operations
> per second devoted to R&D in computer hardware, or within all of computer-aided
> `humanity.'"

If you mean with the t' = e^t, I agree absolutely that it's just as worthless as the Argument from Moore's Law; it's a simple analogy, having none of the reductionistic detail that's needed for a realistic discussion of AI trajectories. As you recall, my actual analysis clamped power at an arbitrary value and treated strictly with optimization and intelligence. Obviously, I don't think that Moore's Law, or for that matter any simple equation for growth rates, is of real value.

Can we return to the ABC/POI discussion now? Focus on the technical discussion, not the immoderate speculations <grin>. But seriously, what about the feedback into the system, which is my main argument?

> If you recall, this was my summary of the assumption behind an
> analysis from a web page of yours, and I gave specific empirical criticisms
> of it. You have not yet responded to them.

Sure I did. I responded by saying that I renounced, repented, and abjured the substance if not the spirit - that the whole Argument from Moore's Law was fundamentally worthless - which didn't seem to be exactly what you wanted. Sigh.

> There really is a rich economic growth literature on when various equations
> like this describe different growing systems, including intelligent systems.
> Growth depends on many factors, and just because a previously fixed factor is
> allowed to grow, that doesn't mean growth suddenly explodes.

"Intelligence is not a factor, it is the equation itself." You've never responded to my basic assertion, which is that sufficient intelligence (which is probably achievable) suffices for nanotech; which in turn suffices to turn the planet into a computer; which in turn counts as "explosive growth" by my standards. It's difficult to see how the literature on the rise of agriculture relates...

"Sufficient" = Wili Wachendon with a headband.
"Achievable" = The end of my seed AI's trajectory, running on 10^13 ops.
"Nanotech"   = What Drexler said in _Engines of Creation_.

--         Eliezer S. Yudkowsky

Disclaimer: Unless otherwise specified, I'm not telling you everything I think I know.