I don't have time to read the Geniebusters site; I just swooped through chapters 18 and 19, since AI is my area of primary expertise.
In chapter 18, Lyle states:
"When I say, "When will we have..." I am looking for answers that take
into account the realities of writing software. What would be involved
in writing a program that serves as a coder? How would that compare with
writing Windows 2000? Would it be more complex? less? about the same? Is
anyone working on such a project?"
I refer him to "Coding a Transhuman AI".
With the background of "Coding a Transhuman AI", I strongly disagree
with chapter 19. Since an AI is "native" to the rich domain of code in
a way that an unenhanced human never can be, I would expect even a
"coder drone", incapable of acting as architect or foreman - although I
have my doubts about that paradigm - to be capable of outprogramming the
entire human race at present, and certainly of outprogramming an average
programmer dollar for dollar, no matter how expensive it was.
The vast majority of coding would occur easily, almost reflexively, and
thus very rapidly, since conscious thought is required only for a tiny
fraction. In this sense, such an AI could be said to be a coding
"genie". But even with domain-specific functions, it could be that
coding is such a general task that a "coding drone" would be (1)
independently motivated and (2) rapidly self-enhancing to the point of
Powerdom. I personally believe (1) and (2).
My objection to chapter 19, then, is that focusing on human-equivalent coding ability misses the point entirely. We should be asking about the effect of incredibly fast but blazingly stupid coding, or of entirely transhuman coding.
--
sentience@pobox.com Eliezer S. Yudkowsky
http://pobox.com/~sentience/AI_design.temp.html
http://pobox.com/~sentience/singul_arity.html
Disclaimer: Unless otherwise specified, I'm not telling you
everything I think I know.