Anders Sandberg, <asa@nada.kth.se>, writes:
> I think I understand your point but I think you miss mine: making a IQ
> X AI requires effort E. Making an IQ 2X AI, does it require effort
> 0.5E, E, 2E or E^2? And will twice the intelligence double the
> programming ability? You seem to assume something like this, the
> ability times the available subjective time (large due to fast
> computers) will make the next step happen very quickly. But if
> development becomes quadratically harder, the AI, even when it devotes
> all its resources, will make slower and slower progress.
Organisms evolve many heuristics for pruning the tree, discarding unlikely branches. We see this very clearly in the studies of chess masters. There the tree search problem is made explicit, but the masters don't perceive the problem in quite that way. The pruning is, for them, subconscious. They see only a few possibilities at each branch, so that they are able to follow the tree very deeply.
It is plausible that a similar phenomenon occurs in all areas of life. We get the tree pre-pruned for us, so that at any moment we can make our choices unconsciously, or face only a few possible options. Then to evaluate those options we consider outcomes, possible further actions, and so on, always pruning away unfavorable branches using our heuristics.
Better intelligence can come from either faster tree search or better heuristics. But either of these is still going to be of limited value given the exponential nature of tree growth. A processor speedup of a factor of n allows you to look a fixed number of steps deeper. A new heuristic which prunes better by a factor of n allows you to look a factor of n deeper, but those are probably going to be much harder to come by, at least unless they are a lot more expensive.
The exponential growth of the possibility tree makes it hard for improvements to gain much ground. Looking twice as deep is going to take the square of the effort. The conservative assumption is that we will face increasing difficulty in improving intelligence.
This is consistent with our experience in industrial research. We keep putting more and more effort into it, but doubling times have remained pretty stable for 100 years or so. That's because we are facing increasingly harder problems.
Hal