[In this post, I wear my participant hat. RH]
Vinge has mostly satisfied me that he does not literally mean that post-singularity things are "unknowable," though he for some reason still likes to use that word. It is hard to object much to his refined descriptor, "qualitatively less understandable," and I grant the possibility of "styles of cognition possible to higher technologies that are radically different from ours." Vinge even acknowledges that future self-aware modules may not be much larger than we are.
Given this, let me focus on what I think is now the central issue: timing.
I think that most thoughtful people will acknowledge that it is hard to place limits on what our descendants will accomplish if substantial numbers of them survive for another billion years, and continue to attempt to improve their technology over this time. This implies both that super-human artificial intelligences (AIs) may well eventually appear, and that we now may find it very difficult to envision what they and their world will evolve into.
In contrast, Vinge's "singularity" concept seems to me to claim much more, so much more that it becomes a radical claim few people endorse. It claims that these super-human AIs will appear within the next few decades, and then within a few years transform themselves and their world into something very difficult for us to imagine. The essential difference between this view and the widely acceptable view I describe above is one of timing. What most people acknowledge to be possible over a billion years, Vinge suggests is likely to transpire within just a few years.
Now Vinge does acknowledge the possibility of a slower transition, and this does serve to signal that Vinge is a reasonable person. But it does not substitute for an analysis in favor of his radical claim.
To me, what Nick calls "Verticality" is just a plausible side effect of
the creation of superintelligence. This is mostly by analogy with past
progress:
o Before humankind, the kingdom of life evolved by natural
selection -- a kind of "simulation" which proceeded as one with
the real world.
o Humans have the ability to run simulations internally (far more
so than other animals). Adaptation is several orders of magnitude
faster than in the previous phase.
o We humans now are developing devices which can run simulations
faster than our internal, biological "hardware" can do. I think
it's plausible that the accompanying speedup will have the
appearance of "Verticality" over the human phase.
Vinge seems to be just rephrasing this paragraph from his original paper:
... When greater-than-human
intelligence drives progress, that progress will be much more rapid.
In fact, there seems no reason why progress itself would not involve
the creation of still more intelligent entities -- on a still-shorter
time scale. The best analogy that I see is with the evolutionary past:
Animals can adapt to problems and make inventions, but often no faster
than natural selection can do its work -- the world acts as its own
simulator in the case of natural selection. We humans have the ability
to internalize the world and conduct "what if's" in our heads; we can
solve many problems thousands of times faster than natural selection.
Now, by creating the means to execute those simulations at much higher
speeds, we are entering a regime as radically different from our human
past as we humans are from the lower animals.
Before this discussion can proceed further, I think we need to get clear
on what exactly Vinge is claiming in these two passages. I'd be most
interested in how others translate these, but my reading is:
"Progress" rates increase with the speed of the processors involved.
Now it's not clear what "progress" metrics are valid here, but if
economists' usual measures are valid, such as growth rate in world
product, there are two immediate problems with this theory:
the processors involved.
(Furthermore, even if we accept this paraphrased claim, it is not clear that we must accept Vinge's timing claims. For this, I think we'd need do say more about just how rapidly progress increases with processor speed.)
Robin Hanson
hanson@econ.berkeley.edu http://hanson.berkeley.edu/
RWJF Health Policy Scholar, Sch. of Public Health 510-643-1884
140 Warren Hall, UC Berkeley, CA 94720-7360 FAX: 510-643-8614