Re: Hyper-AI's vs Transhumans

Anders Sandberg (asa@nada.kth.se)
08 May 1998 16:57:43 +0200


Dan@Clemmensen.ShireNet.com (Dan Clemmensen) writes:

> It might be nice if some of the participants from the prior round of
> the ">Web" discussions would add any new thought they may have had
> on this matter.

While I agree with you about that human-computer collaborations are
the most likely path towards SI, I'm not entirely sure about the
supercooled liquid analogy.

Overall I am very sceptical of all claims that a single breakthrough
will produce a sudden singularity-like bootstrapping process that
leaves everybody but the initiator in the dust. There have been no
such events in the past, and even if we discuss intelligence
amplification techniques (which we so far have not had much experience
with) it is not clear that a given IA invention will make it much
easier to make even stronger IA devices.

And even if the device helps you make better such devices it might
turn out that this is limited by other constraints; for example the
human-software design system might quickly bootstrap to a level where
the user interface becomes the bottleneck, and further expansion
requires intelligence amplification in the service of the
human-computer interaction field instead. Most technologies seem to be
dependent on other technologies to reach their true potential, and
improving these might require even more work further afield.

It seems to me that a reall SI bootstraping process requires a width
of knowledge and skill that is not available in a small
group. Remember the car analogy: there is no single human able to
build a car from scratch, since that process involves mining
technology, refining, aerodynamics, metalworking, semiconductor design
and engineering, electronics, chemistry... and so on - but the
industrial web contains enough knowledge distributed between thousands
of people to do it. To make a true SI, we might need software
engineering, HCI, hardware design, economics, theory of complex
systems, psychology, neuroscience, nanotechnology and Turing knows
what else. And we might need a lot of it, not just a single expert for
every field hired by the would-be-SI team, but a whole community
testing out different solutions and working in parallel.

With IA technologies and an emerging >Web, this could become a
historically very fast process, but it would likely still take a few
years (practical physical delays in building new hardware, costly
mistakes, the problem that people tend to sleep 1/3 of the time
etc). I don't buy the idea that a single technology will bootstrap its
lone inventor to godhood the day after he discovers it (makes a great
sf story though; I recall writing something like that in the third
grade, where a group of children bootstrap themselves to world
domination in roughly the >Web way), but that doesn't mean I don't
think the >Web concept has merit. Seen from a distance in time and
space, it might very well look like a small fraction of humanity
transcends almost immediately, but close up it will be a wide movement
involving plenty of people (definitely more than a few thousand) and
work.

Perhaps a better discussion would be how we *practically* immanentize
the transcension? :-)

-- 
-----------------------------------------------------------------------
Anders Sandberg                                      Towards Ascension!
asa@nada.kth.se                            http://www.nada.kth.se/~asa/
GCS/M/S/O d++ -p+ c++++ !l u+ e++ m++ s+/+ n--- h+/* f+ g+ w++ t+ r+ !y