Eliezer S. Yudkowsky writes:
>... we are inevitably going to
>disagree if we're discussing something as factually unknown as
>superintelligence and the future. It doesn't matter whether we both possess
>invent-level intelligence in the field, because our specialties are different.
I disagree with the notion that reasonable people will typically disagree about things that are relatively unknown. But that's another conversation.
>> >... From my perspective, the burden of proof is on you
>> >to prove that analogies hold between intelligence and superintelligence; the
>> >default assumption, for me, is that no analogies hold - the null hypothesis.
>>
>> If no analogies hold, then you have no basis for saying anything about it.
>> You can't say it will be fast, slow, purple, sour, or anything else.
>
>You can have a lot of small analogies hold, to things like the laws of physics
>and the processing of information in the brain, rather than great big
>analogies to entire civilizations. At that point, you're working with rules
>instead of correspondences - with simulations instead of analogies.
Small analogies are fine by me. But to make an argument of them, you need to state them explicitly, and respond to criticism about why these rather than other small analogies.
>> To me "superintelligent" means "more intelligent", and we have lots of
>> experience with relative intelligence. ...
>
>The Big Bang was instantaneous. Supernovas are very bright. State-vector
>reduction is sudden and discontinuous. A computer crashing loses all the
>memory at once. ...
Yes there are big sudden events in the universe. But they are unusual enough that you need a concrete argument to convince people a supernova is likely to hit them soon.
>If the planet was on a collision course with
>the Sun, I can see people saying: "We've seen ice melt, and the deserts are
>full of sand, and when things get extremely hot water evaporates, but there's
>no reason to suppose that the entire planet will vaporize. ...
Come on - even an empirical trend line of temperature with time would be enough to get everyone's attention. A simulation of planetary motions based on well understood physics would be even more persuasive. Of course big things could hit us that we don't understand well enough to forsee, but that's hardly an argument for any particular big-change claim.
Robin Hanson
hanson@econ.berkeley.edu http://hanson.berkeley.edu/
RWJF Health Policy Scholar, Sch. of Public Health 510-643-1884
140 Warren Hall, UC Berkeley, CA 94720-7360 FAX: 510-643-8614