From: nanowave (nanowave@shaw.ca)
Date: Tue Aug 12 2003 - 23:42:13 MDT
nanowave writes:
>> Yes, the possibility of unwittingly unleashing a kind of "berserker" upon
>> the earth would seem to be not statistically insignificant, [snip]
Robert writes:
>if it is intelligent enough to accumulate resources and
>transport itself it is *REALLY* dangerous (and that does *not* require much
>intelligence).
agreed.
>That is why my hat is off to Eliezer for pointing out the problem of an
>unfriendly AI. (Take SARS and multiply it by many orders of magnitude and
>one gets the hazard level of an unfriendly AI).
>But one doesn't have to deal with only unfriendly AIs. It appears
>that unfriendly viruses have this capability (look at influenza epidemics).
>> Is it completely unreasonable to presume that 'human-level' intelligence
>> implies human-level ethics and values (two kinds of social intelligence)
>> as well as pure knowledge crunching/combinatorial power?
>
>Perhaps. Our ethics/values have been dictated by survival requirements
>(better to have friends than enemies). A novel AI need not have such
>restrictions. (Just program a berserker -- screw the consequences.)
Ok, but such a berserker would presumably be the first auto-evolutionary AI
created by humans if it were to have a chance of fulfilling its dark
purpose. Where technological evolution tends to follow double-exponential
progress curves (kurzweil), it would seem unlikely that a malevolent
AI(2),(5), or (7) could ever technologically overtake (threaten) an evolving
AI(1).
Ok, it's assumed here AI(1) turns out to be friendly or at any rate
loves/esteems/respects its progenitors sufficiently to dedicate minute
resources to protecting us from subsequent rogue AI systems and accidents.
In respect to the rise of human ethics and values, I think it might be more
precise to say they've been 'influenced by survival requirements' rather
than "dictated by survival requirements" since we are presumably talking
more about memetic evolution than legacy genetic.
Russell Evermore
This archive was generated by hypermail 2.1.5 : Tue Aug 12 2003 - 23:54:52 MDT