From: Damien Broderick (damienb@unimelb.edu.au)
Date: Mon Mar 17 2003 - 18:39:56 MST
At 10:13 AM 3/17/03 -0800, Adrian Tymes wrote:
>> (I just turned down a chance to talk about the risks
>> of rogue AI on a TV
>> chat show in Oz with Paul Davies and some other
>> scientists. I feared it
>> would hopelessly trivialize the issue.)
>Logic error: they'll hopelessly trivialize the issue
>with or without your participation. By participating,
>you could at least affect the outcome the way you
>wanted.
It's not an issue of logic but of pragmatics. In fact, it's quite likely
that my reputation as a serious commentator on such topics would be damaged
by a clumsy response to the brilliant and cruel jester running the show. As
it turned out, last night's premier ep indicated that he's reining in his
sarcasm, so the risk might yet turn out to be worth it.
But I should have elaborated on the format of what I declined to
participate in: it was pitched as a brief (5-8 minute?) competitive panel
of `experts' on varieties of global doom: nano goo, superbugs, space
debris, mad AI. I instantly summarized the pitch as `You call *that* a
knife?' and the producer enthusiastically agreed. I pointed out that for
me, the most unexpected kind of `disaster' likely to follow from a super AI
was weirdly two-sided: that in fact, the outcome might be heaven on earth,
but that people would find the prospect of such a happy result terrifying
and ugly since it would mean not being human any longer.
This was all too difficult to convey in a couple of minutes, of course.
Clearly I was meant to say that mad Terminators would wreck the world in
their blinding artificial stupidity. That is actually quite plausible, but
there's no leverage in saying so to several 100,000 humans, and even less
in trying to persuade them that transcension under a benevolent Super AI
will be the bee's knees and devoutly to be wished.
Damien Broderick
This archive was generated by hypermail 2.1.5 : Mon Mar 17 2003 - 18:44:19 MST