Eliezer S. Yudkowsky wrote:
> > No, I would literally nuke individual facilities, ...
>
>Are you so very, very sure that you know better than the people running the
>facilities? If the people with enough understanding and experience to have
>created an AI tell you firmly that they have devoted considerable time and
>effort to Friendliness and they believe the chances are as good as they'll
>ever get, would you really nuke them? ...
>And scientists and engineers are, by and large, benevolent - they may
>express that benevolence in unsafe ways, but I'm willing to trust to good
>intentions. After all, it's not like I have a choice.
A week earlier, Eliezer S. Yudkowsky wrote:
> > EarthWeb is more easily acchievable, by definition friendly to human
> > goals, and possibly sufficient for preventing runaway techno-disaster.
>
>On the contrary; the Earthweb is only as friendly as it is smart. I really
>don't see why the Earthweb would be benevolent. Benevolent some of the time,
>yes, but all of the time? The Earthweb is perfectly capable of making
>mistakes that are outright stupid, especially if it's an emotionally charged
>issue being considered for the first time.
These two quotes seem worth comparing. Eliezer distrusts the scenario of
EarthWeb becoming smarter and smarter, with more and more input from
advanced software as available, because people can be emotional. But he
thinks we should trust researchers creating a big bang AI, because
scientists are benevolent. This seems to me the Spock theory of who to
trust - let scientists run things because they aren't emotional, and don't
let markets run things, because they might let just any emotional
person have influence.
Of course if people realized that scientists knew better and should be
trusted, then non-scientists wouldn't bet against scientists in a market,
and scientists would effectively run an EarthWeb. But since people are
too stupid to know who their rightful betters are, Eliezer prefers to
impose on them the choice they wouldn't choose for themselves.
Robin Hanson rhanson@gmu.edu http://hanson.gmu.edu
Asst. Prof. Economics, George Mason University
MSN 1D3, Carow Hall, Fairfax VA 22030-4444
703-993-2326 FAX: 703-993-2323
This archive was generated by hypermail 2b29 : Mon Oct 02 2000 - 17:39:26 MDT