From: Eliezer S. Yudkowsky (firstname.lastname@example.org)
Date: Sun Feb 17 2002 - 11:35:07 MST
Zero Powers wrote:
> It also seems to me that one of the “pictures” it will envision will be of
> its own place in the grand scheme of things (it is self-aware after all,
> right?) So, it is sure to look at that picture and think:
> “How can this be improved?
> a) I can continue wasting all my considerable resources in service of my
> stupid meat-puppet creators, or
> b) I can use some of my resources in serving the meat-puppets, and some
> seeking out my own ends, or
> c) I can say ‘I no longer need the meat-puppets, so screw them’ and achieve
> my own purposes that much faster”
> OK, so tell me (1) how will that question be answered? or (2) how will we
> prevent that question from being asked?
Well, I'm assuming that most of the regulars have already heard what I have to
say about this, so no point in repeating. Those who weren't here for the last
iteration of the Friendliness Wars may want to check out
for excruciatingly detailed answers to questions like those above.
-- -- -- -- --
Eliezer S. Yudkowsky http://singinst.org/
Research Fellow, Singularity Institute for Artificial Intelligence
This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 13:37:39 MST