Darin Sunley wrote:
>
> From: hal@finney.org <hal@finney.org>
> >
> >Believers in the omnipotence of AIs seem to think that for any given
> >person, in any given situation, there is some input they can be given
> >which will cause them to produce any desired output. If I see a nun
> >in the middle of prayer in the chapel, there is something I can say to
> >her that will make her immediately start screeching like a chimp while
> >jumping around the room scratching herself.
>
> You know, given a trillion high fidelity simulations of that nun to test
> possible responses, I bet I could construct a sentence that would do just
> that. My first order approximation is that it would involve me claiming to
> be sent from God, combined with a thourough demonstration of omniscience
> with respect to her life up to that point, all of which is easily acheivable
> given those simulations, and an arbitrary amount subjective time to think
> about it.
I was going to say exactly that. Congratulations, Sunley, you have restored
my faith in the Extropians list.
I would have added that, in addition to backing up the initial action
specification with a high motivational priority - which can come from
pretending to be God, or offering a million dollars, or any number of other
methods - you need some way to disrupt the binding of the emotional tone of
embarassment to the modeled projection of the reactions of other folks around
the room. There are any number of ways to do this; you can shift the focus of
attention; invoke an interfering emotional tone; convince the nun that the
other people in the room are *expecting* her to do that, or that the other
people don't count for any number of reasons. Or just give a sufficiently
high-priority I-am-God order that it overrides the projected embarassment by
brute force.
-- -- -- -- --
Eliezer S. Yudkowsky http://singinst.org/
Research Fellow, Singularity Institute for Artificial Intelligence
This archive was generated by hypermail 2b29 : Mon Oct 02 2000 - 17:38:49 MDT