From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Mon Jul 14 2003 - 04:21:23 MDT
Dan Fabulich wrote:
>
> Having established that it's possible, there's the question of
> establishing the real answer: just how common *are* these contradiction
> cases? Eliezer and I can only provide prima facie back-of-the-envelope
> responses, e.g. Eliezer claims that he can't think of anybody who has been
> informed of their programming and hasn't tried to shake it off;
Yes.
> he also
> claims that most people who choose betrayal in an informed way turn out
> miserable from guilt.
Actually, I do not claim this. What I said was that most people who were
successfully informed of a previously unseen conflict between their
personal status and the good of the tribe would either choose the good of
the tribe, or feel guilty about not doing so. I make no claim that the
amount of this misery, experienced at the time of the choice, is greater
than the amount of total happiness they would derive from personal status,
nor about the duration of their initial misery, nor about long-run outcomes.
With respect to real modern people who *currently* choose betrayal,
consciously and non-self-deceptively, I have no idea how they turn out.
> Eliezer's arguments seem right to me, but actually
> testing this requires real research to which I can only give an
> "intelligent layman's" response.
Agreed.
-- Eliezer S. Yudkowsky http://singinst.org/ Research Fellow, Singularity Institute for Artificial Intelligence
This archive was generated by hypermail 2.1.5 : Mon Jul 14 2003 - 04:33:49 MDT