From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Sun Jul 13 2003 - 19:42:52 MDT
Dan Fabulich wrote:
> Robin Hanson wrote:
>>
>>A happiness metric does not require that people be informed about the
>>consequences of their choice.
>
> Of course not. Perhaps I'm misunderstanding your argument, but I thought
> the claim was: "We can think of two ways of identifying what you'd
> 'really' want. First, we can see if you'd be happier if you got it.
> Second, we can see if you'd choose to get it when they are briefly and
> privately informed. In the case of betraying the tribe, the person would
> do this if briefly and privately informed. Therefore, this is what they
> really want."
Actually, I strongly disagree with "what you do after being briefly and
privately informed of evidence sufficient to convince a neutral observer"
as a metric of what people "really want". So what if the evidence is
strong enough to convince a neutral observer? If the person doesn't
believe the evidence upon being "briefly informed", their volition (the
cognitive system constituting their volition) is just the same as it was
before. Any gaps or problem areas in their volition will still be there.
A warning label that someone ignores doesn't help, doesn't change
anything, and is not a metric of what people really want. Extrapolating a
major change in someone's volition is a major problem, and "briefly
informing" someone doesn't get to grips with that problem at all. There
is no way to "briefly" inform a Christian that there is no life after
death and ask if they would "really want" cryonics. It may seem *fair* to
give someone enough evidence to convince a neutral observer, and it is
certainly very inconvenient to have to jump through more hoops than that.
But so what? What does fairness to the experimenter or convenience for
the experimenter have to do with it? In reality it is just an unfair and
inconvenient and complicated problem.
Furthermore, I disagree with the premise that people would betray the
tribe if privately and successfully informed that this was, in fact, the
dilemma they were faced with. Who says that this is what all people or
most people would do? I should expect being successfully informed to
bring moral principles and retention of self-respect into play as the
primary components of the decision. Can you name anyone who was
successfully informed... by studying evolutionary psychology over a period
of years, say... who is *not* now trying to cut the puppet strings? It's
a highly biased sample when you consider the kind of personality that
would stick to studying the issue in the first place, but still: people
who *are* successfully informed of the puppet strings generally seem to
want to cut them. Sample bias, availability bias, consensus bias, but
it's still how things look offhand.
-- Eliezer S. Yudkowsky http://singinst.org/ Research Fellow, Singularity Institute for Artificial Intelligence
This archive was generated by hypermail 2.1.5 : Sun Jul 13 2003 - 19:54:00 MDT