Re: Why Does Self-Discovery Require a Journey?

From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Sun Jul 13 2003 - 19:42:52 MDT

  • Next message: Mike Lorrey: "RE: Number of carbon atoms in the Earth's biomass"

    Dan Fabulich wrote:
    > Robin Hanson wrote:
    >>
    >>A happiness metric does not require that people be informed about the
    >>consequences of their choice.
    >
    > Of course not. Perhaps I'm misunderstanding your argument, but I thought
    > the claim was: "We can think of two ways of identifying what you'd
    > 'really' want. First, we can see if you'd be happier if you got it.
    > Second, we can see if you'd choose to get it when they are briefly and
    > privately informed. In the case of betraying the tribe, the person would
    > do this if briefly and privately informed. Therefore, this is what they
    > really want."

    Actually, I strongly disagree with "what you do after being briefly and
    privately informed of evidence sufficient to convince a neutral observer"
    as a metric of what people "really want". So what if the evidence is
    strong enough to convince a neutral observer? If the person doesn't
    believe the evidence upon being "briefly informed", their volition (the
    cognitive system constituting their volition) is just the same as it was
    before. Any gaps or problem areas in their volition will still be there.
      A warning label that someone ignores doesn't help, doesn't change
    anything, and is not a metric of what people really want. Extrapolating a
    major change in someone's volition is a major problem, and "briefly
    informing" someone doesn't get to grips with that problem at all. There
    is no way to "briefly" inform a Christian that there is no life after
    death and ask if they would "really want" cryonics. It may seem *fair* to
    give someone enough evidence to convince a neutral observer, and it is
    certainly very inconvenient to have to jump through more hoops than that.
      But so what? What does fairness to the experimenter or convenience for
    the experimenter have to do with it? In reality it is just an unfair and
    inconvenient and complicated problem.

    Furthermore, I disagree with the premise that people would betray the
    tribe if privately and successfully informed that this was, in fact, the
    dilemma they were faced with. Who says that this is what all people or
    most people would do? I should expect being successfully informed to
    bring moral principles and retention of self-respect into play as the
    primary components of the decision. Can you name anyone who was
    successfully informed... by studying evolutionary psychology over a period
    of years, say... who is *not* now trying to cut the puppet strings? It's
    a highly biased sample when you consider the kind of personality that
    would stick to studying the issue in the first place, but still: people
    who *are* successfully informed of the puppet strings generally seem to
    want to cut them. Sample bias, availability bias, consensus bias, but
    it's still how things look offhand.

    -- 
    Eliezer S. Yudkowsky                          http://singinst.org/
    Research Fellow, Singularity Institute for Artificial Intelligence
    


    This archive was generated by hypermail 2.1.5 : Sun Jul 13 2003 - 19:54:00 MDT