Re: Moving vs. Copying (was Status of Superrationality)

From: Rafal Smigrodzki (rafal@smigrodzki.org)
Date: Sat May 31 2003 - 11:01:57 MDT

  • Next message: Rafal Smigrodzki: "Re: Boy Genius or Craft Idiot?"

    ----- Original Message -----
    From: "Lee Corbin" <lcorbin@tsoft.com>
    To: <extropians@extropy.org>
    Sent: Saturday, May 31, 2003 12:46 AM
    Subject: Moving vs. Copying (was Status of Superrationality)

    > Eliezer writes
    >
    > > > ### But isn't it the same as the totalist position?
    > >
    > > The totalist position says it is better to copy than to move.
    >
    > Whereas I think that maybe we discussed this before, I still
    > cannot understand why you would rather move than copy. In
    > order to perhaps gain some insight, let me formulate a thought
    > experiment to find out your feelings on the matter.
    >
    > Suppose that on one bright and cheery morning, Eliezer is having
    > an extremely good time. This is mainly (let us say) due to
    > a commendable excess of serotonin operating in his CNS, but
    > may also be due to having had a number of good breaks lately.
    > Let us further suppose that there are two delightful problems
    > to work on, A and B, both of which promise nice advances in
    > the theory of AI friendliness.
    >
    > Now Eliezer finds that A is slightly more fun and perhaps
    > slightly more promising than B, but so delightful and promising
    > are both choices that he hesitates in choosing. Now to work
    > on A, however, involves going to another office in another
    > building.
    >
    > Are we to assume that you would copy---instead of move---but
    > do so only because you are making a sacrifice for the sake of
    > humanity? Namely, by getting to work on both A and B, you
    > of course advance the cause of the Singularity (thus alleviating
    > much pain and suffering in the world), but at some cost to
    > yourself. The cost to yourself---if I have understood you
    > correctly---is that it would be better for you to have moved
    > rather than copied.
    >
    > To place myself in a parallel situation, suppose that Lee has
    > two hours free at some particular time T in which to watch a
    > great movie. Even better, there are two movies A and B that
    > would be almost equally pleasurable to watch. Well, what I
    > would do, of course, (being a totalist) is make a copy of
    > myself for the interval of time (T, T+2hrs) and watch both
    > movies. (We are not talking about any events subsequent to
    > T+2hrs, and I hope that no one is distracted by such considerations.)
    > Why shouldn't I get to watch both movies, and enjoy both, and
    > why isn't this the best course of action for one?
    >
    ### You are pointing to a very important feature of human motivation
    system - non-linearity.

    Both the averagist and totalist positions are crude approximations of our
    largely inborn motivational hardware. In real life we have a very strong
    negative response to a set of evolutionarily specified stimuli, or a certain
    level of resources available to us - insufficient nutrition, physical pain,
    and other signals of near-term danger are weighted much more heavily in our
    decisions than time-discounted signals of danger, or, on the other hand,
    signals of available opportunity. Our avoidance system is *not* the same as
    our pursuit system (neurologically exemplified by, but not limited to, the
    amygdala vs. nucleus accumbens). This is why most of us will not leave our
    copy in Hell (averagism), but would consider leaving a copy in a second-rate
    heaven (totalism), especially if the copy still has the chance to improve
    its lot. Since there are interindividual variations in empathy, and
    time-discounting of rewards, there will be many versions of ethics, such as
    the threaded ethics, and hardly any pure totalists or pure averagists.

    I am a non-linear totalist-volitionist, BTW.

    Rafal



    This archive was generated by hypermail 2.1.5 : Sat May 31 2003 - 11:14:25 MDT