Re: Status of Superrationality (was Left/Right... can't we do better than this?)

From: Hal Finney (hal@finney.org)
Date: Tue May 20 2003 - 12:30:14 MDT

  • Next message: Doug Thayer: "Re: "liberal media""

    A few quick comments:

    First, with regard to time travel and playing against your self from
    the past, it must not be the case that you remember playing against the
    future version of yourself, or else you would remember how the future
    you had played, which would produce a free will paradox. Therefore you
    must not remember the specifics of your past play. There should be some
    uncertainty about how your past self will behave, unless you were such
    as to always play the game in a completely predictable fashion.

    Second, an altruist could play against a selfist if the payoffs were
    set up something like this:

                            Selfist Cooperate Selfist Defect
                         --------------------------------------------------------
    Altruist Cooperate | Everyone in the world | Everyone in the world
                         | gains $5 | loses $10 except selfist,
                         | | who gains $10.
                         -----------------------------+---------------------------
    Altruist Defect | Everyone in the world | Status quo; no gain, no
                         | gains $10 except selfist, | loss
                         | who loses $10 |

    The question is whether the altruist would defect in this situation.

    I was trying to set up a PD scenario between two rational altruists,
    but it is hard since they are both motivated the same way, to increase
    the total happiness of the world. The only thing that could lead to
    a difference in payoff for a particular outcome is disagreement about
    how happy it would or does make people. For example, supposed Eliezer
    believed that average happiness were what mattered, while Lee believed
    that total happiness were more important. Then an outcome which increased
    the population while making them more unhappy might be rated highly by
    Lee but low by Eliezer. Or suppose that Eliezer believed that "friendly"
    AI would lead to a utopia, while Lee believed that any such effort would
    inevitably destroy the world; again, an outcome which increased the
    chance of this technology would be rated differently by the two altruists.

    However, these hypotheticals don't really work, because rational people
    can't disagree! See our previous discussion on this issue. So I think
    that for rational altruists, all 2-person games have identical payoffs
    to both individuals, making them in effect one person games.

    A final point; Eliezer points out that the "irrational" action of
    cooperating leads to a greater payoff. I believe the consensus
    among game theorists is that this does not change the fact that it
    is irrational. The reasoning is similar to that in Newcomb's paradox
    (take one box or two, but God has arranged that you will get more money
    if you take only one box). Taking two boxes physically entails getting
    at least as much as in the first box, hence it is the rational action.
    In this paradox, as well, being irrational leads to a greater outcome.
    I copied out some analysis on this issue from a decision theory book at
    http://www.finney.org/~hal/Newcomb.html. The argument doesn't go over
    directly to the PD case, but the flavor is the same: it is possible for
    an irrational action to lead to a greater outcome.

    Hal



    This archive was generated by hypermail 2.1.5 : Tue May 20 2003 - 12:43:17 MDT