RE: Status of Superrationality

From: Lee Corbin (lcorbin@tsoft.com)
Date: Tue May 20 2003 - 19:51:07 MDT

  • Next message: Phil Osborn: "Re: "Hysteria, Thy Name is SARS""

    Eliezer writes

    > > Listen, the whole idea behind the *entries* in the table is that
    > > they are the payoffs of the players. That's what the numbers
    > > *mean*, the utilities of the players.
    >
    > What the experiments *measure* is monetary amounts.

    Yes, but that's just a practical limitation of the "experiments".
    The entire difficulty, as well as the profundity of PD and its
    variants, arises from considering that the entries represent one's
    actual utility.

    I am confident that with some thought, you may be able to represent
    numerically the equivalent game so that the same considerations
    obtain for you as for those of us who view the entries as utilities.
    Perhaps Hal's experiment suffices.

    > > I say that you should Defect (and that is what the entries in the
    > > table say too) whenever you know what your opponent is going to do.
    > > If you know he is going to Defect, then you Defect. If you know
    > > that he is going to Cooperate, then you Defect. It's right there
    > > in the table.
    >
    > Not if your opponent determined his behavior by running
    > an accurate simulation of what you would do if you knew
    > definitely what your opponent would do. Think timeless.

    Let me get this straight. I'm a player in a PD, and all I know
    is that my opponent has determined what he will do by running a
    simulation of me. This still does not enable me to know what he
    is going to do, does it? Yes, *if* he's acting on what I would
    do *if* I knew definitely what he would do---that's one thing---
    but it's pretty far afield from the actual situation in which I
    evidently (in your example) do *not* know what he is going to do.

    > > The highly interesting case is if you don't know what he's going
    > > to do AND his behavior is probably correlated with yours. For example,
    > > I would be afraid to Defect against a physical duplicate if he was
    > > a close (or near) duplicate. After all, whatever thoughts I think
    > > are likely to course through his head as well. But if it any entity
    > > whatsoever, including 1983 versions of myself, then the table says
    > > "Defect".
    >
    > And knowing this, your past self will defect against you, since he knows
    > you know what his behavior will be.

    No, my past self in 1983 believed in superrationality. He would be
    dumbstruck if I returned in a time machine, and then defected. I
    know what he will do. He will cooperate. Now, if I visit 1998,
    I do *not* know what my past self will do. It's probable that he
    will cooperate, but far from certain: he *might* anticipate that I
    will reason through that I should cooperate (then he will defect).
    Thus I *don't* know what he will do. Therefore it is not necessarily
    true that I should defect (although perhaps I will and should).

    > By your logic, in fact, you and your past self should play this way even
    > on the *iterated* Prisoner's Dilemma. After all, you know what his
    > decision will be on every round. Oddly enough, though, the past you
    > remember is a string of defections. How sad.

    How ridiculous. The Iterated PD is a completely different story.
    I tried and tried in 1988 to work out proper strategies for it, but
    had to finally give up; (after all, it's not a zero sum game). The
    only Nash equilibrium is where one defects on each round, which,
    as you go on to say here, is really stupid.

    > Meanwhile, the two cooperating Eliezers walk away, grinning, pockets
    > loaded with utility tokens. So who's the rationalist now, wise guy?

    Quite so. One must Cooperate until "somewhere near the end". But I
    finally decided that I might as well Cooperate right up until the very
    end. I used a weird form of logic to reach this conclusion. Here's
    how it went. Suppose that I begin to think about defecting on the
    (known) last round. Then I can anticipate the enhanced payoff from
    this action, and so must anticipate my opponent's identical analysis.
    Then, it becomes reasonable to Defect on the second-to-last round,
    and so on. Thus, for me, the first step "Suppose that I begin to
    think..." is the fatal one! So in the iterated case---where there
    is a great deal of utility to be gained---as you say, a lot of gold
    coins, or 200 repetitions of the same game---I cooperate, probably
    even unto the last round.

    Lee



    This archive was generated by hypermail 2.1.5 : Tue May 20 2003 - 20:06:49 MDT