From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Sun Jun 01 2003 - 07:06:09 MDT
Hal Finney wrote:
> Eliezer writes:
>
>>I can personally attest that, in practice, this is not how meta-rational
>>Bayesian wannabes actually argue.
>>...
>>Eliezer Yudkowsky: "What's your estimated probability that the $20 bill
>>is yours?"
>>Nick Bostrom: "Fifteen percent."
>>Eliezer Yudkowsky: "Mine is twenty percent."
>>(Calculator referred to.)
>>Eliezer Yudkowsky: "I'll keep the twenty. Here's eight dollars and
>>fifty-seven cents."
>>
>>And that's how it works.
>
> I'm surprised that your two estimates were so low. At the time of your
> conversation, you said that you both knew that the other didn't think it
> was their $20 bill, and you had both spent some effort to verify this,
> counting the money in your wallets and such. It's strange that you
> each thought that there was an 80 to 85 percent chance that the other
> person had made a mistake about how much money he had paid, even after he
> had gone through the same careful verification procedure that you had.
> I would have expected that the kind of objective, symmetrical reasoning
> required by meta-rationality would lead you to place a greater credibility
> on the statements of the other person.
We did. We were giving our estimates without taking the other's statement
into account, exactly so that we didn't run into your infinite recursion
problem. In other words, since we both knew the other thought that he
also hadn't contributed the $20, the real chances were probably somewhere
in the vicinity of 50/50. But to calculate what our actual revised
beliefs should be, we had to state what our probabilities were before we
started taking the other's probabilities into account. I don't recall
whether this was stated out loud, but it is obvious.
> I also wonder how you might have each adjusted your probabilities upon
> hearing the estimates of the other. Given that the two probabilities
> have to sum to 1, what you basically have is an estimate by Nick of 85%
> that the bill was Elizer's, and an estimate by Eliezer of 20% for the
> same fact. I believe Robin's result is that upon hearing these estimates,
> each of you would adjust his own estimate, and would now expect that the
> other's adjusted estimate would be about the same as you's. At least,
> you would not be able to predict whether the other person's estimate
> was now greater or less than your updated value.
>
> It might have made sense for you to have exchanged a few more estimates.
> Chances are you would have converged quickly and been able to split the
> money on that basis.
No, we converged immediately to a probability of 15/35 that the bill was
Bostrom's and 20/35 that the bill was mine.
-- Eliezer S. Yudkowsky http://singinst.org/ Research Fellow, Singularity Institute for Artificial Intelligence
This archive was generated by hypermail 2.1.5 : Sun Jun 01 2003 - 07:16:41 MDT