hal@finney.org wrote:
>
> The part about Robin's paper (http://hanson.gmu.edu/deceive.pdf) that
> I have a hardest time understanding is the discussion of common priors.
Hm. Well, I understood it perfectly. Therefore, you should take this
into account in deciding whether or not the paper makes sense, unless you
don't think we have common priors.
Essentially, Robin's paper gives a rigorous mathematical proof that for
two people to (a) disagree and (b) maintain their disagreement after
interaction, one or both of the parties must believe that they are more
likely to be rational than the other person. This does not necessarily
prove irrationality in *all* cases but it proves irrationality for *most*
cases. If we take into account the evolutionary-psychology arguments,
Robin's paper makes a strong case for a built-in irrationality factor
common to all humans.
If a 100,000:1 genius is interacting with a 10,000,000:1 genius, but
neither of them knows the other's percentile, both will rationally assume
that they are more likely to be rational than the other person. However,
Robin's paper does prove that in *most* cases, rather than in the rarer
instances where two geniuses unknowingly interact, people must be
overestimating their own rationality relative to others, or else must not
be using rigorous Bayesian reasoning with respect to what they are
licensed to conclude from their own thoughts.
-- -- -- -- --
Eliezer S. Yudkowsky http://singinst.org/
Research Fellow, Singularity Institute for Artificial Intelligence
This archive was generated by hypermail 2b30 : Mon May 28 2001 - 10:00:02 MDT