Meta-rationality (was: Ashcroft Antics)

From: Eliezer S. Yudkowsky (
Date: Thu Jan 31 2002 - 18:49:02 MST

Lee Daniel Crocker wrote:
> Eliezer wrote:
> >
> > This thread now concludes, the two former disputants having
> > communicated their views to each other, adjusted their views
> > as a result of new information received, and reached effective
> > agreement.
> That's because you and I realize that we are not really combatants
> in a competitive argument, but fellow components of a collaborative
> truth-seeking system. :-)

Or, we could have unconsciously faked agreeability in order to convince
social observers that we are good Hansonian meta-rationalists.

To take an extreme case, suppose that I argue with you, and your opinion
converges to mine, but mine does not change. It follows that you
(although not I) will appear to be meta-rational, and that I will appear
to be rational but not necessarily meta-rational - the underlying
assumption being that I had good facts, and that you had the ability to
adjust your opinion based on the facts, and that this resulted in the
observed outcome, convergence of your opinion to mine. If anyone else
later disagrees with you on a different issue, you are probably right;
after all, you have demonstrated the ability to adjust opinions based on
new information, and the opposing party has not yet done so. Since the
quality of someone's database has heterogenous causation and is hence
likely to vary more than their meta-rationality, demonstrating
meta-rationality raises total credibility more than winning any single

By demonstrating the ability to "lose" an argument where the case
warrants, one greatly strengthens the Bayesian signal of claiming to be
winning. It follows that losing an argument in front of an audience of
meta-rationalists can operate to raise your credibility, and that, in an
odd way, losing an argument can even be a way of boasting about your own
meta-rationality. Thus when wrapping up the argument, I should probably
have written that I observed your opinion to change without claiming that
my opinion changed, since the latter is "boasting" about my own
meta-rationality, and should be stated by a third party in order to be
credible. On the other hand, if I claim that your opinion changed without
also "conceding" that mine changed, I may appear to non-meta-rationalists
to be claiming to have "won" the argument, which would of course be a
childishly immature way to respond to someone else demonstrating
meta-rationality. An interesting dilemna in public relations.

-- -- -- -- --
Eliezer S. Yudkowsky
Research Fellow, Singularity Institute for Artificial Intelligence

This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 13:37:37 MST