From: Robin Hanson (rhanson@gmu.edu)
Date: Wed Jun 18 2003 - 13:47:21 MDT
At 12:52 PM 6/18/2003 -0400, Eliezer S. Yudkowsky wrote:
>If we suppose the Singularity to be wrong, then we have no idea what the
>next century will bring; hence, not enough information to even try and be
>clever about what we believe. Ergo, since we can't measure the possibly
>catastrophic cost of self-deception, rationality seems like the safest course.
>I have a problem with saying: "But what if people who confidently believe
>that things will go on just as before are right?" That's not a simple
>counterfactual because it strikes at the heart of one of the basic issues
>of rationality - lack of confidence that basic beliefs are right, the
>future being different from the past. This is not just hypothesizing an
>alternate future scenario; it is hypothesizing that beliefs don't need to
>change as much as they do and are more reliable than they are. It is one
>thing to ask about *different changes*, but to counterfactual on *no
>changes* is to start out by denying one of the chief things that makes
>rationality instrumentally necessary to arbitrary coherent goals.
As you say, this goes to the heart of the nature of rationality. There is
the standard paradox of "How long should I think about which car to buy",
"How long should I think about how long to think about what car to buy",
and so on. While a fully rational calculation should be more complex at
each meta level, you clearly must think for less time at each meta level,
and so at some level you are really not being very rational at all. At
some level you just act, without having an explicit analysis to say why.
The situation of evolved creatures with the possibility of costly rational
analysis seems similar to me. They usually just act based on evolved
habits, but sometimes they decide to pay the cost to be more rational about
some choice. But they can't be very rational about that choice of when to
be rational. At some level there is just the evolved habit, which may or
may not be appropriate to the particular situation.
Robin Hanson rhanson@gmu.edu http://hanson.gmu.edu
Assistant Professor of Economics, George Mason University
MSN 1D3, Carow Hall, Fairfax VA 22030-4444
703-993-2326 FAX: 703-993-2323
This archive was generated by hypermail 2.1.5 : Wed Jun 18 2003 - 13:57:51 MDT