Re: The Future of Secrecy

From: Robin Hanson (rhanson@gmu.edu)
Date: Wed Jun 18 2003 - 15:13:58 MDT

  • Next message: Samantha Atkins: "Re: META: Dishonest debate (was "cluster bombs")"

    At 04:52 PM 6/18/2003 -0400, Eliezer S. Yudkowsky wrote:
    >>... one disturbing implication of this is that we may well evolve
    >>to become even *more* self-deceived than we are now, as believing
    >>one thing and thinking another becomes even harder than now.
    >
    >Accepting the scenario, for purposes of discussion... why would
    >rationalization be any harder to see than an outright lie, under direct
    >inspection? Rationalization brainware would be visible. The decision to
    >rationalize would be visible. Anomalous weights of evidence in remembered
    >belief-support networks could be detected, with required expenditure of
    >scrutinizing computing power decreasing with the amount of the shift. It's
    >possible there would be no advantage for rationalization over lying;
    >rationalization might even be bigger and more blatant from a computational
    >standpoint.

    As I argued in my response to Hal Finney, the process that leads to a
    belief seems much larger than the resulting belief itself. To verify a
    belief you may need to just look up the right data structure. To verify
    that a mind that produces beliefs is unbiased required you to look at much
    more than a small data structure. This diffuse nature of belief production
    is in fact what makes it hard for us to see our own self-deception.

    Robin Hanson rhanson@gmu.edu http://hanson.gmu.edu
    Assistant Professor of Economics, George Mason University
    MSN 1D3, Carow Hall, Fairfax VA 22030-4444
    703-993-2326 FAX: 703-993-2323



    This archive was generated by hypermail 2.1.5 : Wed Jun 18 2003 - 15:26:37 MDT