From: Wei Dai (weidai@weidai.com)
Date: Mon Mar 17 2003 - 20:49:30 MST
I think the widespread ability to tell the truth convincingly would cause
a singularity by itself, in the sense that there is no way to tell what's
going to happen afterwards. Virtually all of our current social
institutions (markets, governments, firms, schools, marriage, friendship,
to name a few) are designed to solve the problem of cooperation in the
absence of the ability to tell truth convincingly. All of these
institutions become obsolete in a "World of Knights".
Here's an example of what I mean. Why do we have schools that hire
teachers as employees instead of self-employed teachers who just rent
classrooms? Ultimately, it's because students can'be trusted to honestly
report how much they learned to potential employers. They have to be
evaulated by their teachers and those evaluations have to be backed up by
the reputations of the schools. (Notice how much the existence of this
institution depends on other human cognitive limitations, notably the
inability to track the reputations of a large number of individual
teachers.)
What will replace our institutions is impossible to predict for several
reasons. One, we don't know how this truth-telling technology will work
exactly. Can you convince others of your values as well as beliefs? Does
it prevent you from believing in self-serving rationalizations? Our
current notions of rationality depends on the assumption that not only are
your beliefs private, but there is no way you can convince others that you
truly believe them. If you *are* able to convince others of what your
beliefs are, it's no longer in your self-interest to only believe in what
is true. We already see this to some degree because humans are not able to
lie costlessly. The incentive for self-serving rationalizations becomes
much higher when lying is impossible. It's not clear whether this could be
prevented by any kind of technology. Two, there is a lot of inertia behind
some of the existing institutions, and it's hard to tell which ones will
actually disappear and in which order. Three, once the problem of not
being able to convincingly tell the truth go away, what are the remaining
problems in efficient cooperation and how severe are they? We may be able
to predict some of these, but the incentive to invent new institutions to
minimize them (which is a difficult task) doesn't exist yet.
This archive was generated by hypermail 2.1.5 : Mon Mar 17 2003 - 20:57:04 MST