Re: Truth machines

Eliezer S. Yudkowsky (
Mon, 21 Dec 1998 15:41:45 -0600

Billy Brown wrote:
> A truth machine would reduce the problems of autocracy by telling the
> autocrat who he can trust. As a result, he has less need to actually
> oppress anyone. Stupid dictators will do so anyway, but smart ones will
> realize they can now have their cake and eat it too. They can give the
> subjects enough freedom to create a healthy, functioning free market without
> risking revolt, because it is so easy to round up the actual
> revolutionaries.

Smart dictators? How many dictators do you know of who are smart enough to privately acknowledge that free-market economies work better? Besides which, I find it rather hard to believe that even a system of truth machines could create a healthy, functioning free-market autocracy. If you shoot everyone who thinks about rebelling, nobody is left to do the work. Shoot only those people who actually begin planning, and eventually a completely spontaneous uprising will knock off the dictator. Autocracy fundamentally relies on massive poverty and fear and I don't think truth machines are strong enough to change that. Nanotechnology, where the autocrat is simply stronger than the rest of the population, might make autocracy practicable.

> Democracy, on the other hand, would not function even as well as it does
> now. The truth machine would give us politicians who actually believe
> whatever the majority opinion happens to be one the major issues of an
> election, which is not a good thing when the majority is usually wrong.
> Worse, it would remove one of the major barriers that prevents all those
> stupid regulations from being enforced - imagine what agencies like the IRS,
> the EPA and OSHA would do with it.

The question is whether truth machines enhance the collective intelligence by eliminating liars and exploiters, or dehance by requiring rationalization and eliminating dissent. Perhaps the majority opinion will change for the better when their sole choice is between honest fundamentalists, honest liberals, honest conservatives, honest communists, and honest libertarians. It's the people selling dumb little compromises, and the people dishonestly pretending to go along with the majority opinion, who are the current problem.

Perhaps true rational debate will become possible when everyone has to acknowledge the honesty and goodwill of all the others. The whole opinion space would be reshaped; it's not just a question of the current space being enforced. I think that on the whole it would wind up being an enhancement, but your position is certainly defensible.

If I turn out to be wrong, I'd have to drastically change my current strategy. Why? Similar questions apply to collaborative filtering - does it enhance intelligence by seeking out correct arguments and skeptical counterarguments, or does it dehance intelligence by telling everyone only what they want to believe?

Both, of course, but who wins? Regardless of the initial percentages, is there sufficient conflict for the advantages of the enhanced users to become apparent and the enhanced users to become the majority?

Are there enough democracies for one of them to turn libertarian and incorruptible under the influence of truth machines (or collaborative filtering) and export leaders to all the others?

My intuition perceives truth machines and collaborative filtering as fundamentally similar.

> In a world with truth machines, autocracy could easily be a more viable form
> of government than democracy.

You have a definite point, and perhaps it will depend strongly on initial conditions. A theocracy might have internal faction fights, but could easily fill the government and army with people who are not only "not planning revolution" but who fanatically believe in the current government. Regular interrogations could eliminate doubts (or in extreme cases doubters) and thus allow a wealthy theocracy, without the secularism that usually accompanies affluence.

Basically the question is "Do you prefer Bill Clinton, a slimebucket who's too busy compromising to be a real communist, or do you prefer Al Gore, a reasonably intelligent person who's an environmentalist?" On the whole, I'd poison Clinton any day. Slimebuckets do more damage to a democracy than honest fanatics. I serve intelligence, and there is basically no circumstance under which I prefer stupidity to intelligence. Unless the other guy is, not just the loyal opposition, but actually sighting down a rifle at me.

--         Eliezer S. Yudkowsky

Disclaimer:  Unless otherwise specified, I'm not telling you
everything I think I know.