Re: Uploads and betrayal

Robin Hanson (rhanson@gmu.edu)
Wed, 01 Dec 1999 14:16:48 -0500

Charlie Stross wrote:
> > >will be possible to reach in and select what you want (your motivation).
> > ...
> > replace what we currently want with what we currently want to
> want. And then
> > we will replace that with what we then want to want, ...
>
>Mmmf. I'd like to agree with you, but let me play devil's advocate:
>
>If such technology becomes available, I believe it will be used by certain
>regimes (yes, you know who) to ensure that what we want is *what they
>want us to want* (at least, for a noticeable subset of the human species).
>
>... Unlike all previous totalitarian ideologies, ours is based
>on the abolition of unhappiness. Pissed off? Don't like our way of life?
>Never mind, we'll be able to tune your qualia so that you're perfectly
>happy with the way things are!"

I agree that such technology could be used this way. The question is: how well will such creatures compete with those differently constituted?

We evolved to be happy sometimes and unhappy other times because this was functional in our ancestral environment. Our unhappiness, for example, was often used as a clue to consider trying things differently. If you just make people happy in a crude way, you could disturb these feedback loops to their disadvantage.

On the other hand, it is possible that groups of happily subservient creatures will be competitive with others. That may offend your 20th century sensibilities, but the universe need not care.

Robin Hanson rhanson@gmu.edu http://hanson.gmu.edu Asst. Prof. Economics, George Mason University MSN 1D3, Carow Hall, Fairfax VA 22030
703-993-2326 FAX: 703-993-2323