Re: Is vs. Ought (was: A Physicist Experiments With CulturalStudies)

Eliezer S. Yudkowsky (sentience@pobox.com)
Thu, 18 Nov 1999 17:01:10 -0600

Robin Hanson wrote:
>
> Greg Burch wrote:
> >For now, let me ask you a question, Robin: Can we question that what we DO
> >want is what we SHOULD want?
>
> Sure, within a context where we care about what we want.
> That is, when it makes sense to say "I don't want to want that."
> Such things make sense when we realize that we are not atomic,
> but made of parts, spread across space and time.

Actually, you don't even need atomicity. I would divide things as follows:

  1. The way things are (objective reality);
  2. The way things should be (objective morality);
  3. The way we currently want things to be (objective reality, part of 1);
  4. The way we should want things to be (objective morality, converges to wanting 2)
  5. Our current model of the way we should want things to be (model, converges to 4)

If 5 (your model of 4) conflicts with your model of 3, then you don't want to want what you want.

Or in much simpler terms, and leaving out the objective morality: If you *want* something, then you want a part of the Universe to change. Well, what you want is also a part of the Universe, and you can want to change what you want. 

-- 
           sentience@pobox.com          Eliezer S. Yudkowsky
        http://pobox.com/~sentience/tmol-faq/meaningoflife.html
Running on BeOS           Typing in Dvorak          Programming with Patterns
Voting for Libertarians   Heading for Singularity   There Is A Better Way