Re: Subjective Morality

Eliezer S. Yudkowsky (sentience@pobox.com)
Mon, 11 Jan 1999 11:07:47 -0600

James Ganong wrote:
>
> I believe that rape, murder & genocide are wrong, but I cannot point to
> some Book & demand that sinners repent. At best I can point to economic
> analyses that show these things to be inefficient, but what are the odds
> that a rapist/murderer/genocide will be moved by such? I can only work
> to make a world where they are wrong because *we make it so*, just as we
> work to make our Transhuman/Extropian
> goals real.

Okay, the basic fallacy here is that anybody knows what the Objective Morality is. I ain't got a clue. If I did, I wouldn't be building SIs to clue me in, I'd be building them for quite different reasons.

The three rules of my philosophy, Externalism:

  1. Morality is an objective, observer-independent reality external to ourselves. Questions of right and wrong are questions of true and false; there is no ontological distinction between "what is the mass of this particle" and "what should I do now".
  2. I don't know what the objective morality is and neither do you. Anything we say on the subject is a guess, just like any other scientific statement.
  3. Since objective morality is real and external from our opinions, our instincts can have no influence on it; thus objective morality is probably as strange and unintuitive as quantum physics or general relativity.

> "There is no justice; just us."
> - Terry Pratchett, _Soul Music_

Spoken by Death. And if I, too, may quote:

"Are you too *scared* to change the world?" Death turned. The very sight of his expression made Susan back away. He advanced slowly toward her. His voice, when it came, was a hiss. "YOU SAY THAT TO *ME*? YOU STAND THERE IN THAT PRETTY DRESS AND SAY THAT TO *ME*? YOU? YOU PRATTLE ON ABOUT CHANGING THE WORLD? COULD YOU FIND THE COURAGE TO ACCEPT IT? TO KNOW WHAT *MUST* *BE* *DONE* AND DO IT, WHATEVER THE COST? IS THERE ONE HUMAN ANYWHERE IN THE WORLD WHO KNOWS WHAT DUTY *MEANS*?" 

-- 
        sentience@pobox.com         Eliezer S. Yudkowsky
         http://pobox.com/~sentience/AI_design.temp.html
          http://pobox.com/~sentience/sing_analysis.html
Disclaimer:  Unless otherwise specified, I'm not telling you
everything I think I know.