On Fri, 4 Dec 1998, Nick Bostrom wrote:
> I think Eliezer's interesting argument is unsound, because I think
> one of it's premisses (RA1) is false.
> I don't see why a rational debate about morality would be impossible
> if you or I knew the "objective morality". People often
> rationally debate issues even when one party already knows where the
> truth lies.
> As for RA1, I think it could well be argued that it is false. We may
> not know in detail and with certainty what the moral facts are (so
> sure we'd want to assign probabilities), but it doesn't follow that
> we know nothing about then at all. In fact, probably all the people
> on this list know that it is wrong to torture innocent people for a
> small amount of fun. We could no doubt write down a long list of
> moral statements that we would all agree are true. Do you mean that
> we all suffer from a huge illusion, and that we are all totally
> mistaken in believing these moral propositions?
I don't think you could put up many (any?) statements that even all the relatively like-thinking people of this list would agree to. I, for one, am not sure about the example moral 'fact' presented above. I may be a very small minority but i would say large enough to refute its claim to moral fact. And judging from Eliezer's rational, i would say you wouldn't get him to agree to any moral statement you might put forward. Are we going to assign probabilities in terms of percentage of people agreeing with each moral?
> You might say that our human morality - our desire that *we* survive
> - is an arbitrary effect of our evolutionary history. Maybe so, but I
> don't see the relevance. If our morality is in that sense arbitrary,
> so what? You could say that the laws of physics are arbitrary, but
> that does not make them any less real. Don't forget that "moral" and
> "good" are words in a human language. It's not surprising, then, if
> their meaning is also in some way connected to human concepts and
> anthropocentric concerns.
But what about the response coming from most of the people i talk to who reject the idea of avoiding death and seem disgusted with the idea of even living any 'longer than you should', maybe even sometimes using terms like 'morally wrong'? Is surviving really a 'good' thing? They obviously don't think so. I see no common all-of-humanity ground to create an objective morality set.