From: Ramez Naam (mez@apexnano.com)
Date: Mon Mar 10 2003 - 18:08:32 MST
From: Eliezer S. Yudkowsky [mailto:sentience@pobox.com]
> I finally did work out the theory that
> describes what it is going from point A to point B when a moral
> human builds a moral AI, and it turns out that if you build an AI
> and you don't know exactly what the hell you're doing, you die
> Period. No exceptions.
>
> Do you have any ideas for dealing with this besides building
> FAI first? Because as far as I can tell, humanity is in serious,
> serious trouble.
What exactly do you mean by humanity? Do you mean the individual
humans? If so, none of them are counting on living forever, and
frankly the odds are that the vast vast majority of them will die in
the next several decades.
I'm not trying to be callous here. I care about people, even those I
haven't met, and I do my small bit to help eliminate needless death.
But individuals die. They always have and they always will.
Alternately, when you said "humanity" did you mean the human species?
If so, should we care? I care about individuals. If those
individuals are AIs or post-humans or such, is that any worse than if
those individuals are humans? I don't see why. But maybe I'm unusual
in having more sentience-loyalty than species-loyalty.
Let me put it another way. Go back and put yourself in the position
of a primate ancestor of ours, maybe 5 million years ago. Someone
magically gives you intelligence for a day or so, and gives you a tour
of the future. He shows you these things called "humans" that are
smarter than you are, have richer social interactions, create this
amazing art and technology, unravel the secrets of the universe,
etc... He also shows you how the range of habitats for primates
shrinks in the human era, how humans use primates for medical
research, and so on. Then he gives you the choice. Should humanity
be allowed to come into being or not?
I would choose yes. Today, I would rather see creatures with more
intelligence, awareness, creativity, passion, and curiosity than
humans come into being. I don't want to do so in a way that hurts
people, but I know that evolution goes hand in hand with strife.
Given the choice between humanity continuing in its current form for
millennia vs. humanity succumbing to a post-human type of life, I'd
choose the latter.
mez
This archive was generated by hypermail 2.1.5 : Mon Mar 10 2003 - 18:15:05 MST