Yet another robot set free from its designer

From: Jacques Du Pasquier (
Date: Tue Jan 29 2002 - 16:58:01 MST

Eliezer S. Yudkowsky wrote (29.1.2002/02:53) :
> The morals encoded in our genes pale by comparison to the genetically
> encoded adaptations for reasoning and arguing about morality. The latter
> is also important in a hunter-gatherer society, after all. Don't look at
> the starter-set of adaptations, look at the adaptations for choosing
> between moralities. Don't look at the system, look at the dynamic for how
> the system changes.
> One way to answer the question "What is the meaning of life?" is to
> understand fully what it is that impels humans to ask this question,
> phrased in this way. Trying to explain that life "really" has no meaning
> doesn't explain how it is that evolution accidentally constructed
> organisms who think it does. We may be very certain that evolution had no
> "deeper intentions" in doing so. Evolution simply doesn't work that way.
> But if you really want to answer this question, you have to understand how
> evolution managed to accidentally construct organisms who, as an emergent
> outcome of their set of locally adaptive rules for moral reasoning, look
> for a "meaning of life" instead of being content with their inbuilt
> goals. Without this knowledge you can't answer the question "What is the
> meaning of life?" You can't even say for sure that it has no real
> answer. Until you understand where the question comes from, you don't
> understand the question.

I agree to this last point, but I am not sure that "moral reasonning"
is really key to the psychological explanation for the Question.

There is one simple point, which is that we were not meant, as
individuals, to realize we are mortal, and to reflect about it.
Animals seek what they are made to desire, and the stream of life (the
genes' purpose) goes through them that way. When one animal, to which
a strong sense of self has been given, realizes that he or she was
designed as instrumental to this stream of life (he is not the end --
in fact, though apparently *made to survive*, he is actualy *certain
to die*), a strange situation results. (in which we spent all of our
history so far)

We were built for a well defined purpose (copying our genes), and
autonomy was given to us, so to speak, for seconding that purpose more
efficiently ; but too much autonomy it would seem, and too much
cognition, as it has now led us to reflect on that purpose and take
distance from it, and consequently to wonder what purpose we should

That's the origin of the Question. Not the fact that we of all things
seek to have purpose ("moral reasoning") ; but the fact that we used
to act blindly according to a definite purpose, and by successive
steps we woke up from this and ended up strangely free (but mortal).

If you think of evolution (or the genes) as a conscious designer, you
could say that he (or she) messed up, that he overlooked the
consequences of his design, and lost control of some of his robots
(humans). It's an old story told by many traditions (but in which the
creator is either God -- think Satan -- or Man -- think Dr Frankenstein's
creature --, not evolution), and it turns out that, with all the
support of our scientific understanding, we now realize this is actually
*our* story.

The question is not really linked per se with consciousness of
mortality ; but that's a huge hint (that we are instrumental), and
that's why, though having no clue about evolution, humans started
suspecting something long ago.


This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 13:37:37 MST