den Otter wrote:
>
> Ok, let me explain...again.
Please do. I think your position is very peculiar; you've gotten through most of the logic of causality and then, as far as I can tell, stopped halfway.
> Emotions may be "arbitrary evolved adaptations", but they're
> also *the* meaning of life (or: give life meaning, but that's
> essentially the same thing). That's how we humans work.
I disagree. Let's suppose I accept your basic logic, as a premise for discussion. Fine. Emotions are still only part of the mind, not all of it. For as long as I continue doing things, making choices, instead of committing suicide, then my life is as meaningful as yours. It doesn't matter whether the particular set of cognitive contents that causes me to eat a slice of pizza, or the cognitive contents that make me happy as a result, are in the cortex or the limbic system - whether they're built-in "emotions" or software "thoughts". How can you draw a hard line between the two? They're both made of neurons.
> The only reason why logic matters is because it can be a useful
> tool to achieve "random" emotional goals. Rationality is practical
> ...in an emotional context. It has no inherent meaning or value.
> NOTHING has. Even the most superb SI would (by definition) be
> _meaningless_ if there wasn't someone or something to *appreciate*
> it. Value is observer-dependent. Subjective.
Okay, suppose I accept this. I still don't see why any SI will automatically drop off all its emotions *except* selfishness and then you think this is a good thing. That's really where you lost me.
> At least, that's what I suspect. I could be wrong, but I seriously
> doubt it. So what's the logical thing to do? Risk my life because
> of some emotional whim ("must find the truth")? No, obviously not.
But why not? There's one emotional whim, "must find the truth". And another emotional whim, "stay alive". And another, "be happy". In me the first one is stronger than the other two. In you the two are stronger than the one. I don't get the part where your way is more rational than mine, especially if all this is observer-dependent.
> The rational thing to do is to stay alive indefinitely, sticking
> to the default meaning of life (pleasant emotions) until, if ever [*]
> , you find something better. So maybe you'll just lead a "meaningful",
> happy life for all eternity. How horrible!
Trying to divide this "meaning of life" into a set of declarative "goals" and a set of procedural "rationality" strikes as being artificial and subject to all sorts of challenge. Maybe the "rationality" is another set of goals - "Goal: If A and A implies B, conclude B." - and the new procedural meta-rationality is "follow all these goal rules". Like taking a Turing machine with "rational cognition" and "goals", and turning the state transition diagram into goals, and making the new rational rules the Universal Turing machine rules. Has anything really changed? No. The "meaning of life" is the entire causal matrix that produces choices and behaviors; you can't really subdivide it any further. The only reason we call something a "goal" in the first place is because it produces goal-seeking behavior. There's nothing magical about the LISP atom labeled "goal" that turns it into an actual goal.
> [*] Actually, as I've pointed out before, uncertainty is
> "eternal"; you can never, for example, know 100% sure that
> killing yourself is the "right" thing to do, even if you're
> a SI^9. Likewise, the nonexistence of (a) God will never be
> proven conclusively, or that our "reality" isn't some superior
> entity's pet simulation etc. You can be "fairly sure", but
> never *completely* sure. This isn't "defeatism", but pure,
> hard logic. But that aside.
And I repeat: "You know this for certain?" But that aside.
> So stay alive, evolve and be happy. Don't get your ass killed
> because of some silly chimera. Once you've abolished suffering
> and gained immortality, you have forever to find out whether
> there is such a thing as an "external(ist)" meaning of life.
> There's no rush.
I'll certainly agree with that part. Like I said, the part I don't get is where you object to an AI. And I also don't entirely understand what you think the difference is between human and AI cognitive architectures that makes humans so much more stable and trustworthy. Do you have any idea what kind of muck is down there?
> Or:
>
> #1
> Your goal of creating superhuman AI and causing a
> Singularity is worth 10 emotional points. You get killed
> by the Singularity, so you have 10 points (+any previously
> earned points, obviously) total. A finite amount.
>
> #2
> You upload, transcend and live forever. You gain an infinite
> amount of points.
>
> Who wins?
Nobody, unless you say that "getting points" is the objective, provable, rational meaning of life, in which case obviously the rational thing to do is to create a superhuman AI, which will naturally follow the same logic and attempt to earn an infinite number of points.
Or:
#1
"Your goal of creating superhuman AI and causing a
Singularity is worth 10 emotional points. You get killed
by the Singularity, so you have 10 points (+any previously
earned points, obviously) total. A finite amount."
This scenario is worth 20 emotional points to me.
#2
"You upload, transcend and live forever. You gain an infinite
amount of points."
This scenario is worth 5 emotional points to me.
Who wins?
--
sentience@pobox.com Eliezer S. Yudkowsky
http://pobox.com/~sentience/tmol-faq/meaningoflife.html
Running on BeOS Typing in Dvorak Programming with Patterns
Voting for Libertarians Heading for Singularity There Is A Better Way