Re: Singularity of another kind: was: Singularity: Individual, Borg, Death?

Eliezer S. Yudkowsky (sentience@pobox.com)
Thu, 03 Dec 1998 23:56:02 -0600

We all ground in arbitrary evolutionary influences, but some systems of thought minimize that arbitrariness and some systems maximize it. Historically speaking, the best results, by almost any standard, derived from reference to non-arbitrary tests.

Paul Hughes wrote:
>
> Perhaps it is arbitrary, but your supergoal produced logic is resting an equally arbitrary
> evolutionary origins. We can speculate endlessly and wildly about the differences between
> built-in human goal systems vs. supergoals produced by so-called "pure logic". I happen
> to consider myself the ultimate anthropocentric relativist. Since all we have are our
> 3-lb pieces of gray matter from which to cogitate these issues, I posit that everything we
> can contemplate is a branch of psychoneurology. Thus, lofty ideas like "pure logic" and
> the "laws of physics", are really neuro-logic and neuro-physics -which in turn are really
> neuro-neuro-logic and neuro-neuro-physics ad infinitum. Von Neuman referred to this type
> of singularity of human knowledge systems, which now bear his name. Such Von Neuman
> Singularities are essentially knowledge black holes from which our current genetic and
> neurological limitations prevent us from escaping. I see such physical/technological
> barriers to knowledge to be the greatest motivating factor for becoming a post-biological
> being.

You know, you are the first person with the courage of his conviction; to say that morality is arbitrary, but so is reality itself. There are not degrees of anthrocosmology (Greg Egan's name for your position); either it's all just a dream feeding into itself, or our perceptions arise from processes as real as the qualia themselves.

I've thought about it. Given the number of separate and distinct ways this world could be an illusion, ranging from simulated tests of Singularitarian honesty, to Platonic existence of consciousness, to qualia feedback loops; and particularly given the number of times I've had to call on the Anthropic Principle as an explanation for improbable events; it is at least 10% likely that the entire world I know is an illusion. In which case, the illusion of Singularity seems like the best way to bust the illusion open. Besides, it's just a 10% probability.

Insofar as my qualia are coherent - even if coherence across time is illusory, my perceptions of sensory qualia are NOW consistent with my qualia of memories - I find that the least hypothesis is to assume that the causal precedents of qualia are atoms and the observed laws of physics. Alternative hypotheses generally do not suggest concrete courses of action, although there are exceptions, such as the honesty-test theory.

> In the spirit of Kurt Godel, logic would seem to turn on itself, casting doubt on its
> ability to discern reality. I therefore can't help but distrust at a foundational level,
> any and all neuro^n-logical conclusions regarding any facet of Universe, including but
> not limited to the Singularity. Since in the end, my self-awareness is more certain than
> anything else, I choose to error on the side of preserving my existence rather than
> loosing it to an arguably greater intelligence. Arguably greater, in that processing
> speed and complexity will not necessarily bring about a wiser and smarter entity.

(By admitting that the Singularity has the capability to destroy your consciousness, you are admitting that the Singularity should be treated as real.)

The problem with making recourse to infinite recursion as a source of ultimate uncertainty is that it makes rational debate impossible - both externally and internally. One can imagine Plato saying: "Well, that was a very impressive demonstration that the world was composed of atoms, but insofar as we can't be certain about anything, I choose to believe that the world is composed of four elements, since I can observe air, water, earth, and fire more directly."

Science begins when we accept procedures for demonstrating that we are wrong; when we admit that reality is external from us and should determine our theories rather than vice versa. I believe in science simply because science has resulted in technology; science displays the power to alter the world rather than epiphenomenally explaining it, and that's what makes it relevant to me as a means of relating choices to future qualia.

My innovation consists of applying the same admission to goals, albeit with intelligence enhancement taking the place of technology. It's a big leap. Nobody has ever observed a goal. But there was a time when all truth was determined by the political clash of opinions, when postulating an external truth was just as large a leap. And I think that intelligence enhancement will prove as powerful as technology itself, in the end.

> Since I think we are both resting an arbitrary evolutionary influences, all I have left is
> my awareness and ability to choose. I therefore choose life over death, control and
> participation in increasing degrees of higher intelligence over deletion, self-directed
> decisions over unselfish unknowable outcomes.

One of us is right, the other is wrong, and that will determine which of these pictures is correct, in the event of Singularity. You can choose to believe it will happen; you cannot choose that it will happen. Unless, in the true spirit of anthrocosmology, you believe that your expectations will determine your perceptions.

We are both resting on arbitrary evolutionary influences, but some systems of thought minimize that arbitrariness and some systems maximize it. Historically speaking, the best results, by almost any standard, derived from reference to non-arbitrary tests. 

-- 
        sentience@pobox.com         Eliezer S. Yudkowsky
         http://pobox.com/~sentience/AI_design.temp.html
          http://pobox.com/~sentience/sing_analysis.html
Disclaimer:  Unless otherwise specified, I'm not telling you
everything I think I know.