Re: Subjective counterfactuals

Eliezer S. Yudkowsky (sentience@pobox.com)
Sun, 04 Apr 1999 22:16:36 -0500

hal@rain.org wrote:
>
> The problem is, I can't give a meaningful answer to the question.
> Any answer I could give about what larger external environment I might
> be a part of cannot be correct in most cases. Even if my consciousness
> only exists during a causally based run of a computer, it could still be
> run multiple times. Each time, I will behave in exactly the same way.
> (I am assuming that there are not uncomputable elements necessary
> for consciousness.) Each time, I will think exactly the same thing.
> If I think to myself, "*this* instance of my consciousness is happening
> on the third run of the experiment", I will think that every time.
> It has to be wrong most of the time.

I think we all agree that even if a million monkeys typing on a million typewriters generated the paragraph above, it wouldn't make the Hal who wrote it real. Only Hal's brain or a simulation thereof - being instantiated - can put actual qualia behind the statement. It's not enough to have the inputs and outputs in a Giant Lookup Table. So the question is, if the playback was generated by random chance, can the Hal it "records" be said to exist? Will he ever, even once, have said "cogito ergo sum", or is there only a text-based representation of the words? For the purposes of this argument we are *assuming* that an actual neuron-by-neuron simulation would make the Hal real; the question is, *given* that, does a randomly generated recording also make Hal real?

-- 
        sentience@pobox.com          Eliezer S. Yudkowsky
         http://pobox.com/~sentience/AI_design.temp.html
          http://pobox.com/~sentience/singul_arity.html
Disclaimer:  Unless otherwise specified, I'm not telling you
everything I think I know.