Congrats, you just made my quotes file.
"But I am not an object. I am not a noun, I am an adjective. I am the way
matter behaves when it is organized in a John K Clark-ish way. At the present
time only one chunk of matter in the universe behaves that way; someday that
could change."
- John K Clark (johnkc@well.com)
> >If I somehow (another thought experiment) have the exact same thought
> >as you (I'm not sure how you'd go about measuring that) at exactly
> >the same time, are our two persons converging.
>
> Yes.
>
> >And if this continues, to the point where ALL our thoughts and
> >emotions are exactly the same, have we become one person?
>
> Yes.
Invert it. We can transfer the thought experiment from the material world to
the computer world without losing anything, assuming perfect simulation (and
not questioning how it got there; maybe our whole Universe is just a computer
simulation). You start out as a great big piece of code. Then, "Copy" and
"Paste" in such a manner that there is a clearly defined 'original' and a
clearly defined 'copy', both perfectly synchronized.
How many of you are there?
Would there be six of you if your computer had a triply redundant processor?
Now I take an element of code from the original, and an element of code from
the copy, and I slowly make them switch places. I.e., I take weighted factors
of input and output for some basic quantity, and shift the weighting over.
I do this for every element of code. Both copies are still perfectly
synchronized. No elements have been disturbed, only switched.
Which one is you?
The original made up of copied code, or the copy made up of original code?
Which one is you when the process is halfway complete?
Who are "you"?
--
sentience@pobox.com Eliezer S. Yudkowsky
http://tezcat.com/~eliezer/singularity.html
http://tezcat.com/~eliezer/algernon.html
Disclaimer: Unless otherwise specified, I'm not telling you
everything I think I know.