Christopher McKinstry wrote:
>
> "Eliezer S. Yudkowsky" wrote:
>
> > Regardless of whether you agree, I would ask that if you in the future
> > happen to discuss the possibility of evolutionary, war-to-the-death
> > competition between humans and AIs, you also at least mention the
> > possibility of Friendly AI, even if it consists of the phrase "There have
> > been proposals for Friendly AI, but I think they're unworkable."
>
> Just one more point before I go off to read your 'Friendly AI'... if
> Kurzweil is right, and in the future I can scan my personality into a
> computer, the event will create an instant conflict simply because the
> copied version of myself will fight to the death not to be turned off by
> the original version.
This is assuming you have the bad taste of booting the scan and then
wish to end its existence. It would rightly consider you an infanticidal
maniac and defend itself against your predations.
Do not assume that a scan will automatically be concious.
> I would be just as friendly as I am now to my
> fellow 'virtual' humans (as long as I could verify they were virtual),
> but I would see 'real' humans as potentially very dangerous to my
> continued consciousness. No matter how friendly I am, I would have a
> very strong objection to my reality being externally controlled. Bad
> things will happen unless we can develop some form of protocol of trust,
> which I am not confident we can.
We are all under the mortal power of others our entire lives, from our
parents, neighbors, their dogs, and unknown strangers and foreign
governments and terrorists. A scanned mind would, with its much more
logical circuitry of its new substrate, be able to more accurately
calculate the risks and assign trust ratings based on observed behavior,
which are generally far less and far higher, respectively, when analysed
rationally. Automatic assumption of Other as Enemy is a sign of a
diseased mind, which if not healed by the process of upload, would
prevent upload due to system conflicts.
This archive was generated by hypermail 2b30 : Fri Oct 12 2001 - 14:39:42 MDT