Re: Why would AI want to be friendly?

From: Samantha Atkins (samantha@objectent.com)
Date: Mon Sep 25 2000 - 13:17:49 MDT


"J. R. Molloy" wrote:
>
> > No, because this is not the Singularity Seed but simply successful
> > genetic programming projects. You seem to be answering a very different
> > question than I was asking.
> >
> > - samantha
>
> Right. You had asked, "In what way is this good for or even compatible with the
> nature of human beings?"
>
> But I maintain that what is good for or compatible with human nature is not
> necessarily the most extropic path to higher levels of self-organized complexity
> or more sentient life forms.
> What is good for human beings may, after all, be bad for transhuman beings.
> If AI is not friendly toward humans, that doesn't mean it will be unfriendly
> toward transhumans.
>

So now we have at least three camps that may have conflicting
interests/goals: humans, transhumans and AIs? I assume that a
transhuman is an evolved/augmented form of a human in many respects, and
thus shares many goals with humans, especially in the early near-human
stages. So my question is as applicable to them as to "mere" humans. I
am worried when I hear about a proposed AI that is so much smarter than
all humans (and transhumans for that matter) that it can and should make
decisions for all of us that we should (will be forced to?) obey without
questioning especially since our puny brains/processing units cannot
match Its Intelligence. This is not a very palatable goal or state for
humans or for transhumans or for any other category of being that
treasure its independent ability to make its own mistakes and to
evolve/grow over time. It is a capping, an utter topping, an asymptote
reached, that makes other forms relatively irrelevant and insignificant
and would lead them imho to be rather dis-spirited unless the super AI
very carefully left trans/humans freedom and room to develop.

I also have a problem with opposing transhumans to humans. It is the
classic problem of the elite. There is a large problem of whether and
how space is left for those who do not wish to follow the transhuman
path.

- samantha



This archive was generated by hypermail 2b29 : Mon Oct 02 2000 - 17:38:57 MDT