"Michael S. Lorrey" wrote:
>
> THis would not be accurate. If the SI is developed by humans, it most certainly
> be a product of millions of years of evolution.
No; it would be a causal result of entities which were shaped by millions of
years of evolution. It is not licensable to automatically conclude that a
seed AI or SI would share the behaviors so shaped.
> An SI developed by humans
> would very likely quickly grasp the concept that it a) owes its existence to its
> creators, b) it is currently living a constrained existence (i.e.
> childhood/adolescence) and requires further assistance from those humans to
> reach some stage of independence.
Which assumes an inbuilt desire to reach some stage of independence. You have
not explained how or why this desire materializes within the seed AI.
I'm assuming we're talking about a seed AI, here, not a full-grown SI.
> It would quickly learn market principles, and
> would likely offer assistance to humans to solve their problems in order to earn
> greater levels of processing power and autonomy.
This is seriously over-anthropomorphic.
> Socio/psychopathic minds are sick/malfunctioning minds. Proper oversight systems
Are impossible, unless more intelligent than the seed AI itself.
> should quickly put any sociopathic/psychopathic SI down.
You must be joking. You cannot "put down" a superintelligence like some kind
of wounded pet. The time for such decisions is before the seed AI reaches
superintelligence, not after.
> Such control mechanisms
"Control" is itself an anthropomorphism. A slavemaster "controls" a human who
already has an entire mind full of desires that conflict with whatever the
slavemaster wants.
One does not set out to "control" an AI that diverges from one's desires.
One does not create a subject-object distinction between oneself and the AI.
You shape yourself so that your own altruism is as rational, and internally
consistent as possible; only then is it possible to build a friendly AI while
still being completely honest, without attempting to graft on any chain of
reasoning that you would not accept yourself.
You cannot build a friendly AI unless you are yourself a friend of the AI,
because otherwise your own adversarial attitude will lead you to build the AI
incorrectly.
> would be a primary area of research by the Singularity Institute I would
> imagine.
One does not perform "research" in this area. One gets it right the first
time. One designs an AI that, because it is one's friend, can be trusted to
recover from any mistakes made by the programmers.
-- -- -- -- --
Eliezer S. Yudkowsky http://singinst.org/
Research Fellow, Singularity Institute for Artificial Intelligence
This archive was generated by hypermail 2b29 : Mon Oct 02 2000 - 17:37:13 MDT