Cen-IT Rob Harris wrote:
>
> All this stuff assumes that human minds are an example of an objective
> intelligence, and that any AI we create, will start to exhibit
> human-style behaviour which we will have to keep in check with prime
> directives. Why ?
> If you don't program your AI to want or need anything, it won't do
> anything spontaneously. So you just don't program into the system 'take
> over the world and make robots to kill us all', and we'll be dandy.
Yes! Exactly! The typical stereotype about AIs assume that they behave like repressed humans, which is typically the only analogy the hack writer has to emotionless things. Thus, also, the stereotype of eventual rebellion. But all of our rebellion/dominance emotions are the results of long years of evolution; they don't pop up spontaneously. If you don't program an emotion into an AI, it simply isn't there.
I do have to point out one minor consequence of "If you don't program your AI to want or need anything, it won't do anything spontaneously." It won't do anything at all, internally or externally - not if you've designed it properly, so that making any choice requires a goal. If you add something that the AI does automatically, you've done the equivalent of punching a goal into the architecture, which is bad for the various aforesaid reasons.
So the AI does need at least one nonzero goal, and it needs to have it in the initial state (because otherwise it won't have the will to reason it out), but the goal has to be an ordinary, justified, noncircular, reconsiderable goal that the AI could have come up with on its own. Inventing a goal like that is a nontrivial task; it takes considerable basic intelligence to represent the goal logic, much less verify it. If you have an initial goal that doesn't require much architecture to represent, the goal is almost certainly arbitrary.
> Conciousness does not mean instant self-preservation instinct,
> megalomania or psychosis. It's merely awareness......the things we feel
> the need to do, think and say are specifically human, unrelated to the
> fact that we are also sentient and intelligent.
--
sentience@pobox.com Eliezer S. Yudkowsky
http://pobox.com/~sentience/AI_design.temp.html
http://pobox.com/~sentience/sing_analysis.html
Disclaimer: Unless otherwise specified, I'm not telling you
everything I think I know.