From: Brett Paatsch (bpaatsch@bigpond.net.au)
Date: Fri Sep 05 2003 - 01:49:51 MDT
Emlyn O'regan writes:
> [brett]
> > Just help me with this first bit. How does Emlyn-the-
> > AI become self aware and then go out a hire his first
> > employee or interact with the world in any commercial
> > way. I can see how it might learn a lot as the protégé
> > of an experienced entrepreneur and teacher but
> > when does it usurp the teacher or doesn't it?
>
> There are three obvious paths that I see.
>
> 1 - The AI is kept as a slave, self enhancing until it is
> superintelligent. Super intelligent AIs do mostly what
> they want to; if they want to convince the executive to
> sign over all power in a company to them, they'll do it,
> eventually. You think nobody would be dumb enough
> to let one enhance that far? Those who put the least
> checks on their AIs will probably do best in the short
> term, and how would they know exactly how smart
> their AI was at any point?
slave/tool/pet. If it achieve recognition as a "slave" its legal
battle is probably largely over. I DO think some folks
would be "dumb" enough to let one enhance that far in
terms of general intellectual power if one was a corporate
exec that perhaps inherited a "seedling" that ones
predecessor had been playing with for R&D and that one's
researchers had developed the AI over time and taught to
do something useful in a sort of expert system MIS manner
for sure.
I grant your point on the premiums for risk taking on the
checks and balances, but an AI that produces a commercial
or military or political return would seem to be selected for,
or specifically encouraged to learn in that direction, rather than
one that was just "friendly". I can see the CEO saying "screw
the "friendly" modifications or lessons or extra rule handling
routines or whatever" (sorry for over simplifying Eliezer) and
"just get junior AI here hooked up to the news services and
the stock markets information. When he shows promise
in that direction duplicate him if you can and experiment
on various ways to make the duplicates outperform each
other commerciall." The same guy that encourages junior AI
to develop (gives it electricty and resources ie. hardware)
when junior can't fend for itself is the guy that has a subjective
sence of how "friendly" junior AI needs to be. It needs to be
apparently friendly by his, the ceo lights, not by any other
more general criteria of friendly.
> 2 - AI sympathisers (eg: SingInst?) set up the structure
> on purpose to allow their AI to have autonomy. Only one
> group has to do this, and then the AI might help other AIs,
> or spawn ther AIs, or work to spread the Happy, Shiny,
> Helpful AI meme. I suspect there will always be a subset
> of people willing to assist (potentially) oppressed AIs.
Shades of the abolitionists in the US pre the Civil War.
But with no disrespect to Eliezer or the singularity institute.
I'd still want to make up my own mind on the friendliness
or otherwise of any artificial intelligence purported to be
better at looking after everybody's own good then I'd be
at looking after my own good without surrendering any
of my personal "sovereignty" to it.
Perhaps I'm missing something fundamental in the notion
of "friendly". Perhaps there is some trick for making it
universal that someone has cottoned on to that is not
just *their* notion of how a friendly AI should behave
and be directed by its goals but it actually "objectively"
friendly? -but that seems tricky - help Eliezer ?
> 3 - The enslaved AI is simply so damned good at
> running a company that more and more decision making
> functions are delegated to it over time; management
> automation. It'd make sense; decisions would be far
> more timely and extremely good. Over time, if many
> corporations head down the same path, singularity
> pressure alone would force this choice; you either do it
> or you crash and burn. So no-one sets the AI free
> for moral reasons, it doesn't trick anyone, commercial
> forces just compel this event to happen.
Yep. Ala Moravec's Robot and I think Damien's Last Mortal
Generation.
> Note that in this last case, the management automation
> software need not even be self aware, just really good at
> what it does.
An expert system.
> You could end up with the majority of the world's capital
> controlled by complex, but non-sentient software, with
> decreasing amounts of human input. If these corporations
> become truly effective, they may end up owning controlling
> interests in each other, cutting us out of the loop entirely.
> A few more iterations, and they turn off life support as a
> cost control measure...
Ah I didn't follow the bit where the expert system or better
expert systems amongst competing ones grabbed most of
the worlds assets without running into the regulators of other
countries etc.
But more to the point aren't you making a case for a sort
of expert system become general AI with particular skills
that gets good at "appearing" friendly and bountiful to its
stakeholders, the shareholders in its owning corp, or is
actually really friendly to eveyone somehow, in which
case its perhaps not doing its best by the shareholders?
Goal conflict?
I'm back to my concept of speaking to this AI thats
telling me, "trust me, I know better than you, I've been
well brought up to have no alliances, and to be universally
altruistic, so I'm REAL friendly, and you should just do as I
say and as quickly as you can and all will be A-Ok."
Hmm. Metaphorically speaking -Mr Serpent - I'm not
sure I'd like those "apples".
Brett
Emlyn
This archive was generated by hypermail 2.1.5 : Fri Sep 05 2003 - 01:59:28 MDT