From: Adrian Tymes (wingcat@pacbell.net)
Date: Sun Mar 09 2003 - 19:46:36 MST
--- "Michael M. Butler" <mmb@spies.com> wrote:
> Adrian Tymes <wingcat@pacbell.net> wrote:
> > You have a *chance* of dying. You could, totally
> by
> > accident, wind up doing the right thing anyway.
> This
> > is not the same thing as guaranteed death.
>
> And the combinatorics could wind up resulting in a
> likelihood smaller than
> the quantity 1 over the number of seconds that have
> elapsed since the Big
> Bang. That's not the same as guaranteed death, but
> it's the way to bet if
> those are the odds. So are odds of 1 in 100.
>
> Adrian, just for my couriosity's sake, how do you
> figure
> the odds? I'm not just asking that question
> rhetorically.
To be honest, I don't. I haven't gone through the
effort to make a hard statistical analysis, largely
because most of the data this analysis would be based
on is so fuzzy as to make the analysis near-useless
for
convincing anyone.
What I have done, however, is draw parallels to my own
knowledge and study of intelligence and power, and
make
the assumption that AIs of any import to the world
would live in the same reality as I have. Which means
that similar solutions for granting other people power
work, for much the same reasons, if you make the
assumption (which I believe most people would admit)
that AIs are, essentially, "people" with certain
enhancements (for instance, the ability to calculate
much much faster, the ability to hold more points of
data - though still a finite number, however large -
in
their minds, and so forth; the exact list can be
argued).
Which means that an AI could be sociopathic (i.e.,
uninterested in mere human survival, or even opposed
to
it as in so many Hollywood flicks), greedy,
solipsistic
(i.e., refusing to believe that we humans are
sentient,
even if it were proven that we created it), and so
forth, like any person. If that is the case, then
planning on the case where you make an AI without
those
flaws is, itself, flawed: while you could never
succeed
in that goal, you *could* be pressured to declare
success, pretend you have succeeded (and thus, do not
need to defend against those flaws), and move on to
grant the AI power. Which is a scenario for disaster.
On the other hand, if you account for that and treat
the AI like a human being, with potentials for flaws
imagined and unimagined, then you can still set up
systems where, no matter how intelligent the AI is, it
can not magically outthink you and gain freedom until
you, or someone, trusts it. And to gain your trust,
it
would have to emulate friendly (dare I even say
Friendly?) practices. The only real data point we
have
for such scenarios is human behavior, and the human
data shows that, the vast majority (easily over 99%)
of
the time, a human being who fundamentally acts
friendly
really is friendly. This trend does not go down with
increasing intelligence; if anything, it goes up -
which would seem that the odds of a seemingly-friendly
AI actually being friendly would be even higher.
Besides, any plan for dealing with and developing AIs
must take into account the abilities of the humans
developing it, else the plan can not be put into
practice (though, as described above, fatally flawed
versions of the plan can be; sadly, this happens all
the time in politics and business). Humans do not
always have the ability to thoroughly review and
understand programs, especially nontrivial programs.
(For this, I cite my own professional knowledge.) But
they do have the ability to evaluate human-like beings
as if they were humans (cite anthropormorphism
throughout the ages).
> Against that, I think a case can be made that, for
> issues as hairy as this,
> Bayes isn't applicable in a vacuum; we don't get to
> refine our estimates
> over multiple trials that include catastrophe--maybe
> the human race does
> (if
> we luck out), maybe not, but probably not we here
> assembled. Conflict
> levels
> of 11 (that's 10^11 casualties--hundreds of millions
> dead) /or more/ also
> surely change the weight.
And what are the odds of such a conflict if we do
*not*
go this route? Neither way prevents or guarantees it;
it just makes the outcome more or less likely, over
whatever time span you wish to designate significant.
> >> Do you have any ideas for dealing with this
> besides
> >> building FAI first? Because as far as I can tell,
> humanity is in
> >> serious, serious trouble.
> >
> > Yes. Build an AI that you can trust, even if it
> > quickly goes beyond your control.
> ...
> > perspective), but if you can trust someone else to
> be
> > gifted with the capabilities that an AI would
> have...
>
> I agree with what you're saying but (and perhaps
> this is also what you're
> saying) the only way I can see to apply the level of
> trust I give to a
> person is to guarantee the physical limitations and
> the
> developmental psychological ground of an actual
> human.
That's certainly one way to do it. In fact, one
scenario I think rather likely to work - and to
increase our chances of this being done in a way that
guarantees the survival of most presently human
sentient beings - is to find a way to augment human
intelligence into AIs. Certain elites will gain this
capability before most people; that can not feasably
be
avoided. But if this power can be allowed to spread
rapidly and freely (despite the inevitable
restrictions
some of these early adopters, or non-adopter elites
afraid of the power this could bring to non-elites,
will impose), then IMO the only significant risk of
human extinction would be to the mere biological
shells
we presently inhabit. And even a good number of those
would likely survive, at least for a time, by uploads
with a sentimental attachment to the past. (If
nothing
else. Mmaybe the first uploads will be upgraded
organic circuitry, using some of the best life-support
equipment for such things we have available today:
already grown, living bodies, like the ones the brains
came from in the first place.)
> I wonder how likely this is, given the fraction of
> people with technical
> chops that would dismiss this out of hand as
> [quasi]paranoid.
> We internet-enable hot tubs, for goodness' sake. So
> the first aspect
> (physical limitations) is likely to be ignored in
> some way by even the
> "embodied consciousness" folks.
You also have to consider their arguments in light of
their likelihood of sucess. Intelligence, or at least
that which we are trying to replicate here (whatever
its true nature, and whether or not it really has a
name as yet), is defined as a uniquely human trait (at
the moment). Biomimicry has proven successful in many
other fields, and I would argue that AI is no
exception
- if you do not define Eliza and similar chatbots as
"successful". The most robust, adaptable, and (again,
I would argue) true intelligence to date has been the
very close mimicing of bugs' control algorithms, and
there is every reason to believe that this approach
will scale up at least to human levels (existence
proof: it's done so with organic evolved products, and
the main limitation to doing it in artificial products
is one which Moore's Law looks like it will beat down
in due time - assuming it holds long enough, and
despite the hue and cry from certain sectors about its
imminent failure, most of the evidence suggests that
it
will).
All of which is to say, when true AIs finally do
arrive, the odds are that they will have something
like
a human upbringing. And given that the odds favor
this
outcome as it is, actions (including strenuous debate)
to prevent any other approach are of diminished value
in achieving their approach, while their costs
(including, for some actions, retarding potential
advances along any approach to AI, even the preferred
one) remain unaffected. Therefore, it is best to
allow
even the "likely to produce monsters" paths to
succeed,
such that the "likely to produce what we want" paths
may benefit from the results.
All of which is a long way to say: Don't Panic.
This archive was generated by hypermail 2.1.5 : Sun Mar 09 2003 - 19:51:51 MST