From: Anders Sandberg (asa@nada.kth.se)
Date: Wed Aug 13 2003 - 11:50:06 MDT
On Wed, Aug 13, 2003 at 09:45:55AM -0700, Samantha Atkins wrote:
> On Wednesday 13 August 2003 02:48, Anders Sandberg wrote:
> >
> > If the seed AI exponential self-amplification scenario is true,
> > then this approach will of course be too late. In that scenario,
> > the only policies seem to be to call for draconian restrictions,
> > centralising the project (into whose hands?) or have a
> > free-for-all to become Mother of God.
>
> True. But locking the barn door before the horse exists hardly seems like
> reasonable policy. Too much paranoia now about some theoretical scenarios
> could very well rob us of the many benefits of all levels of AI, benefits
> that are very much required if we are to have the intelligence to chart our
> course wisely.
Exactly. The problem is to keep a balance between worrying about
possibilities and waiting too complacently. If we show that we care about
the risks and want to minimize them, many people react favorably - but too
much disaster scenarios may make people fearful. Showing that many worries
are groundless is important to help research develop, but we should not be
polyannas assuming everything automatically turns out the best.
> > The precautionary principle in its active form suggests that we
> > should actively find ways of protecting ourselves from the risks
> > of a bad AI even if it is an unknown factor; friendliness theory
> > is a start, building institutions is another. But that
> > shouldn't distract us from actually getting a better knowledge
> > of the problem.
>
> The precautionary principle is one of the worst notions I have ever
> encountered as it proposes to base policy on the ability to prove a negative.
Actually, in its simple and correct form it is just common sense. What you
react to is how it is *usually* used - "don't do anything until you have
proven it safe!" which is actually a misinterpretation of it (just look at
how it is stated in the Rio Protocol, as an example). The corrupted
version is unfortunately the one that is used most widely by conservatives
of all colors. The active form I mentioned (absence of scientic proof of a
danger is no reason not to proactively minimize dangers) can of course be
used to stifle development too, but judiciously used it makes a lot of
sense when venturing into the unknown.
-- ----------------------------------------------------------------------- Anders Sandberg Towards Ascension! asa@nada.kth.se http://www.nada.kth.se/~asa/ GCS/M/S/O d++ -p+ c++++ !l u+ e++ m++ s+/+ n--- h+/* f+ g+ w++ t+ r+ !y
This archive was generated by hypermail 2.1.5 : Wed Aug 13 2003 - 11:57:55 MDT