From: Samantha Atkins (samantha@objectent.com)
Date: Wed Aug 13 2003 - 10:45:55 MDT
On Wednesday 13 August 2003 02:48, Anders Sandberg wrote:
> On Wed, Aug 13, 2003 at 01:19:20AM -0700, Samantha Atkins wrote:
> > Existence proof please. Could we please have an AI that even
> > remotely approaches human level intelligence in practice
> > *before* we start talking about restricting the field in
> > various probably draconian ways?
>
> If the seed AI exponential self-amplification scenario is true,
> then this approach will of course be too late. In that scenario,
> the only policies seem to be to call for draconian restrictions,
> centralising the project (into whose hands?) or have a
> free-for-all to become Mother of God.
True. But locking the barn door before the horse exists hardly seems like
reasonable policy. Too much paranoia now about some theoretical scenarios
could very well rob us of the many benefits of all levels of AI, benefits
that are very much required if we are to have the intelligence to chart our
course wisely.
>
> The problem is indeed that we do not have any good theory or
> evidence that the seed AI scenario is feasible or likely. This
> is where IMHO *we* really can make a difference, by doing good
> thinking and research into the issue. If one can show seeds are
> unlikely (say by demonstrating that general intelligence is
> computationally very hard to enhance) or possible (say by a
> existence proof that such a program can exist) then the
> discussion becomes much more informed.
>
> The precautionary principle in its active form suggests that we
> should actively find ways of protecting ourselves from the risks
> of a bad AI even if it is an unknown factor; friendliness theory
> is a start, building institutions is another. But that
> shouldn't distract us from actually getting a better knowledge
> of the problem.
The precautionary principle is one of the worst notions I have ever
encountered as it proposes to base policy on the ability to prove a negative.
- samantha
This archive was generated by hypermail 2.1.5 : Wed Aug 13 2003 - 10:53:08 MDT