Re: Is this safe/prudent ? (was Re: Perl AI Weblog)

From: Anders Sandberg (asa@nada.kth.se)
Date: Wed Aug 13 2003 - 03:48:10 MDT

  • Next message: Brett Paatsch: "Re: Is this safe/prudent ? (was Re: Perl AI Weblog)"

    On Wed, Aug 13, 2003 at 01:19:20AM -0700, Samantha Atkins wrote:
    >
    > Existence proof please. Could we please have an AI that even
    > remotely approaches human level intelligence in practice
    > *before* we start talking about restricting the field in
    > various probably draconian ways?

    If the seed AI exponential self-amplification scenario is true,
    then this approach will of course be too late. In that scenario,
    the only policies seem to be to call for draconian restrictions,
    centralising the project (into whose hands?) or have a
    free-for-all to become Mother of God.

    The problem is indeed that we do not have any good theory or
    evidence that the seed AI scenario is feasible or likely. This
    is where IMHO *we* really can make a difference, by doing good
    thinking and research into the issue. If one can show seeds are
    unlikely (say by demonstrating that general intelligence is
    computationally very hard to enhance) or possible (say by a
    existence proof that such a program can exist) then the
    discussion becomes much more informed.

    The precautionary principle in its active form suggests that we
    should actively find ways of protecting ourselves from the risks
    of a bad AI even if it is an unknown factor; friendliness theory
    is a start, building institutions is another. But that
    shouldn't distract us from actually getting a better knowledge
    of the problem.

    -- 
    -----------------------------------------------------------------------
    Anders Sandberg                                      Towards Ascension!
    asa@nada.kth.se                            http://www.nada.kth.se/~asa/
    GCS/M/S/O d++ -p+ c++++ !l u+ e++ m++ s+/+ n--- h+/* f+ g+ w++ t+ r+ !y
    


    This archive was generated by hypermail 2.1.5 : Wed Aug 13 2003 - 03:53:49 MDT