Re: FAQ Additions

Nick Bostrom (bostrom@ndirect.co.uk)
Sun, 21 Feb 1999 15:58:35 +0000

Eliezer S. Yudkowsky:

> I suggest that the section:
>
> ==
> In the first case, we could make sure that the values of tolerance and
> respect for human well-being are incorporated as core elements of the
> programming, making part of an inviolable moral code.
> ==
>
> Be amended to read:
>
> ==
> In the first case, we could make sure that the values of tolerance and
> respect for human well-being are incorporated as core elements of the
> programming, making part of an inviolable moral code. (However, some
> think this would be a hideous mistake from a <a
> href="http://tezcat.com/~eliezer/AI_design.temp.html#PrimeDirective>programming</a>
> perspective; some also question the morality of such an action.)

That looks like the kind of comment for which CritLink would be ideal.

> This faction is composed at least of Billy Brown in addition to myself.
> I'm not sure, but I think this might also be Hans Moravec's position.

Actually, in his latest book "Robot", Moravec explicitly proposes that laws require that robots be built "securely nice in the first place". "Every neuance of their motivation is a design choice. They can be constructed to enjoy the role of servant to humankind." And it is a "matter of life and death to humans" that they "do not have a right to vote on the laws that govern and tax them." (pp139-40). In the long run, Moravec looks forward to advanced robots superseding humans and taking over the universe, but we both can and should ensure a comfortable retirement for humanity by programming in suitable "internal laws" in our mind children.

> I further suggest that the section: "Won't it be boring to live forever
> in the perfect world?" be amended to contain the following statements
> from "Great Mambo Chicken", p. 97:
>
> "Ordinary life is sometimes boring. So what?"
> "Eternal life will be as boring or as exciting as you make it."
> "Is being dead more exciting?"
> "If eternal life becomes boring, you will have the option of ending it
> at any time."

Good suggestion. I will incorporate that.

Nick Bostrom
http://www.hedweb.com/nickb n.bostrom@lse.ac.uk Department of Philosophy, Logic and Scientific Method London School of Economics