Re: The AI revolution

Anders Sandberg (
02 Jul 1998 17:44:41 +0200

Brent Allsop <> writes:

> Hara Ra <> commented about Anders reference to
> Asimov's laws for robots:
> > So, what if a robot has this choice:
> >
> > Kill someone, and allow 100 others to live, or
> > not kill, and allow the 100 others to die.
> >
> > This would probably immobilize the robot, which is the worst choice,
> > so the Zero'th Law is:
> I would think that robots simply be subject to the same laws
> (or law) that we all try to adhear to. And that is simply to do the
> best possible. Part of that law is, to rationally reason and figure
> out what that best possible is as best as is possible.

But what is best? You have to supply the robot with valuations in order to have this kind of reasoning. Asimov's laws have the advantage of being clear what a robot may and may not do, and do not require open ended reasoning ("... but if I save him, what if he is a killer? But if he is a killer...").

> I think a
> robot could logically calculate that a person living is better than a
> person dieing and by induction that 100 people living and only one
> dieing is better than one person living and 100 dieing.

This kind of reasoning was most likely too unconstrained for Asimov or his contemporaries - or anybody building a robot today. Imagine the litigation if your robot does something that leads to the death of somebody, and it is not possible to show that this was a clear logical results of the laws of robotics. People would feel much more at home with a robot that simply couldn't harm them due to the first law, than a robot that just *might* harm them because it had deduced that it was for the best due to some obscure twist of logic.

Anders Sandberg                                      Towards Ascension!                  
GCS/M/S/O d++ -p+ c++++ !l u+ e++ m++ s+/+ n--- h+/* f+ g+ w++ t+ r+ !y