"Eliezer S. Yudkowsky" <sentience@pobox.com> writes:
> Prosecutor: You're accused of killing eight nuns with an icepick.
Hmm, sounds a bit like the problem with "truth" machines. How can we
be sure they are never wrong?
Of course, using neural networks for sifting through databases and
getting an opinion can be very useful. The trouble seems to be that
people tend to offload thinking and responsibility too easily.
> Defendant: What's the evidence?
> Prosecutor: Our neural network says you did it.
> Defendant: Why does it say that?
> Prosecutor: Um, nobody knows.
> Defendant: That's it? That's your evidence?
> Prosecutor [to jury]: Hey, look at all these cool blinking lights!
> Technology! Whoo-ah! It's never wrong!
> Jury: Guilty.
> Judge: I sentence you to death.
Anders Sandberg Towards Ascension!
asa@nada.kth.se http://www.nada.kth.se/~asa/
GCS/M/S/O d++ -p+ c++++ !l u+ e++ m++ s+/+ n--- h+/* f+ g+ w++ t+ r+ !y