>What should one ask of such a machine?
`Guide me, O Lord. Sincerely, yr hmble & obdt svt.'
>What *shouldn't* one ask for?
The way to San Jose.
Um, well, the standard trick question posed to a demon offering three
wishes (or even one) is this: `Kindly satisfy every wish I utter
henceforth' (see, e.g., STRIPED HOLES, by yr hmble & obdt svt). Some have
supposed that the toxic self-reference of this instruction voids the
guarantee. In the case of an AI/nano genie, however, its effectiveness can
be taken for granted, except that the explicit statement of one's wishes is
not required and their content would be unnecessarily limited in any case.
Damien Broderick