hal@finney.org wrote:
>
> Eliezer writes:
> >
> > Actually, I have this wacky idea that says it's ethically prohibited to
> > apply Bayesian priors to people. In other words, you should judge a
> > person's properties - intelligence levels, for example - only by
> > information that is directly revealing of those properties, and not
> > through other properties that happen to be weakly or strongly associated,
> > especially if those properties happen to be outside the person's direct
> > manipulative control.
>
> That's a generous position, to judge each person only by their actions
> rather than other attributes. No doubt in your own case since you are
> a highly unusual person, you would be frequently misjudged when other
> characteristics are used.
Totally, unequivocally CAUGHT over on this end. Yes, this particular
ethical heuristic is largely derived from the way I wish I'd been treated.
> > For example, if a nine-year-old were to walk up to
> > me and ask a question that I would take seriously if an adult had asked
> > it, I am obliged to answer as I would an adult, and am not permitted to
> > take any action predicated on my knowledge of chronological age until the
> > person in question makes some statement that is actually revealing of
> > nine-year-old characteristics. (Not in the sense of "being interpretable
> > as being revealing of youth", but in the sense of being a statement that
> > would *only* have been generated by a youth and not by an adult, or a very
> > high-probability equivalent of same.)
>
> I suspect that you're going to end up talking over the heads of a lot of
> nine year olds with this policy. A more practical approach would be to
> ask a few clarifying questions to get an idea of the child's knowledge
> and maturity before attempting to answer. Maybe the kid really is bright
> and knowledgeable and would appreciate a detailed response, but maybe he
> just accidentally asked a deep question (three year olds are especially
> good at this).
If you talk over the "kid's" head (I said "nine-year-old", not "kid"),
she'll go "Huh?" or "What?" or presumably make some other comment
indicative of nonadulthood. Until she does so, I wouldn't want to ask "a
few clarifying questions" that I wouldn't ask an adult. Wouldn't you be a
bit offended if you asked me a question about Friendly AI and I said
"Well, do you know how computers work?" That's one of the reasons the
ethical action differs from the Bayesian prediction - there's a
qualitatively different penalty for underestimation relative to the
penalty for overestimation.
> > So if someone says "I'm smarter than you, Eliezer!" I would certainly feel
> > obliged to assume (or rather, act under the assumption) that the person in
> > question is as likely to be smarter than me as I am to be smarter than
> > them, at least until the person makes some other statement revealing of
> > actual intelligence levels, which generally doesn't take too long.
>
> ...assuming that you judge such a claim to be indicative of smartness,
> which would presumably depend on the context.
No, I just judge the claim to be an ethical, if not Bayesian, indicator of
symmetry. Human ethics is to some extent about the behaviors that you
implement yourself because you want them to be implemented globally, and a
global conduct needs some way of resolving symmetrical statements about
intelligence. If a 10,000,000:1 genius confronts a 100,000:1 genius,
their Bayesian priors may be quantitatively different, but their
self-sensory experiences of cognitive structures ("It is improbable that I
will run into someone smarter than me") are qualitatively identical.
-- -- -- -- --
Eliezer S. Yudkowsky http://singinst.org/
Research Fellow, Singularity Institute for Artificial Intelligence
This archive was generated by hypermail 2b30 : Mon May 28 2001 - 10:00:02 MDT