Re: Galaxy Brain Problem

Nicholas Bostrom (bostrom@mail.ndirect.co.uk)
Sun, 17 Aug 1997 02:26:58 +0000


Robin Hanson wrote:

> Anders Sandberg writes:
> >> As someone in another thread pointed out, game theory does not
> >> apply to post-singularity entities.
> >
> >Huh? Could this someone please explain why it would not apply after
> >the singularity? Many situations in game theory will not be changed
> >if the players are ultra-intelligent (in fact, game theory often
> >assumes the players are very rational, more rational than most humans
> >are).
>
> Let me echo Anders here. The relevance of game theory isn't going to
> go away, it should get stronger. Not only will players get more
> rational, but they should be able to better manage firewalls between
> their internal plans and beliefs and the external image they portray.

I agree. However, I think there is a sense in which the opposite may
well be true. (It was not I who made the original statement, though.)
I think that game theoretical considerations *might* show (I am not
at all certain of this yet) that there will most likely be only one
superintelligence originating from earth. The sense in which game
theory would not apply after the singularity is that there might only
be one agent, and the game he is playing against Nature may be such
that game theory is not at all useful in explaining it.

What I said in another thread might be relevant here (please forgive
me for quoting myself):

"We should be on our guard against possible biases that
might make us inclined to believe in something just because it would
be interesting it it were so. (Is this bias toward intellectual
interestingness our equivalent to emotional biases more prominent in
other groups?)"

Nicholas Bostrom
London School of Economics
Department of Philosphy, Logic and Scientifc Method
email: n.bostrom@lse.ac.uk
homepage: http://www.hedweb.com/nickb