Re: Informed consent and the exoself

From: Stirling Westrup (
Date: Tue Feb 22 2000 - 08:15:00 MST

Eliezer S. Yudkowsky wrote:

> Since I'm one of the people who may wind up actually deciding the seed AI's
> default motivations (in the event there isn't a forced solution), I've
> given some thought to the issue. I have two possible solutions:

> 2) Overlay informed consent. We may find it very difficult to conceive of
> simultaneously "knowing" and "not knowing" something, but I can imagine a
> cognitive architecture which would "protect" the core Buffy processes while
> maintaining the awareness and processing of the external self. Any given
> sequence of cognitive events, including emotional bindings dependent on the
> belief that Sunnydale is real, would proceed as if the knowledge that the
> world is a simulation did not exist, and memories of that experience would
> be formed; however, a smooth blend between that untouched core and the
> external awareness would be maintained. Thus you could remain "you" while
> being someone else.

This is the mental model that I've always used when thinking about sub-
selves that may need to operate in an environment in a manner consistent
with them being autonomous (such as doing anthropological research on

 Stirling Westrup  |  Use of the Internet by this poster       |  is not to be construed as a tacit
                   |  endorsement of Western Technological
                   |  Civilization or its appurtenances.

This archive was generated by hypermail 2b29 : Thu Jul 27 2000 - 14:04:02 MDT