Re: Seed AI

Eliezer S. Yudkowsky (sentience@pobox.com)
Thu, 01 Jul 1999 01:44:29 -0500

"O'Regan, Emlyn" wrote:
>
> Eliezer,
>
> The seed AI concept has been rattling around in my head for a while. What's
> buggin me is the concept of environment. I asked you what environment you
> expected would be necessary for the AI a while back, and you said that the
> environment would be the AI's code itself

Yup.

> (I think - I could have missed
> your point, please have patience with a mere mortal).
>
> Absolutely, this must be the core of the environment. But there must be more
> than this, mustn't there? For an AI to optimise itself, there must be some
> definition of optimal, which implies a frame of reference. I think that
> frame must be external to the code, because the idea of optimising you code
> to make you better at optimising your code has an unfortunately circular and
> empty feel to it.

Only to the same mathematicians who gave us propositional logic. I mean, it sounds circular in theory, but in practice, it doesn't work that way, for a very simple reason which, unfortunately, doesn't translate into English without a blackboard. What I'd like to say is that the AI is reductive and the elements present obvious suboptimization metrics, and that the elements sum to noncodic abilities as well as codic abilities, so codic optimization is non-sterile.

Okay, try this. The AI isn't composed of a single, "code-optimizing" domdule, right? It's composed of a causal analysis module and a combinatorial design module and a heuristic soup module and so on. These architectural modules, plus analogies to other application domdules, plus the codic domdule, all sum to the "code-optimizing" ability. In a given optimization problem, you have subproblems that are spread across the domdules. The performance on the subproblems, and the contribution of individual domdules to the success on subproblems, allow for local optimization.

No, I'm still being too complex. Remember EURISKO? It had heuristics optimizing heuristics? There were even heuristics being optimized to be better heuristic-optimizers? It wasn't sterile. Why? Because the heuristics were also being optimized for all sorts of other problems. And, more importantly, because "Examine nearby cases" being applied to "Investigate extreme cases" to yield "Investigate cases close to extremes" is a lot more specific than "optimize heuristics for heuristic-optimizing". The general case sounds sterile and circular because it's monolithic and general. But in actuality, you have sub-abilities dealing with subproblems.

You don't even need any other kinds of problems at all. The subproblems of the general problem of self-optimization provide enough diversity to prevent the circularity you're worried about. The major reason for programming other environments would be to provide sources of analogies and incremental paths to ideas that would bottleneck otherwise - the same reason a hacker learns languages ve'll never program in. But it's not *necessary*.

-

BTW - the surrealism involved in saying "Stop holding on to the past" to a Singularitarian is considerably larger than that involved in saying it to a 19-year-old. 

-- 
           sentience@pobox.com          Eliezer S. Yudkowsky
        http://pobox.com/~sentience/tmol-faq/meaningoflife.html
Running on BeOS           Typing in Dvorak          Programming with Patterns
Voting for Libertarians   Heading for Singularity   There Is A Better Way