Re: Seed AI

Eliezer S. Yudkowsky (
Fri, 02 Jul 1999 16:56:42 -0500

Christopher Maloney wrote:
> Sending this again ... sorry if it gets duplicated, I don't know
> what the deal is. I've waited 16 hrs and it didn't show up.
> Eliezer S. Yudkowsky wrote:
> >
> > Okay, try this. The AI isn't composed of a single, "code-optimizing"
> > domdule, right? It's composed of a causal analysis module and a
> > combinatorial design module and a heuristic soup module and so on.
> > These architectural modules, plus analogies to other application
> > domdules, plus the codic domdule, all sum to the "code-optimizing"
> > ability. In a given optimization problem, you have subproblems that are
> > spread across the domdules. The performance on the subproblems, and the
> > contribution of individual domdules to the success on subproblems, allow
> > for local optimization.
> It sounds like one could argue that that is all humanity is
> engaged in now, and ever has been. In a way, everything is
> a "sub-problem" of the problem of advancement to the next
> level. Or is that stretching it too much?

Yes, it is stretching it too much. When I say "sub-problem", I'm not talking about sub-goals. I mean that a large problem breaks up into a lot of little problems which require qualitatively different types of thinking, just as a corporation breaks up into a lot of different specialties. Because each specialty can be optimized independently - by all of the abilities working in concert - there isn't the sterility/bottleneck problem you'd get from one ability working to optimize itself.

> I mean, when I think about your suggestion, I can imagine what
> (for example) my wife would say. Something like "what a dry
> and sterile existence"! But I would argue, and maybe you would
> too, that the "sub-problems" are really what life is all about.
> And I always think it's rather Anthro-centric to think that
> appreciating art and beauty and so forth are peculiar human
> abilities.

Actually, I'm still not quite sure where beauty and music and laughter come from. I think it has something to do with sexual selection, but I don't know enough to duplicate it. They don't get any more peculiar than that.

> I personally think that they are just ways of
> thinking about patterns and analogies, an ability which an AI
> will certainly have. I.e., "sub-problems"!

But then why do they cause joy? And is the joy of seeing a beautiful MotOS the same kind of joy as seeing an elegant theorem? If not, which category does the joy of music belong to, and does it matter whether the music is "Holding Out for a Hero" or "Little Fugue in G Minor"?

(MotOS = Member of the Opposite Sex)

           Eliezer S. Yudkowsky
Running on BeOS           Typing in Dvorak          Programming with Patterns
Voting for Libertarians   Heading for Singularity   There Is A Better Way