Re: Singularity: Human AI to superhuman

Emmanuel Charpentier (
Mon, 14 Sep 1998 03:24:06 -0700 (PDT)

---"Eliezer S. Yudkowsky" <> wrote:
> Emmanuel Charpentier wrote:
> >
> > ---"Eliezer S. Yudkowsky" <> wrote:
> > >
> > > I disagree with the basic concept that human brains are
> > > based on neural nets.
> >
> > ... What else do you propose for memory, processes, learning,
> > pain/pleasure taking place in you and me.
> Perhaps it would be better to say that "association" is not the
foundation of
> thought. The source of "pattern", in memory, learning, pain and
> does not derive from the associational nature of neural networks,
but programs
> which use neural networks for processing power. A brain is not
> built on neurons any more than a spreadsheet is built on silicon
atoms. The
> properties of small human-built neural networks, such as the
association of
> features, will not necessarily show up as high-level properties of
the human brain.

You take the side of cognitive psychology: the brain supports high level functions, functions we can discover, analyse and eventually synthetise. Only there is not a set of such functions, we only end up with high level descriptions that don't account for all of our features.

On the other side, we can try to copy the (natural) neural network. The artificial neural networks that we can program nowadays are so simple (and yet so effective in some tasks) that we can easily predict the emergence of many more features. Memory, analogy, imagination, semantics, intuition... etc.

How do you program analogy? How do you translate sentences from russian to english (and back:-)? How do you make poesy?

One more thing, natural neural network can hold conflicting beliefs: I can believe that the earth is flat (sitting atop of four giant elephants atop a great turtle) and try to calculate its radius using angles of the sun shadows in deep wells. No problem. -I/we- can be unconsistant! (and so easily) And it's a great feature, because finally, when you look at science, it's only a set of beliefs, some of which might conflict between each other (until better beliefs come into play).

So, why do you want to program a perfect AI? And how do you manage unconsistency and/or uncompleteness (not having all/enough data)?

> > > Human
> > > brains use more powerful principles.
> >
> > You need to give me some hinsight here. I don't see what you mean.
> For example, I think that the cerebellum performs some type of
> propagation, or rather constraint assembly, and that symbolic memory
is based
> on abstracting a set of high-level constraints which the cerebellum
> While the constraint propagation is almost certainly optimized on
the neural
> level, there are no "neural networks" I am aware of that perform
> propagation, since that activity is fundamentally distinct from
> as we know it.

It looks to me you will end up in a wall: how do you manage to link together two subsystems (domdules) that collide. One time or another, just like rule based systems, you will end up with conflicts/unconsistency.

> > Markov nets would probably do a better job at it, at least it
> > allows to 'associate' things together!
> I doubt very strongly indeed that the memory/symbolic domdule
(equivalent of
> our hippocampus) could be implemented by a simple Markov net.

The animal brain doesn't use symbols. You approach your hand to the fire, it hurts, you don't do it again, that's all. I know it's simplistic. But the next time you see a fire, you might think about your hurt hand, imagine what would happen to your body if it was set to fire, you might experience fear (a great feature again, that you want to implement in an AI), and there is no function for that.

Symbols are for communication: computers don't use symbols! Internally, because externally they need to express their internal states through sometimes imperfect means. Sometimes it's drawings that we call letters, sometimes those letters are called number, sometimes it's by sounds.

> > Come on, human body and brain do it all the time. That's what
> > happen when you become an expert on a task: you don't need to think
> > about it! It's wired!!! And you didn't answer about perfection: you
> > can't design perfection into an AI, and have that AI work its way
> > around in an unperfect (from our models point of view) universe!!!
> You're confusing high-level "perfection" with low-level
"perfection". When
> was the last time your neurons got confused over a matter of
philosophy? When
> did your neurons get bored with firing? You run on an AI Advantage,
but you
> can't use it consciously - can't tell your neurons to multiply two
> twenty-digit numbers for you, even though they could do it in a
second; you
> have to use your entire brain and probably a paper-and-pencil to
perform this
> simple procedure, and even then you'll drop a digit or two. AIs
will still
> use error-prone high-level conscious thought to solve uncertain
> they'll simply have the capability of pouring in massive amounts of
> procedural thought when necessary.

What you want is to reprogram, rewire, a brain. Some people (let's call them experts) can multiply two big numbers together, in a blink! And we all have the potential to, only it's not easy, it's possible we would have to decrease some other abilities, or learn numbers in a different fashion. How do you call a person with a great ability in a domain, but the inability to live on their own in society, or even in their home?

Some people can do very complex things without thinking about it (in a conscious manneer), while I might have to think about typing on a keyboard, some people type without thinking... While I'm not good at karate, some people will kick an agressor without thinking about it.


Get your free address at