From: Anders Sandberg (asa@nada.kth.se)
Date: Tue Jan 28 2003 - 16:31:16 MST
On Tue, Jan 28, 2003 at 12:10:19PM -0800, Michael M. Butler wrote:
> Anders Sandberg wrote:
>
> >In my own neural network research each state of a neuron can be viewed
> >as a dimension, and the dynamics of my network corresponds to movement
> >of a point through a N-sqrt(N) dimensional space. Rather hard to
> >visualize, unfortunately.
>
> I seem to be doing a lot of replying to Anders today.
>
> So you're saying that, as a vector, each neuron could be fairly notated as
>
> (0,0,0,...0,1,0...0,0)
>
> at any point in time, with the "1" sort of wandering around the vector?
> That's what one dimension per state sounds like to me.
Not really. Put the activities of all neurons as a big vector:
(0,1,0,...,0.4,0.1,0,0.9) would mean that neuron two had activity 1 and
some of the others had mixed activities. The state of the entire system
can be viewed as a vector (or point) in a huge-dimensional space. As the
system changes, the vector changes.
In my network groups of neurons form "hypercolumns" where the sum of the
activities is always one (they correspond to completely covering sets,
like which of ten possible colors a thing is), so the vector is a bit
constrained to subspaces of the big N-dimensional space. In a sense I'm
moving on a product of sqrt(N) sqrt(N)-1 dimensional spaces, if that
makes any sense.
> Pardon my ignorance, but how distinct are the neural states you're talking
> about in *real* neurons? (I know the rejoinder... "What do real neurons
> have to do with neural network research, anyway?" :) )
In real neurons things are *much* messier, since there is no single
state. A pyramidal cell can have parts of its dendrites firing or at
least being quite active while the cell body is silent or firing; it is
probably best to describe it as a large number of tiny compartments
which can each be assumed to have the same potential, chemical balance
and membrane state. I would guess a reasonably detailled cell model has
around a thousand dynamical variables - so the total state space of a
"realistic" neural network would be around 1000N at least (not counting
what the synapses are up to).
As you say, neural network research tends to ignore this kind of
complexity, which is a pity. That way it just turns into statistics with
slightly dated buzzwords, and who cares? But doing real biological
models give you the chance to hit the network with simulated dopamine
while cackling with glee...
BTW, I loved the brinista song! Keep it coming!
-- ----------------------------------------------------------------------- Anders Sandberg Towards Ascension! asa@nada.kth.se http://www.nada.kth.se/~asa/ GCS/M/S/O d++ -p+ c++++ !l u+ e++ m++ s+/+ n--- h+/* f+ g+ w++ t+ r+ !y
This archive was generated by hypermail 2.1.5 : Sun Feb 02 2003 - 21:26:03 MST