Re: Nanotech control systems (was Re: Transhuman Beach Party)

Aaron Davidson (ajd@ualberta.ca)
Sun, 12 Sep 1999 00:04:35 -0600

>On Fri, 10 Sep 1999, J. R. Molloy wrote:
>
>The problem isn't "wiring them together" but having them "do
>something" once wired. Nobody has ever had a 70 million cell
>neural net before. How deep do you make it? How broad do you
>make it? How many inputs & outputs should each neuron have?
>
>I'm not sure how flexible the de Garis architecture really is
>but if the depth can be up to 10 deep, and the net has
>1000 inputs and 100 outputs, and each neuron can interact with 500
>others, the possible number of configurations is huge. [I'm not an
>expert on neural nets, so if someone can explain this better
>please do so.]

It gets even worse than this -- and what you mention above still is indeed an active researche problem. There isn't even a good formula for choosing depth as a function of say, input size. Once you have the physical layout of the NN, there are dozens of different ways to connect the neurons. There are timing issues, and there are a kazillion ways to train the net. The learning algorithm is very important because in solving different problems, some algorithms which work great for one type of domain get easily stuck at local maxima or oscillate wildly, in another domain.

Some attempts at solving these problems involve dynamically reconfiguring nets, that change their structure as they learn. Neurons are killed off if they are deemed to be hindering or useless. New layers can be removed if the net is too large to generalize a problem (and instead turns into a giant look-up table), or added if the net is underfitting the data. Genetic Algorithms have been used to try and find optimal organizations. The list goes on and on. No one has yet found a way to build a general purpose neural net system. We still need to custom design NN's to work well for different domains.

--Aaron

+-------------------------------------------------------------------------+
| Aaron Davidson <ajd@ualberta.ca> http://ugweb.cs.ualberta.ca/~davidson/ |
+-------------------------------------------------------------------------+