**Next message:**Anders Sandberg: "Re: tech miracles of the year 2000 as seen from 1950"**Previous message:**Francois-Rene Rideau: "Re: politeness in posting"**In reply to:**Christian Szegedy: "Lagrangian multipliers and neural learning?"**Messages sorted by:**[ date ] [ thread ] [ subject ] [ author ]

måndagen den 14 maj 2001 19:55 Christian Szegedy wrote:

*> An interesting thing about them (to me) is that they seem
*

*> to be closely related to neural learning algorithms, since
*

*> this lagrangian multipliers can be interpreted as a
*

*> weighting of some components of the dual of the objective
*

*> function. So during the algorithms, this dual variables
*

*> punish the "bad" variables by increase their weight, so
*

*> the algorithm can learn that they are "dangerous" and
*

*> should not increase (or decrease) them.
*

I am thinking of something similar. I have spent far too mych time trying to

give a strict derivation of the Bayesian confidence propagation neural

network (my professor invented it, but has never got around to make a

satisfying derivation - which is bad for getting *my* papers on it published

;-), and my current best bet is that it can be derived as some kind of

lagrangian multiplier formalism where some information measure (maybe B-L

dimension) is optimised under the constraint of some other information

measure (maybe entropy).

*> My question: is this connection recognized and exploited
*

*> by the researchs of AI?
*

*> Do they use the results and strength of the theory and
*

*> methods of Lagrangian relaxation explicitly or implicitely?
*

I have not seen much of it, but I recall some derivations in Haykin's Neural

Networks that used the multipliers. That doesn't prove much, since he is

rather fond of using excessive math to derive simple networks (the chapter of

radial basis functions is amazing - he uses Green's functions and functional

analysis! Talk about sharpening pencils with millstones).

**Next message:**Anders Sandberg: "Re: tech miracles of the year 2000 as seen from 1950"**Previous message:**Francois-Rene Rideau: "Re: politeness in posting"**In reply to:**Christian Szegedy: "Lagrangian multipliers and neural learning?"**Messages sorted by:**[ date ] [ thread ] [ subject ] [ author ]

*
This archive was generated by hypermail 2b30
: Mon May 28 2001 - 10:00:05 MDT
*