måndagen den 14 maj 2001 19:55 Christian Szegedy wrote:
> An interesting thing about them (to me) is that they seem
> to be closely related to neural learning algorithms, since
> this lagrangian multipliers can be interpreted as a
> weighting of some components of the dual of the objective
> function. So during the algorithms, this dual variables
> punish the "bad" variables by increase their weight, so
> the algorithm can learn that they are "dangerous" and
> should not increase (or decrease) them.
I am thinking of something similar. I have spent far too mych time trying to
give a strict derivation of the Bayesian confidence propagation neural
network (my professor invented it, but has never got around to make a
satisfying derivation - which is bad for getting *my* papers on it published
;-), and my current best bet is that it can be derived as some kind of
lagrangian multiplier formalism where some information measure (maybe B-L
dimension) is optimised under the constraint of some other information
measure (maybe entropy).
> My question: is this connection recognized and exploited
> by the researchs of AI?
> Do they use the results and strength of the theory and
> methods of Lagrangian relaxation explicitly or implicitely?
I have not seen much of it, but I recall some derivations in Haykin's Neural
Networks that used the multipliers. That doesn't prove much, since he is
rather fond of using excessive math to derive simple networks (the chapter of
radial basis functions is amazing - he uses Green's functions and functional
analysis! Talk about sharpening pencils with millstones).
This archive was generated by hypermail 2b30 : Mon May 28 2001 - 10:00:05 MDT