Over the past few weeks, my colleagues and I have been involved in an attempt to provide a basic constructive proof showing how Fisher Information can be used to derive a logically consistent operational theory of the brain. Basically, the approach is to assume that we are limited to understanding the universe via our senses. By this we mean to say that any information(here meant in the conventional entropic sense) that we obtain is limited by the various filters that is part of our physical makeup. This information(which we treat as a signal in a lossy channel) is then subjected to various forms of pattern recognition...which, in this context, is the function of the brain. Thus, any theory of the brain purporting to describe the brain's capabilities MUST account for the fundamental fact that the perception of any entropic information is done in the limit as bounded by it's Fisher information.
As you can see, we use both entropic information aswell as fisher information....both definitions are useful in that they are measuring two facets of the same thing. Entropic information provides us with s set of 'mechanics' with which we can refer to information in a dynamic sense. Fisher Information provides us with a general physical description of information.
On Thu, 12 Aug 1999, Sasha Chislenko wrote:
>
> From New Scientist, 30 January 1999
>
>
> ----------
> I is the law
>
>
> Illustration: Charlie Ward
>
> 같같같같같같같같같같같같같같같같
>
> It's the ultimate big idea, the source of everything we know about the physical world. And it all comes from one simple question, says Robert Matthews
>
> Where do the laws of physics come from? It's the sort of question only children and geniuses ask--certainly most physicists are far too busy putting the laws to work.
>
> Take quantum theory, the laws of the subatomic world. Over the past century it has passed every single test with flying colours, with some predictions vindicated to 10 places of decimals. Not surprisingly, physicists claim quantum theory as one of their greatest triumphs. But behind their boasts lies a guilty secret: they haven't the slightest idea why the laws work, or where they come from. All their vaunted equations are just mathematical lash-ups, made out of bits and pieces from other parts of physics whose main justification is that they seem to work.
>
> Now one physicist thinks he knows where the laws of quantum theory come from. More amazingly still, Roy Frieden thinks he can account for all the laws of physics, governing everything from schoolroom solenoids to space and time. Sounds incredible? You haven't heard the first of it. For Frieden believes he has found the Law of Laws, the principle underpinning physics itself.
>
> The laws of electricity, magnetism, gases, fluids, even Newton's laws of motion--all of these, Frieden believes, arise directly from the same basic source: the information gap between what nature knows and what nature is prepared to let us find out. Using sophisticated mathematics, Frieden has shown that this notion of physics as a "quest for information" is no empty philosophical pose. It can be made solid, and leads to a way of deriving all the major laws of fundamental physics--along with some new ones.
>
> The sheer power of Frieden's approach is beginning to catch the eye of other researchers. "The results already obtained are extremely spectacular and I'm an enthusiastic supporter," says theorist Peter Hawkes from the CNRS laboratories in Toulouse, France.
>
> Unlike most of the mathematical Schwarzeneggers now trying to unify the whole of physics, Frieden does not normally spend his waking hours wrestling with 26-dimensional space-time. As a researcher at the Optical Sciences Center of the University of Arizona, he has an international reputation in the more practical field of optical image enhancement.
>
> In the early 1970s, he pioneered techniques to "clean up" fuzzy images of everything from distant galaxies to stolen car number plates. He was put on the trail of a radical new view of physics while investigating alternative ways of capturing the information content of images. "For years, I had kept in the back of my mind a passage I had read in a textbook on information theory, which talked about something called 'Fisher Information'. Someday, I thought, I was going to investigate that--and now the time was ripe."
>
> Named after the Cambridge statistician Ronald Aylmer Fisher in the mid-1920s, Fisher information--usually contracted to I--captures how much information you can squeeze out of a physical system. Suppose you want to know where a gas molecule is. You can try measuring it, but no measurements are perfect--they all come with a certain amount of error. What's more there are inherent "errors" in the system--random disorder, jitters associated with the temperature of the gas and the "jolts" caused by the very act of observing, made famous by quantum theory. All of these errors are governed by statistical distributions, such as the famous bell-shaped curve. Plugging these distributions into a formula worked out by Fisher, you end up with a measure of how much information you can extract from a physical system, given all the errors.
>
> At first, Frieden simply used Fisher information calculations as a way of prising more information out of blurry images. But it was while he was reading around the subject that he found himself being pointed in another, more profound, direction. "I came across a 1959 paper by the Dutch mathematician A. J. Stam, who showed that I could be used to derive Heisenberg's famous uncertainty principle," recalls Frieden. "And being a physicist, this set me thinking."
>
> Studying Stam's work, Frieden noticed that it made use of a result from information theory called the Cramer-Rao inequality. This little-known mathematical result shows, roughly speaking, that when the error in a measurement is multiplied by the amount of Fisher information in the measurement, the result is a number that is never less than one.
>
> It's a relationship strikingly similar to the uncertainty principle. Multiply together the uncertainties in your knowledge of a particle's position and its momentum, and the result is never less than a certain value. The more precisely you know the position, the less precisely you can know the momentum. Or put another way, the act of measuring the position influences the measured value of the momentum--and vice versa.
>
> The similarity between the Cramer-Rao inequality and the uncertainty principle started Frieden wondering whether information--and Fisher information in particular--had a much deeper role in physics. "Since Heisenberg's principle is so basic, it occurred to me that perhaps every physical phenomenon occurs in reaction to measurement--that measurement acts as a kind of catalyst for the effect," says Frieden. "And the possibility that physical laws occur as answers to questions excited my curiosity."
>
> Digging into this possibility, Frieden soon found another mathematical "coincidence". Whenever he did calculations using the Fisher information, the final results were differential equations. "What struck me," he recalls, "is that virtually all of physics can also be expressed in terms of differential equations."
>
> Differential equations are formulae showing how the rate of change of a certain quantity changes under outside influences. For instance, Newton's second law of motion relates the acceleration of an object to the force applied: F = ma. The acceleration in this formula is the rate of change of velocity, which in turn is the rate of change of distance. Quantum theory has its own, more abstract, examples, such as Schr�dinger's famous wave equation and Dirac's relativistic equation for the electron. The same format shows up across the whole of physics.
>
> Again, it's the kind of observation that is apt to provoke a shrug of the shoulders. But now Frieden was sure he was on to something really deep. The ubiquity of these types of equation, he believed, is intimately linked to one of the most profound mysteries in science: despite the vast range of phenomena covered by the fundamental laws of physics, all of those laws can be made to drop out of mathematical objects known as Lagrangians. And no one knows why.
>
> Put simply, Lagrangians are made up of the difference between two quantities which together form something called the "action". For reasons as yet utterly mysterious, this quantity stays as small as possible under all circumstances. This curiosity--known as the principle of least action--is reflected in the fact that the fundamental laws of physics are differential equations, since that's what you need to minimize the action.
>
> In Newton's laws of motion, for example, the relevant action turns out to be the difference between the kinetic energy and the potential energy of a body. Kinetic energy is the energy associated with how fast something is moving, and potential energy with its location. It turns out that to keep the difference between these two to a minimum, the object's mass times its acceleration always has to equal the force applied. Minimising this particular action leads to Newton's second law of motion.
>
> Beyond action
>
> Theorists are convinced that action must be incredibly important--so much so that the discovery of any new fundamental law prompts a race to work out the particular action needed to produce it. The trouble is that no one understands the principles behind nature's infatuation with action, and so no one can calculate it directly. Instead, they have to reverse-engineer it, working backwards from the newly discovered law.
>
> It is the puzzle of action--and thus the origin of the laws of physics--that Frieden now reckons he has solved. And, he says, it all comes down to information--the information we try to prise from nature by making observations and the information nature has, but is reluctant to part with.
>
> If you look at Lagrangians for gravity or electromagnetism, says Frieden, they all have more or less the same mathematical form. They are all made up of the difference between I, the Fisher information from observing the phenomenon, and another statistical quantity, J, which is the amount of information bound up in the phenomenon you're trying to measure.
>
> It is from this that Frieden has built his radically new vision of physics based not on the mysterious "action", but on something more intuitive: our attempt to come up with the best possible description of phenomena. All the information needed for such a description exists, in the form of J, and we want as much of it as possible to be extracted by our measurements, in the form of I. In other words, we want the information difference--I minus J--to be as small as possible. And it turns out that for this difference to be as small as possible, the phenomenon must obey a differential equation.
>
> Frieden's information-based methods provide a stunningly clear interpretation of the laws of physics: they represent the best we can possibly do in our quest to extract information using our inevitably error-prone methods. "Through the very act of observing, we thus actually define the physics of the thing measured," says Frieden. He adds that while unfamiliar, the idea that "reality"--or, at least, the laws of physics--are created by observation is not new. During the 18th century, empiricist philosophers such as Bishop Berkeley were raising similar ideas. Much more recently, John Wheeler, a physicist at Princeton University who is widely regarded as one of the deepest thinkers on the foundations of physics, has championed remarkably similar views. "Observer participancy gives rise to information and information gives rise to physics," he says.
>
> That's not to say Frieden's approach implies that the laws of physics are "all in the mind". Rather, it means that any physical attempt to extract information about nature determines the answer we obtain--and the best information we can ever extract is what we call the laws of physics.
>
> So Frieden's achievement is to give a philosophical view of physics a solid mathematical foundation. For any given system, I and J are statistical quantities which can be calculated using Frieden's methods. And the payoff is spectacular: with these two quantities, you can fulfill the 200-year-old dream of deriving the Lagrangian for that system, and thus of deriving the physical law that rules it.
>
> Over the past 10 years, in a series of papers in such journals as Physical Review, Frieden and colleagues including Bernard Soffer of the Hughes Research Laboratories in Malibu, California, have been steadily working their way through physics, showing that all of its laws are the result of a kind of cosmic game between ourselves and the "real" world. To derive each law--or, more accurately, each Lagrangian--we have to ask an incredibly simple yet fundamental question, such as "what is the precise location of a particle in space and time?"
>
> Any attempt to answer such questions requires the same two quantities: the information that exists in any given thing or system, J, and the information we can acquire, I. Frieden has developed methods of calculating both for a wide range of phenomena in physics. Subtracting J from I then leads straight to the appropriate Lagrangian, and when this is made as small as possible, the appropriate law of physics "emerges". No reverse engineering, no fancy use of mathematical tricks, no inspired guesses.
>
> Take that question about the precise location of a particle in space and time. Frieden's approach leads directly to the Lagrangian for the Klein-Gordon equation. This is the central law of relativistic quantum theory which describes the way particles move through space and time. If, on the other hand, you want to know about the location of a particle in space alone, Frieden's approach leads to Schr�dinger's wave equation.
>
> That this one principle can act as a key to unlock the fundamental laws is impressive enough, but if it really is the key to all physics, it should do more than reproduce what physicists already know. It should also reveal the secrets of unsolved mysteries.
>
> Turbulence tamed
>
> Some researchers are finding that it can. Take turbulence, the roiling motion of fast-moving fluids whose understanding Einstein himself regarded as the biggest challenge to classical physics. In 1996, John Cocke at the University of Arizona showed that using Frieden's approach on the question of what is the flow of mass at a particular time and place leads to a law governing the size of density fluctuations in turbulent fluids. This law makes sense of otherwise baffling results from studies of fluid behaviour.
>
> The quantum world offers an equally demanding challenge that has effectively defeated the world's best theorists for decades. Quantum theory--which sees everything in terms of discrete jerks, jumps and packets--just does not sit easily with Einstein's concept of smooth expanses of curved space-time.
>
> Yet Frieden found last year that by asking what space-time like is, he arrives at a Lagrangian which leads straight to the Wheeler-deWitt equation: a formula giving a quantum description of space-time. The Wheeler-deWitt equation, now more than 30 years old, is one of the few concrete results in quantum gravity theory.
>
> Until now, however, the principles behind Wheeler-deWitt have been far from clear. Frieden's theory not only shows how to derive the Wheeler-deWitt equation, but also seems to shed light on what the equation means. Frieden is already examining these clues to see how they may help theorists go beyond the equation to a full-blown theory of quantum gravity.
>
> Frieden is still struggling to spread his message among other theorists, many of whom are reluctant even to study his approach. "Part of the reason is probably simple inertia to learning about a new concept like Fisher information," he says.
>
> But others are more enthusiastic. "Frieden's shown that a host of what used to be regarded as fundamental equations of physics are actually capable of derivation," says Hawkes. Cocke agrees: "It is a sort of unifying principle, and I see it as a method of solving tough problems in statistical physics."
>
> Frieden hopes that his new book, which shows in detail how to apply Fisher information to physical problems, will help to convince others how powerful his approach is, and encourage them to join in. "What I and my co-workers have done so far is by no means the final word, but it does offer a systematic way to finding laws for new phenomena. And it seems that information is what physics is all about."
>
> Robert Matthews is science correspondent of The Sunday Telegraph
>
> Further reading:
> * Physics from Fisher Information by Roy Frieden, Cambridge University Press
>
> ----------
> From New Scientist, 30 January 1999
>
> � Copyright New Scientist, RBI Limited 1999
>
>
> ---------------------------------------------------------------
> Sasha Chislenko <http://www.lucifer.com/~sasha/home.html>
>
>