# Re: Definitions of Probability (was: Re: Clint & Robert on "Faith

David Blenkinsop (blenl@sk.sympatico.ca)
Thu, 28 Oct 1999 00:03:50 -0600

Earlier, Amara Graps wrote:
>
> James Wetterau (jwjr@ignition.name.net) Tue, 26 Oct 1999 writes:
>
> >So far, the only non-circular definition I have found is a
> >mathematical definition claiming certainty in results iff we
> >take limits as the number of experiments goes to infinity.
>
> That's the "frequentist" definition of probability. I don't find the
> definition satisfying either. You may find the Bayesian perspective
> of probability more natural and intutive.
>
>
> The Bayesian probabilistic ideas have been around since the 1700s.
> Bernoulli, in 1713, recognized the distinction between two
> definitions of probability: (1) probability as a measure of the
> plausibility of an event with incomplete knowlege, and (2)
> probability as the long-run frequency of occurrence of an event in a
> sequence of repeated (sometimes hypothetical) experiments. The
> former (1) is a general definition of probability adopted by the
> Bayesians. The latter (2) is called the "frequentist" view,
> sometimes called the "classical", "orthodox" or "sampling theory"
> view.

Yes, I've read most of the chapters of Prof Jayne's Bayesian theory text which you referenced, that *were* available on the Web. Now, the calculus of variations which Jaynes employs in his most advanced "maxent" examples seems just a bit "steep" to me these days, but hey, at least his background discussions are readable. Unfortunately, Lee Daniel Crocker is correct in saying that the Washington University, St. Louis, Missouri web site for this is no longer accessible to the public -- they just can't stand to "share the wisdom", I suppose :^(

On a more positive note, my original inspiration for reading about Bayesian statistics came from my May 1996 issue of Discover magazine, and yes, you *can* get the full text of the article at the http://www.discover.com/ web site, by clicking on "archives" and searching for that issue. The article is called "The Mathematics of Making Up Your Mind", by Will Hively.

Amara Graps wrote:

>
> The Bayesian approach to scientific inference takes into account not
> only the raw data, but also the prior knowledge that one has to
> supplement the data. Using a logical framework for
> prior and current information, the Bayesians infer a probabilistic
> answer to a well-posed question . . .
>
> There are a handful of scientists using Bayesian Probability Theory
> to clarify some fundamental scientific theories. . . .
> The confusion may arise when one interprets probability
> as a real, physical property, that is, when one has failed to see a
> distinction between reality and our _knowledge of_ reality. In the
> Bayesian view, the probability distributions used for inference do
> not describe a real property of the world, only a certain state of
> information about the world. . . . A Bayesian probabilistic approach to
> these fields offers some new chances to clarify many physical
>

Later, Lee Daniel Crocker wrote:

> Yes, Bayes' Theorem and Bayesian interpretations of phenomena are
> useful and interesting, but calling it a "new kind" of pobability
> or implying that it is at odds with the frequentist view that it is
> derived from and depends on is just empty rhetoric.

I believe it is true that the two approaches, Bayesian/Laplacian vs the "standard" or "frequentist" approach, connect quite strongly, such that the most basic, standard calculations, like analysis of gambling games, also practical applications of standard distributions, confidence intervals in such distributions, etc, would *usually* tend to come out the same. The reason for this is that everyone, even a Bayesian theorist, would accept that probability should tend to predict "frequencies of occurrence" in situations of many trials where the calculated sampling distributions are not known to be misleading in some way. The *difference* is that the Bayesians are, ideally, always on the lookout for the new information that would invalidate or force a revision of such probability estimates. In contrast, the frequentist approach is to set up an arbitrary significance test for jumping from one distribution function to another, so you are stuck with function "A" as a "truly real" probability, until some accredited frequentist expert says "oops, that distribution failed The Test, now jump to function B, *that's* the truly real distribution"!

It seems pretty clear to me that the Bayesian criticisms are a lot closer to the truth of what should be concluded from the drug trials in the situation described, than are the "standard" or frequentist, conclusions. Although both sides talk about seemingly somewhat arbitrary standards of what results are significant, it sure looks as though the Bayesian approach is more consistent, despite that fact that the article ends by hinting at the contentiousness of perhaps trying to average the results of several different trials? Not to get too deep into math, but I'm almost sure that normal distribution methods, or maybe Jayne's "maxent" methods, would tend to make a really proper, Bayesian, analysis more manageable than the article seems to suggest at the very end.

Anyway, it seems there *is* a difference between the two approaches, if the frequentists are jumping to a whole new conclusion of "what the future probability or success rate will be" based on a drug's just barely passing a pre-calculated significance level! It just looks like the Bayesian alternative is the logical, reasoned out approach that says "based on that result alone, the chances are just 50% for the drug to have that level of comparative success, or better, the next time around".

David Blenkinsop <blenl@sk.sympatico.ca>