Earlier, Amara Graps wrote:
>
> James Wetterau (jwjr@ignition.name.net) Tue, 26 Oct 1999 writes:
>
> >So far, the only non-circular definition I have found is a
> >mathematical definition claiming certainty in results iff we
> >take limits as the number of experiments goes to infinity.
>
> That's the "frequentist" definition of probability. I don't find the
> definition satisfying either. You may find the Bayesian perspective
> of probability more natural and intutive.
>
>
> The Bayesian probabilistic ideas have been around since the 1700s.
> Bernoulli, in 1713, recognized the distinction between two
> definitions of probability: (1) probability as a measure of the
> plausibility of an event with incomplete knowlege, and (2)
> probability as the long-run frequency of occurrence of an event in a
> sequence of repeated (sometimes hypothetical) experiments. The
> former (1) is a general definition of probability adopted by the
> Bayesians. The latter (2) is called the "frequentist" view,
> sometimes called the "classical", "orthodox" or "sampling theory"
> view.
Yes, I've read most of the chapters of Prof Jayne's Bayesian theory text which you referenced, that *were* available on the Web. Now, the calculus of variations which Jaynes employs in his most advanced "maxent" examples seems just a bit "steep" to me these days, but hey, at least his background discussions are readable. Unfortunately, Lee Daniel Crocker is correct in saying that the Washington University, St. Louis, Missouri web site for this is no longer accessible to the public -- they just can't stand to "share the wisdom", I suppose :^(
On a more positive note, my original inspiration for reading about Bayesian statistics came from my May 1996 issue of Discover magazine, and yes, you *can* get the full text of the article at the http://www.discover.com/ web site, by clicking on "archives" and searching for that issue. The article is called "The Mathematics of Making Up Your Mind", by Will Hively.
Amara Graps wrote:
>
> The Bayesian approach to scientific inference takes into account not
> only the raw data, but also the prior knowledge that one has to
> supplement the data. Using a logical framework for
> prior and current information, the Bayesians infer a probabilistic
> answer to a well-posed question . . .
>
> There are a handful of scientists using Bayesian Probability Theory
> to clarify some fundamental scientific theories. . . .
> The confusion may arise when one interprets probability
> as a real, physical property, that is, when one has failed to see a
> distinction between reality and our _knowledge of_ reality. In the
> Bayesian view, the probability distributions used for inference do
> not describe a real property of the world, only a certain state of
> information about the world. . . . A Bayesian probabilistic approach to
> these fields offers some new chances to clarify many physical
> "paradoxes".
>
Later, Lee Daniel Crocker wrote:
> Yes, Bayes' Theorem and Bayesian interpretations of phenomena are
> useful and interesting, but calling it a "new kind" of pobability
> or implying that it is at odds with the frequentist view that it is
> derived from and depends on is just empty rhetoric.
I believe it is true that the two approaches, Bayesian/Laplacian vs the "standard" or "frequentist" approach, connect quite strongly, such that the most basic, standard calculations, like analysis of gambling games, also practical applications of standard distributions, confidence intervals in such distributions, etc, would *usually* tend to come out the same. The reason for this is that everyone, even a Bayesian theorist, would accept that probability should tend to predict "frequencies of occurrence" in situations of many trials where the calculated sampling distributions are not known to be misleading in some way. The *difference* is that the Bayesians are, ideally, always on the lookout for the new information that would invalidate or force a revision of such probability estimates. In contrast, the frequentist approach is to set up an arbitrary significance test for jumping from one distribution function to another, so you are stuck with function "A" as a "truly real" probability, until some accredited frequentist expert says "oops, that distribution failed The Test, now jump to function B, *that's* the truly real distribution"!
If that seems a bit obscure, read the Discover article that I mentioned, with special attention to the real life drug testing scenario described there. In this scenario, you have "standard" analysts proclaiming the superiority of one drug over another, because the survival rate of patients in a trial came out 1 percentage point better on that drug -- and handily, one percent just happened to be the arbitrary number that the analysts picked beforehand as what it would take to cause someone to jump from the slightly *worse* survival rate to assuming a "real" or predictably *better* survival rate. Now, needless to say, the Bayesian theorists criticized this severely as the mere jumping of an arbitrary hurdle, and point out that, if you take the observed one percent difference as your most basic "measurement", then there is only a 50/50 chance that the "good" drug would actually beat that performance in the future! Now, if the original analysts had decided on a smaller "hurdle", for one drug to beat another, like a tenth of a percent or something, they might still have been correct to conclude that the observed difference was very significant in those terms. From what I can see, this would have been very difficult for them to justify, since it would seem to imply an absurdly high degree of precision in such results, such as might be represented by figuring out the standard deviation in the data, for instance.
It seems pretty clear to me that the Bayesian criticisms are a lot closer to the truth of what should be concluded from the drug trials in the situation described, than are the "standard" or frequentist, conclusions. Although both sides talk about seemingly somewhat arbitrary standards of what results are significant, it sure looks as though the Bayesian approach is more consistent, despite that fact that the article ends by hinting at the contentiousness of perhaps trying to average the results of several different trials? Not to get too deep into math, but I'm almost sure that normal distribution methods, or maybe Jayne's "maxent" methods, would tend to make a really proper, Bayesian, analysis more manageable than the article seems to suggest at the very end.
Anyway, it seems there *is* a difference between the two approaches, if the frequentists are jumping to a whole new conclusion of "what the future probability or success rate will be" based on a drug's just barely passing a pre-calculated significance level! It just looks like the Bayesian alternative is the logical, reasoned out approach that says "based on that result alone, the chances are just 50% for the drug to have that level of comparative success, or better, the next time around".
David Blenkinsop <blenl@sk.sympatico.ca>