English theologian and mathematician Thomas Bayes has greatly contributed to the field of probability and statistics. His ideas have created much controversy and debate among statisticians over the years.

Thomas Bayes was born in 1702 in London, England. There appears to be no exact records of his birth date. Bayes's father was one of the first six Nonconformist ministers to be ordained in England. (4) Bayes's parents had their son privately educated. There is no information about the tutors Bayes worked with. However, there has been speculation that he was taught by de Moivre, who was doing private tuition in London during this time.

Bayes went on to be ordained, like his father, a Nonconformist minister. He first assisted his father in Holborn, England. In the late 1720's, Bayes took the position of minister at the Presbyterian Chapel in Tunbridge Wells, which is 35 miles southeast of London. Bayes continued his work as a minister up until 1752. He retired at this time, but continued to live in Tunbridge Wells until his death on April 17, 1761. His tomb is located in Bunhill Fields Cemetery in London.

Throughout his life, Bayes was also very interested in he field of mathematics, more specifically, the area of probability and statistics. Byes is believed to be the first to use probability inductively. He also established a mathematical basis for probability inference. Probability inference is the means of calculating, from the frequency with which an event has occurred in prior trials, the probability that this event will occur in the future. (5) According to this Bayesian view, all quantities are one of two kinds: known and unknown to the person making he inference. (6) Known quantities are obviously defined by their known values. Unknown quantities are described by a joint probability distribution. Bayesian inference is seen not as a branch of statistics, but instead as a new way of looking at the complete view of statistics. (6)

Bayes wrote a number of papers that discussed his work. However, the only ones known to have been published while he was still living are: Divine Providence and Government Is the Happiness of His Creatures (1731) and An Introduction to the Doctrine of Fluxions, and a Defense of the Analyst (1736). The latter paper is an attack on Bishop Berkeley for his attack on the logical foundations of Newton's Calculus. Even though Bayes was not highly recognized for his mathematical work during his life, he was elected a Fellow of the Royal Society in 1742.

Perhaps Bayes's most well known paper is his Essay Towards Solving a Problem in the Doctrine of Chances. This paper was published in the Philosophical Transactions of the Royal Society of London in 1764. This paper described Bayes's statistical technique known as Bayesian estimation. This technique based the probability of an event that has to happen in a given circumstance on a prior estimate of its probability under these circumstances. This paper was sent to the Royal Society by Bayes's friend Richard Price. Price had found it among Bayes's papers after he died. Bayes's findings were accepted by Laplace in a 1781 memoir. They were later rediscovered by Condorcet, and remained unchallenged. Debate did not arise until Boole discovered Bayes's work. In his composition the Laws of Thought, Boole questioned the Bayesian techniques.

Boole's questions began a controversy over Bayes's conclusions that still continues today. In the 19th century, Laplace, Gauss, and others took a great deal of interest in this debate. However, in the early 20th century, this work was ignored or opposed by most statisticians. Outside the area of statistics, Bayes continued to have support from certain prominent figures. Both Harold Jeffreys, a physicist, and Arthur Bowley, an econometrician, continued to argue on behalf of Bayesian ideas. (1) The efforts of these men received help from the field of statistics beginning around 1950. Many statistical researchers, such as L. J. Savage, Buno do Finetti, Dennis Lindley, and Jack Kiefer, began advocating Bayesian methods as a solution for specific deficiencies in the standard system. (1)

However, some researchers still argue that concentrating on inference for model parameters is misguided and uses unobservable, theoretical quantities. (1) Due to this skepticism, some are reluctant to fully support the Bayesian approach and philosophy.

A specific contribution Thomas Bayes made to the fields of probability and statistics is known as Bayes Theorem. It was first published in 1763, two years after his death. It states:

It uses probability theory as logic and serves as a starting point for inference problems. (3) It is still unclear what Bayes intended to do with this calculation.

The left hand side of the equation is known as the posterior probability. It represents the probability of a hypothesis H when given the effect of E in the context of C. The term P(H/C) is called the prior probability of H given the context of C by itself. The term P(E/H, C) is known as the likelihood. The likelihood is the probability of E assuming that H and C are true. Lastly, the term 1 / P(E/C) is independent of H and can be seen as a scaling constant. (3)

Bayes Theorem can be derived from the Product Rule of probability. The Product Rule is P(A, B/I) = P(A/B, I) * P (B/I) = P(B/A,I) * P(A/I). Rearranging this and extending the rule to multiple sequential updates gives: (3)

This becomes very complicated because as each new piece of E is brought into the equation, the effect is conditional on all previous E. However, making the assumption P(E2/E1,C) = P(E2/C) and P(E1/E2,C) = P(E1/C) avoids this difficulty. This assumes given C knowing E2 gives no information about E1 and vice versa. (3) The product rule then reduces to P(E1,E2/C) = P(E1/C) * P(E2/C). (3) This must be used carefully though. Conditional Independence does not always hold. These principles are what much of the controversy is centered around.

As you can see, Thomas Bayes has made many important contributions to the development of probability and statistics. Although his work has been controversial, it has brought forth many new ideas that the world of mathematics continues to research and benefit from.

1. Bradley, P. & Louis, T. (1996). Bayes and Empirical Bayes Methods for

Data Analysis. London: Chapman & Hall.

2. http://ic.arc.nasa.gov/ic/projects/bayes-group/html/bayes-

theorem.html

3. http://ic.arc.nasa.gov/ic/projects/bayes-group/html/bayes-

theorem-long.html

4. http://www-groups.dcs.st-and.ac.uk/~history/Mathematicians/

Bayes.html

5. http://www.stat.ucl.ac.be/ISpersonnel/beck/bayes.html

6. Johnson, N. & Kotz, S. (1982). Encyclopedia of Statistical Sciences,

1, 197-205. New York: John Wiley & Sons, Inc.