When I was in high school, I asked several a person, “Why the normal distribution?”. After all, the function looks like a pretty bizarre function to guess, when there are other functions like which produce perfectly fine bell-shaped curves. One answer I received was that the normal distribution is in some sense the limit of the binomial distribution. While this answer seems fair enough, I tried my hand at the mathematics and did not succeed, so I was still confounded. None the less I believed the answer, and it satisfied me for the time.
The real answer that I was looking for but did not appreciate until university was the central limit theorem. For me, the central limit theorem is the explanation of the normal distribution. In any case, the calculation that I attempted was basically a verification of the central limit theorem in a simple case, and it is a testement to the force of the central limit theorem that that simple case is difficult to work out by hand.
In this post, I rectify that calculation that I should have accomplished in high school (with the benefit of hind-sight being the correct factor of rather than ). On the way, I will also check both laws of large numbers in this simple case.
Consider a “random walk” on . Specifically, let be a random variable taking the values and each with probability , and let
where each is an independent copy of . Note that is Bernoulli, so is binomial, so
where we make the convention that the binomial coefficient is zero when it doesn’t make sense. Hence, if ,
By Stirling’s formula, , so
as . Hence, for constant ,
Let . Note that for , that , and that
so for all . Hence
and by symmetry,
as . Thus we have verified the weak law.
In fact, because the convergence above is geometric,
(This is the Borel–Cantelli lemma.) Thus almost surely. Thus we have verified the strong law.
It remains to check the central limit theorem, i.e., to investigate the limiting distribution of . Now, for constant ,
where is the atomic measure assigning a mass to each point of . Now Stirling’s formula strikes again, giving
where as . Moreover this convergence is uniform in , so
where the last equality follows from the theorem that continuous functions are Riemann integrable. Thus we have verified the central limit theorem.