- About
- Statistics
- Number Theory
- Java
- Data Structures
- Cornerstones
- Calculus
The Poisson distribution is actually a limiting case of a Binomial distribution when the number of trials, n, gets very large and p, the probability of success, is small. As a rule of thumb, if $n \ge 100$ and $np \le 10$, the Poisson distribution [taking $\lambda = np$] can provide a very good approximation to the binomial distribution.
This is particularly useful as calculating the combinations inherent in the probability formula associated with the binomial distribution can become difficult when $n$ is large.
To better see the connection between these two distributions, consider the binomial probability of seeing $x$ successes in $n$ trials, with the aforementioned probability of success, $p$, as shown below.
$$P[x]={}_nC_x p^x q^{n-x}$$Let us denote the expected value of the binomial distribution, $np$, by $\lambda$. Note, this means that
$$p=\frac{\lambda}{n}$$and since $q=1-p$,
$$q=1-\frac{\lambda}{n}$$Now, if we use this to rewrite $P[x]$ in terms of $\lambda$, $n$, and $x$, we obtain
Using the standard formula for the combinations of $n$ things taken $x$ at a time and some simple properties of exponents, we can further expand things to
$$P[x] = \frac{n[n-1][n-2] \cdots [n-x+1]}{x!} \cdot \frac{\lambda^x}{n^x} \left[ 1 - \frac{\lambda}{n} \right]^{n-x}$$Notice that there are exactly $x$ factors in the numerator of the first fraction. Let us swap denominators between the first and second fractions, splitting the $n^x$ across all of the factors of the first fraction's numerator.
$$P[x] = \frac{n}{n} \cdot \frac{n-1}{n} \cdots \frac{n-x+1}{n} \cdot \frac{\lambda^x}{x!}\left[ 1 - \frac{\lambda}{n} \right]^{n-x}$$Finally, let us split the last factor into two pieces, noting [for those familiar with Calculus] that one has a limit of $e^{-\lambda}$.
$$P[x] = \frac{n}{n} \cdot \frac{n-1}{n} \cdots \frac{n-x+1}{n} \cdot \frac{\lambda^x}{x!}\left[ 1 - \frac{\lambda}{n} \right]^n \left[ 1 - \frac{\lambda}{n} \right]^{-x}$$It should now be relatively easy to see that if we took the limit as $n$ approaches infinity, keeping $x$ and $\lambda$ fixed, the first $x$ fractions in this expression would tend towards 1, as would the last factor in the expression. The second to last factor, as was mentioned before, tends towards $e^{-\lambda}$, and the remaining factor stays unchanged as it does not depend on $n$. As such, $$\lim_{n \rightarrow \infty} P[x] = \frac{e^{-\lambda} \lambda^x}{x!}$$
Which is what we wished to show.
The short answer is that the Poisson approximation is faster and easier to compute and reason about, and among other things tells you approximately how big the exact answer is.
Here's a simple example: suppose you're trying to get something to happen in a video game that is rare; maybe it happens 1% of the time you do something, independently. You'd like to know how likely it is to happen at least once if you try, say, 100 times. Here we have $p = \frac{1}{100}, n = 100$ and so the binomial distribution gives us an exact answer, namely
$$1 - \left[ 1 - \frac{1}{100} \right]^{100}.$$
But how big is this? Do you know off the top of your head? Is it, say, bigger or less than 50%?
The Poisson approximation answers this question quickly and easily: in this special case, it amounts to the approximation
$$\left[ 1 - \frac{1}{100} \right]^{100} \approx e^{-1} \approx 0.368 \dots $$
which gives
$$1 - \left[ 1 - \frac{1}{100} \right]^{100} \approx 1 - e^{-1} \approx 0.632 \dots $$
so we get that the odds are about 63% that we'll succeed at least once, which is bigger than 50% but maybe smaller than you might hope.
We learn something else too: the Poisson approximation tells us more generally that the odds of success are approximately a function of the product $np = \lambda$ only [which is the expected number of successes], so that e.g. if we had $p = \frac{1}{1000}$ and $n = 1000$ the answer would still be about 63%. This is valuable information and not entirely obvious from the exact answer, and knowing it saves you from having to recompute a bunch of binomials.
Sometimes $n$ can get large enough that it would actually be infeasible to calculate the exact binomial answer. For example, suppose $n = 10^{25}, p = 10^{-25}$; numbers this big regularly appear in physics or chemistry since Avogadro's number is so large. I can confidently say that the answer is still about 63% even though it is no longer feasible to exactly calculate $[1 - p]^n$ [just try it!]. The funny thing here is that the larger $n$ gets the harder it becomes to exactly compute the binomials, but the more accurate the Poisson approximation gets; for numbers this large it is for all intents and purposes basically exact.