Can you explain the Central Limit Theorem?

The Central Limit Theorem (CLT) is a fundamental result in probability theory and statistics, which states that the sum (or average) of a sufficiently large number of independent random variables, each with finite mean and variance, approaches a normal distribution, regardless of the original distribution of the variables.

Here’s the formal statement of CLT:

Let X₁, X₂, ..., X_n be a sequence of independent and identically distributed (i.i.d.) random variables with an expected value μ = E[X_i] and variance σ² = Var(X_i), both of which are finite. Define $S_n = \sum_{i=1}^n X_i$ and $\bar{X_n} = \frac{S_n}{n}$. Then, as n → ∞, the random variable $\frac{(S_n - n\mu)}{\sqrt{n}\sigma}$ converges in distribution to a standard normal random variable, i.e.,

$$\lim_{n \to \infty} P \bigg( \frac{(S_n - n\mu)}{\sqrt{n}\sigma} \leq z \bigg) = \Phi(z),$$
where Φ(z) is the cumulative distribution function (CDF) of the standard normal distribution.

The CLT has significant implications for statistical inferences and financial modeling. Its general idea is that the process of averaging large numbers of independent random variables can generate a normal distribution, even if the individual variables are not normally distributed. This allows us to apply normal-distribution-based tools to analyze problems with several other underlying distributions potentially.

Consider a simple example: the rolling of a fair die. The outcome of each roll (integer from 1 to 6) has a uniform distribution. If you roll the die n times and calculate the average value $\bar{X_n}$, the distribution of these averages converges to a normal distribution as n gets large. This can be visualized using a histogram where the rolls are separated into bins representing the outcome ranges. As n increases, the histogram will resemble the bell curve of the normal distribution.

Here is a demonstration in Python using sampling:

import numpy as np
import matplotlib.pyplot as plt

np.random.seed(42)
n_rolls = [1, 10, 50, 100, 1000]
n_experiments = 10000

for n in n_rolls:
    averages = []
    for _ in range(n_experiments):
        rolls = np.random.randint(1, 7, n)
        averages.append(np.mean(rolls))
    plt.hist(averages, bins='auto', density=True, alpha=0.5, label=f'n={n}')
plt.legend()
plt.show()

Upon running the code, you will see that as the number of rolls (‘n‘) increases, the histograms of the averages start resembling the bell-shaped curve of the normal distribution.

The CLT is a powerful tool in mathematical finance, as it is widely used for option pricing, risk management, and portfolio optimization. Because many financial variables can be modeled as the sum of a large number of smaller, independent variables, the CLT allows us to apply the normal distribution’s properties to various complex scenarios in finance.