Statistical independence is a fundamental concept in probability theory and statistics that describes the relationship between two or more random variables, events, or outcomes. Two variables or events are considered statistically independent if the occurrence or realization of one does not provide any information about the occurrence or realization of the other. In other words, the probability of one event happening is not affected by whether the other event has happened or not.
Formally, two events A and B are considered statistically independent if the joint probability of both events occurring is equal to the product of their individual probabilities. This can be expressed as:
P(A ∩ B) = P(A)P(B)
where P(A ∩ B) denotes the joint probability of events A and B, while P(A) and P(B) represent the individual probabilities of events A and B, respectively.
For two random variables X and Y, statistical independence could be defined as:
P(X = x, Y = y) = P(X = x)P(Y = y) ∀x, y
That is, the joint distribution of X and Y equals the product of their marginal distributions.
Let’s consider an example of tossing two fair coins, where the random variable X represents the outcome of the first coin and Y represents the outcome of the second coin. Let H denote heads, and T denote tails. The possible outcomes are:
S = {(H, H), (H, T), (T, H), (T, T)}
The individual probabilities for each outcome are:
$$P(X = H) = P(X = T) = \frac{1}{2} \\[2ex]
P(Y = H) = P(Y = T) = \frac{1}{2}$$
The joint probabilities of the outcomes are:
$$\begin{aligned}
P(X = H, Y = H) &= P(X = H, Y = T) = P(X = T, Y = H) \\
&= P (X = T, Y = T) = \frac{1}{4}\end{aligned}$$
Let’s check if the condition for statistical independence holds:
$$P(X = H, Y = H) = \frac{1}{4} = P(X = H)P(Y = H) = \frac{1}{2} \cdot \frac{1}{2}$$
$$P(X = H, Y = T) = \frac{1}{4} = P(X = H)P(Y = T) = \frac{1}{2} \cdot \frac{1}{2}$$
We can see that the condition for statistical independence holds for all possible outcomes in this example.
In contrast, let’s consider an example with two dependent events. Suppose we draw a single card from a standard deck, where the random variable X represents the event of drawing a red card (hearts or diamonds) and Y represents the event of drawing a heart. These two events are not independent, as drawing a heart implies drawing a red card. In this case, the individual probabilities are:
$$P(X = \text{Red}) = P(X = \text{Black}) = \frac{1}{2} \\[2ex]$$
$$P(Y = \text{Heart}) = P(Y = \text{Other}) = \frac{1}{4}$$
However, the joint probabilities do not satisfy the condition for statistical independence:
$$P(X = \text{Red}, Y = \text{Heart}) = \frac{1}{4}$$
≠
$$P(X = \text{Red})P(Y = \text{Heart}) = \frac{1}{2} \cdot \frac{1}{4} = \frac{1}{8}$$
In conclusion, statistical independence is an important concept that describes the relationship between two or more random variables or events. Understanding statistical independence is crucial in many areas of probability and statistics, as well as various fields of science, including finance, where it’s used to model and analyze the impacts of various factors on financial markets and investment strategies.