What is the difference between a PDF and a CDF?

In probability theory and statistics, the Probability Density Function (PDF) and the Cumulative Distribution Function (CDF) are two fundamental concepts that describe the distribution of a random variable. Both functions provide valuable insights into the behavior of the underlying variable, but they convey different types of information. Let’s discuss them individually and then highlight the differences between them.

1. Probability Density Function (PDF):

The PDF of a continuous random variable gives the relative likelihood of any point within a given range to occur. In other words, it is a function that describes the probability that a random variable takes a specific value or lies within a particular range of values. The PDF is represented by lower case f(x) when the random variable is denoted by X.

Mathematically, the PDF must satisfy two conditions:

(a) f(x) ≥ 0 for all x. (Probability is always non-negative)

(b) ∫_− ∞^∞f(x)dx = 1. (The sum of probabilities across the entire range must equal 1)

For instance, a common and widely known PDF is the normal (Gaussian) distribution, with probability density function given by:

$$f(x) = \frac{1}{\sqrt{2\pi\sigma^2}} e^{-\frac{(x-\mu)^2}{2\sigma^2}}$$

Here, μ is the mean, and σ is the standard deviation of the Gaussian distribution.

2. Cumulative Distribution Function (CDF):

The CDF of a random variable gives the probability that the variable takes a value less than or equal to a specific value. The CDF is represented by F(x) for a continuous random variable X.

Mathematically, the CDF for a continuous random variable X can be defined as:

F(x) = P(X ≤ x) = ∫_− ∞^xf(t)dt

Where f(t) is the PDF of the random variable X.

The CDF has the following properties:

(a) 0 ≤ F(x) ≤ 1 for all x. (It is a probability)

(b) F(x) is a non-decreasing function of x. (It cannot decrease as x increases)

(c) lim_{x →  − ∞}F(x) = 0,  lim_{x → ∞}F(x) = 1

Using the Gaussian distribution example, the CDF for a normal distribution can be expressed as:

$$F(x) = \frac{1}{2}\left[1 + \text{erf}\left(\frac{x-\mu}{\sigma\sqrt{2}}\right)\right]$$

Where erf(x) is the error function.

Now, let’s highlight the differences between PDF and CDF:

1. PDF is the probability density function of a random variable, i.e., it represents the relative likelihood of a specific value or a range of values. The CDF is the cumulative distribution function of a random variable, i.e., it gives the probability that a random variable takes a value less than or equal to a particular value.

2. PDF can be used to find the probability of a random variable lying within a specific range using the integral of the PDF over that range, whereas CDF directly gives the cumulative probability up to a certain value.

3. The PDF is always non-negative, but it can be greater than 1, while the CDF is always between 0 and 1.

4. CDF is the integral of the PDF. To go from the CDF back to the PDF, we can differentiate the CDF with respect to the random variable:

$$f(x) = \frac{dF(x)}{dx}$$

In conclusion, the PDF and the CDF are two different but related ways to describe the distribution of a random variable. The PDF provides information on the specific probabilities or likelihood of values, while the CDF gives cumulative probabilities up to a certain value.