In a Markov chain, a system transitions between a finite or countable number of states according to certain probabilities. The steady-state probabilities refer to the likelihood of the system being in a particular state after many iterations (or time steps). Mathematically, these steady-state probabilities form a probability distribution over the states that remains unchanged as time progresses.
To define steady-state probabilities, we need to introduce the concept of a transition probability matrix. Let P denote the matrix of transition probabilities pij, where pij represents the probability of transitioning from state i to state j. That is:
$$P = \begin{pmatrix}
p_{11} & p_{12} & \cdots & p_{1n} \\
p_{21} & p_{22} & \cdots & p_{2n} \\
\vdots & \vdots & \ddots & \vdots \\
p_{n1} & p_{n2} & \cdots & p_{nn}
\end{pmatrix}.$$
Each row of the matrix sums to 1, as probabilities must add up to 1 for each initial state.
Let π be the row vector of steady-state probabilities (π1, π2, …, πn). Since π is a probability distribution, the sum of its elements is also 1:
$$\sum_{i=1}^{n} \pi_i = 1.$$
As the steady-state probabilities remain constant over time, the following equation must hold:
πP = π.
Additionally, having a row vector π means that we need to satisfy the probability distribution requirement:
$$\sum_{i=1}^{n} \pi_i = 1.$$
To find the steady-state probabilities π, we need to solve the above system of linear equations. For example, let’s consider the following Markov chain transition probability matrix:
$$P = \begin{pmatrix}
0.4 & 0.6 \\
0.8 & 0.2
\end{pmatrix}.$$
The steady-state equation to be solved is:
$$\begin{cases}
\pi_1 = 0.4\pi_1 + 0.8\pi_2 \\
\pi_2 = 0.6\pi_1 + 0.2\pi_2
\end{cases}$$
with the constraint:
π1 + π2 = 1.
Solving this system yields:
$$\pi_1 = 0.5714 \approx \frac{4}{7}, \quad \pi_2 = 0.4286 \approx \frac{3}{7}.$$
This means that if the Markov chain were to run for many time steps, the probabilities of being in state 1 or state 2 would converge to π1 and π2, respectively.