An Autoregressive (AR) model of order 1, denoted as AR(1), is defined as:
Yt = α + βYt − 1 + ϵt
where Yt is the time series data at time t, α and β are the parameters to be estimated, and ϵt is the error term at time t assumed to be white noise (i.e., Gaussian noise) with mean 0 and constant variance, σ2.
To estimate the parameters α and β of an AR(1) model, we can use the method of least squares or the maximum likelihood estimation. We will explain both methods below.
1. Method of Least Squares (MLS):
We try to minimize the sum of squared residuals by taking derivatives of the objective function with respect to the parameters and setting them equal to zero. By doing so, we get the following normal equations:
$$\begin{cases}
\sum_{t=2}^T (Y_t - \alpha - \beta Y_{t-1}) = 0 \\
\sum_{t=2}^T (Y_t - \alpha - \beta Y_{t-1})Y_{t-1} = 0
\end{cases}$$
Solving these equations simultaneously yields the estimators for α and β as:
$$\hat{\beta} = \frac{\sum_{t=2}^T (Y_t - \bar{Y})(Y_{t-1} - \bar{Y}_{T-1})}{\sum_{t=2}^T (Y_{t-1} - \bar{Y}_{T-1})^2}$$
α̂ = Ȳ − β̂ȲT − 1
where Ȳ is the mean of Yt and ȲT − 1 is the mean of Yt − 1.
2. Maximum Likelihood Estimation (MLE):
Under the assumption of Gaussian white noise for the error term, the log-likelihood function for an AR(1) model is given by:
$$\mathcal{L}(\alpha, \beta, \sigma^2) = -\frac{T-1}{2}\log(2\pi\sigma^2) - \frac{1}{2\sigma^2}\sum_{t=2}^T \epsilon_t^2$$
By substituting the AR(1) model equation in the log-likelihood function, we get:
$$\mathcal{L}(\alpha, \beta, \sigma^2) = -\frac{T-1}{2}\log(2\pi\sigma^2) - \frac{1}{2\sigma^2}\sum_{t=2}^T (Y_t - \alpha - \beta Y_{t-1})^2$$
Now, we need to find the parameters α, β, and σ2 that maximize the above log-likelihood function. We would take the partial derivatives of the log-likelihood function with respect to the parameters, set them equal to zero, and solve for the parameters.
Doing so, we will get the same estimators as in the method of least squares:
$$\hat{\beta} = \frac{\sum_{t=2}^T (Y_t - \bar{Y})(Y_{t-1} - \bar{Y}_{T-1})}{\sum_{t=2}^T (Y_{t-1} - \bar{Y}_{T-1})^2}$$
α̂ = Ȳ − β̂ȲT − 1
The estimator for the variance σ2 is given by:
$$\hat{\sigma}^2 = \frac{1}{T-1} \sum_{t=2}^T \epsilon_t^2$$
In practice, you can use statistical software, such as R or Python, to estimate the parameters of an AR(1) model using the above methods, or by using more advanced optimization techniques.