In the context of regression analysis, R-squared (R2) is a statistical measure that represents the proportion of the variance in the dependent variable that is explained by the linear regression model. It ranges from 0 to 1, with a value closer to 1 indicating that the regression model fits the data well, and a value closer to 0 indicating a poor fit. The formula for R-squared is:
$$R^2 = 1 - \frac{SS_{res}}{SS_{tot}}$$
where SSres is the sum of squares of residuals (i.e. the difference between the actual values of the dependent variable and the predicted values from the regression equation), and SStot is the total sum of squares (i.e. the difference between the actual values of the dependent variable and the mean value of the dependent variable).
However, R-squared can be biased, especially when there are multiple independent variables in the model. This is where adjusted R-squared (Radj2) comes in as a more useful measure. Adjusted R-squared is a modified version of R-squared, which penalizes the addition of independent variables that do not improve the fit of the model. The formula for adjusted R-squared is:
$$R_{adj}^2 = 1 - \frac{(1 - R^2)(n - 1)}{n - k - 1}$$
where n is the sample size and k is the number of independent variables in the model. Adjusted R-squared ranges from 0 to 1, with a value closer to 1 indicating that the regression model explains a larger proportion of variance in the dependent variable, while taking into account the number of independent variables in the model.
To illustrate the differences between R-squared and adjusted R-squared, consider the following example. Suppose we want to build a regression model to predict the weight of a person based on their age, height, and gender. We fit two models, one with all three independent variables, and another with only age and height as predictors. The resulting R-squared and adjusted R-squared values are as follows:
- Model 1 (with all three predictors): R2 = 0.65, Radj2 = 0.60 - Model 2 (with only age and height as predictors): R2 = 0.50, Radj2 = 0.48
In this case, we can see that the R-squared values indicate that Model 1 explains a larger proportion of variance in the dependent variable than Model 2. However, when we look at the adjusted R-squared values, we can see that Model 2 has a slightly higher value, indicating that it is a better fit than Model 1, when taking into account the number of predictors in the model. This example highlights the importance of using adjusted R-squared instead of R-squared, especially when comparing models with different numbers of predictors.