In Machine Learning, model evaluation is the process of assessing the performance of a trained model on a given dataset. The goal of model evaluation is to determine the effectiveness of the trained model in accurately predicting the target variable. This process is critical for selecting the best model out of several potential models.
The following are some common evaluation metrics used for classification and regression problems:
#### Classification Metrics:
##### 1. Accuracy:
Accuracy is defined as the total number of correct predictions made by the model divided by the total number of predictions made by the model. It is the fundamental metric used for classification problems, and it is a good metric when the classes are roughly balanced.
$$Accuracy = \frac{TP + TN}{TP + FP + TN + FN}$$
where TP (True Positive) is the number of correctly predicted positives, TN (True Negative) is the number of correctly predicted negatives, FP (False Positive) is the number of incorrectly predicted positives, and FN (False Negative) is the number of incorrectly predicted negatives.
##### 2. Precision:
Precision is the number of true positive predictions divided by the total number of positive predictions made by the model. It is a useful metric when the cost associated with false positives is high.
$$Precision = \frac{TP}{TP + FP}$$
##### 3. Recall:
Recall is the number of true positive predictions divided by the total number of actual positives present in the data. It is a useful metric when the cost associated with false negatives is high.
$$Recall = \frac{TP}{TP + FN}$$
##### 4. F1 Score:
F1 Score is defined as the harmonic mean of precision and recall. It is a useful metric when the goal is to make a trade-off between precision and recall.
$$F1 Score = 2 * \frac{Precision * Recall}{Precision + Recall}$$
#### Regression Metrics:
##### 1. Mean Squared Error (MSE):
MSE is a common metric used to evaluate regression models. It is the average of the squared differences between the predicted and actual target values.
$$MSE = \frac{1}{N}\sum_{i=1}^{N}(y_i - \hat{y_i})^2$$
where N is the number of data points in the dataset, yi is the actual target value, and $\hat{y_i}$ is the predicted target value.
##### 2. Mean Absolute Error (MAE):
MAE is similar to MSE, but instead of squaring the differences between the predicted and actual target values, we take the absolute difference.
$$MAE = \frac{1}{N}\sum_{i=1}^{N}|y_i - \hat{y_i}|$$
##### 3. R-squared (R2Μ) score:
R-squared score is a statistical measure that represents the proportion of variance in the target variable that is explained by the model. R-squared ranges from 0 to 1, where a higher value indicates a better model fit.
$$R^2 = 1 - \frac{\sum_{i=1}^{N}(y_i - \hat{y_i})^2}{\sum_{i=1}^{N}(y_i - \bar{y})^2}$$
where yΜ is the mean of the target variable.