Feature Scaling is an important preprocessing step in machine learning to bring all the features or variables to the same scale for better interpretation by the algorithms. The goal of feature scaling is to make sure that no variable should have undue influence over the algorithm learning process.
The need for feature scaling arises because the different features we have may have different magnitudes, scales or numerical ranges. This may give some features undue importance and make them dominant over other features. Hence, when doing predictive analytics, itβs often recommended to rescale the features to get better accuracy and precision for different algorithms.
There are mainly two types of feature scaling techniques that we usually see: 1. Min-Max Scaling (Normalization) 2. Standardization
Min-Max Scaling: It scales the data to lie between a minimum and maximum value we provide. The formula to normalize the data is:
$$X_{sc} = \frac{X-X_{min}}{X_{max}-X_{min}}$$
where X is the original data, Xmin and Xmax are the minimum and maximum values of the feature βXβ, respectively.
Thus, the value of scaled data, Xsc, lies between [0,1]. Min-Max Scaling is a popular normalizing technique where we can apply to all the features simultaneously, as it maintains the distribution of the variable.
Standardization: It scales the data in a way that the distribution has a mean of 0 and standard deviation of 1. The formula to Standardize the data is:
$$X_{std} = \frac{X - \mu}{\sigma}$$
where X is the original data, ΞΌ is the mean of the data and Ο is the standard deviation of the data.
Standardization is generally preferred over normalization when we need to have better accuracy because it is less affected by outliers. Also, when we work on algorithms like PCA (Principal Component Analysis) and clustering techniques, we need to standardize the input variables. PCA works based on finding the variable which has the maximum variance and standardization helps the algorithm to work properly, as all variables will be standardized to the same scale.
There are other types of scaling techniques and algorithms like log-based scaling, sigmoid scaling, deciduous scaling, Z-score normalization, and robust scaling, among others, which are used based on the problem space and the distribution of the variables. Overall, feature scaling is an essential step to avoid biased analysis and to get better prediction and accuracy.