The L1 and L2 norms are two different methods to measure the size or "magnitude" of a vector in a vector space. They are also called "distance metrics" because they help to quantify the distance between two points in a multi-dimensional space. Let me explain both norms and their differences in detail.
Let x = (x1, x2, …, xn) be a n-dimensional vector.
1. L1 Norm (also known as Manhattan Distance or Taxicab Norm):
The L1 norm is the sum of the absolute values of the components of the vector. It is given by the formula:
$L1(\mathbf{x}) = \sum_{i=1}^{n} |x_i|$
In geometric terms, the L1 norm corresponds to the distance between two points in a grid-like path, which resembles the path a taxi takes in a city with a rectangular grid of streets (hence the name "Taxicab Norm").
2. L2 Norm (also known as Euclidean Distance or Frobenius Norm):
The L2 norm is the square root of the sum of the squares of the components of the vector. It is given by the formula:
$L2(\mathbf{x}) = \sqrt{\sum_{i=1}^{n} x_i^2}$
The L2 norm is the straight line distance between two points in Euclidean space, which is the most common and intuitive way to measure the distance between points.
Now let’s compare the differences between the L1 and L2 norms:
1. The L1 norm measures the distance in a grid-like path, while the L2 norm measures the straight line distance. Therefore, L1 norm tends to be larger than the L2 norm.
2. L1 norm is more robust to outliers compared to L2 norm. This is because the L2 norm squares the component values, which has a significant impact on the magnitude when a component is large (an outlier).
3. L1 norm promotes sparsity in the solution space compared to L2 norm. In optimization problems, L1 norm encourages some components of the solution vector to be exactly zero, which leads to a sparse solution, while L2 norm tries to distribute the weights evenly across all components.
Example:
Let’s consider a 2-dimensional vector x = (3, 4).
The L1 norm of this vector is:
L1(x) = |3| + |4| = 3 + 4 = 7
The L2 norm of this vector is:
$L2(\mathbf{x}) = \sqrt{3^2 + 4^2} = \sqrt{9 + 16} = \sqrt{25} = 5$
In this case, the L1 norm is 7, while the L2 norm is 5.
In summary, the L1 and L2 norms are different ways to measure the magnitude of vectors in a vector space. They have their own properties and are suitable for different applications depending on the problem and the design requirements.