Eigenvectors and principal components are related concepts in linear algebra and statistics, respectively. Let’s discuss both in detail before pointing out the differences between them.
1. Eigenvector:
An eigenvector is a non-zero vector in a linear transformation that only changes by a scalar factor when that transformation is applied to it. Given a square matrix A and a vector v, if the following equation holds:
Av = λv
where λ is a scalar (called the eigenvalue), then v is called the eigenvector of A. Eigenvectors and their corresponding eigenvalues reveal important properties of a linear transformation or a matrix, such as the axis along which the system scales the most.
2. Principal Component:
Principal components are derived from a statistical method called Principal Component Analysis (PCA). PCA is a technique used to reduce the dimensionality of a dataset while retaining as much information (variance) as possible. It involves finding a set of orthogonal vectors (principal components) that capture the most significant patterns in the data.
Given a dataset X with columns representing variables and rows representing observations, the first step in PCA involves normalizing the data by centering it around the mean and, in some cases, scaling by the standard deviation. The covariance matrix C of the centered dataset is then computed:
C = (XTX)/(n − 1)
where n is the number of observations (data points). The principal components are then the eigenvectors of the covariance matrix C, sorted in decreasing order of their corresponding eigenvalues. The first principal component captures the largest amount of variance in the data, the second principal component captures the second largest amount, and so on.
Now, let’s discuss the differences between eigenvectors and principal components:
- Eigenvectors are a linear algebra concept, whereas principal components are derived from a statistical method (PCA).
- Eigenvectors are associated with a linear transformation or a square matrix, while principal components are associated with a dataset and its covariance matrix.
- Principal components are eigenvectors of the covariance matrix (C) of the centered dataset (X), while eigenvectors are not necessarily related to any specific type of matrices.
- Principal components are orthogonal and capture the variance in the data, while eigenvectors do not necessarily have this property.
In conclusion, eigenvectors and principal components stem from different domains (linear algebra and statistics, respectively) but share some similarities. Principal components can be seen as a specific application of eigenvectors, where the main goal is to capture the significant patterns in a dataset.