Principal Component Analysis finds orthogonal axes of maximum variance in data and projects it onto fewer dimensions while preserving as much information as possible.
Principal Component Analysis computes eigenvectors of the data covariance matrix and orders them by eigenvalue magnitude. The top eigenvectors become new axes called principal components. Projecting data onto the first few components reduces dimensionality while retaining most variance.
That variance-maximizing property makes PCA a natural first step before visualization, noise reduction, or feeding high-dimensional data into a downstream model. It is unsupervised and linear, so it cannot capture curved manifolds, but it remains a baseline because it is fast, deterministic, and interpretable.
Think of it like this. Think of photographing a rotating sculpture from one ideal angle. PCA finds the viewpoint that keeps the most recognizable shape in the fewest pixels, discarding redundant views.
Data is centered by subtracting the mean. The covariance matrix captures how features vary together. Eigendecomposition returns eigenvectors and eigenvalues. The top eigenvectors by eigenvalue become principal components. Projection onto these components produces lower-dimensional coordinates. Explained variance quantifies how much information each component preserves.
"PCA always improves downstream performance." It can remove signal if variance is not aligned with the task. "PCA is only for visualization." It is also used for compression and noise reduction. "More components is better." Beyond the elbow, added components mostly add noise and overfit risk.
Fast, deterministic, and interpretable, but only captures linear structure. Works best when the most important variance aligns with the task; less effective when important signals are nonlinear or directionally subtle.