A support vector machine is a classifier that finds the hyperplane with the maximum margin between classes, using kernel methods to handle nonlinear boundaries when needed.
A support vector machine finds the hyperplane that maximizes the distance to the nearest training examples from each class. Those nearest examples are the support vectors. For nonlinear data, the kernel trick implicitly maps inputs into a higher-dimensional space where a linear separator exists.
That maximum-margin principle gives SVMs strong generalization on clean medium-sized datasets. The kernel trick avoids the cost of explicitly computing high-dimensional coordinates by replacing dot products with kernel functions. The downside is training cost on large datasets and sensitivity to feature scaling and kernel choice.
Think of it like this. Think of building the widest possible road between two neighborhoods. The road edges are defined by the closest houses on each side, not by every house. If the neighborhoods are separated by a river, a bridge is the kernel that makes a straight road possible.
The algorithm solves a quadratic optimization problem to find the hyperplane that maximizes the margin. Only support vectors, the points closest to the boundary, determine the solution. Kernels such as radial basis function or polynomial transform feature interactions implicitly, allowing nonlinear decision boundaries without explicit coordinate expansion.
"SVMs are always better than logistic regression." They excel on small clean datasets with clear margins; logistic regression often wins on noisy or large-scale data. "Kernels make SVMs perfect." Kernel choice and regularization are critical, and bad choices overfit quickly. "SVMs scale to millions of examples." Training complexity is superlinear in sample count.
Strong theoretical guarantees and good performance on small-medium datasets, but training and prediction cost scale poorly with sample count. Best for text classification and medium tabular tasks; less practical for very large datasets.