K-nearest neighbors is a simple instance-based algorithm that classifies or predicts a new example by majority vote or averaging over its closest neighbors in feature space.
K-nearest neighbors stores the training set and makes predictions by finding the k closest examples to a query point using a distance metric. For classification, it takes a majority vote among those neighbors. For regression, it averages their target values.
That lazy design means there is no training phase in the traditional sense. The model is the dataset. Prediction cost grows linearly with dataset size unless you use approximate nearest-neighbor indexes. KNN also exposes the curse of dimensionality clearly: in high dimensions every point is roughly equidistant, so neighborhood search loses meaning.
Think of it like this. Think of a new student joining a class who decides what to wear by looking at the three students closest to them in age, region, and style, then copying the majority choice.
Given a query point, the algorithm computes distances to every training example using Euclidean, Manhattan, or another metric. It selects the k nearest neighbors and aggregates their labels or values. Distance weighting can improve performance by giving closer neighbors more influence than distant ones.
"KNN is a simple model so it is easy to use." Feature scaling and neighbor count matter enormously. "More neighbors is better." Large k over-smooths decision boundaries. "KNN scales well." Exact search is linear or worse without specialized indexes.
No training phase and easy to explain, but prediction cost and memory use grow with dataset size. Works well on small clean data with meaningful distance metrics; degrades in high dimensions without feature selection.