Cross-validation estimates how well a model will do on new data by training it several times on different splits and averaging the results.
A single train and test split can be lucky or unlucky. Cross-validation runs the split several ways, so your estimate no longer depends on which rows happened to land in the test set.
Think of it like this. Think of judging a restaurant. One meal could be a good night or a bad one. Five visits on different days gives you something you can actually rely on.
How it works
- Split the data into k equal folds, usually five or ten.
- Train k times. Each run holds out one fold and trains on the rest.
- Score each held-out fold.
- Average the k scores. That average is your estimate, and the spread across folds tells you how stable it is.
Types
- K-fold. The standard. Five or ten folds balances reliability against compute.
- Stratified k-fold. Keeps the class proportions the same in every fold. Essential for imbalanced classification.
- Leave-one-out. k equals the number of examples. Nearly unbiased, high variance, and very expensive.
- Time series split. Always trains on the past and tests on the future. Random folds leak future information and are simply wrong for time-ordered data.
Common mistakes
- "It evaluates my model." It evaluates the procedure that produces models. Each fold trains a different one.
- "Use the same folds to tune and to report." Tuning against your folds means the final number is optimistic. Nested cross-validation is the honest version.
- "More folds is better." More folds costs more and raises variance in the estimate. Five or ten is the usual sweet spot.
- "Shuffling is optional." Data often arrives sorted. Unshuffled folds can end up with skewed classes or leak structure.
