An epoch is one complete pass through the training dataset. After one epoch, the model has seen every training example exactly once.
Most models need many passes, because one look at each example is not enough to settle the weights. The question is how many, and the answer is decided by watching validation performance rather than by picking a number.
Think of it like this. Think of reading a textbook cover to cover. One read gives you the shape of it. Several reads make it stick. Fifteen reads and you are memorising the page layout rather than the subject.
How it works
- One epoch equals every example once.
- An iteration is one batch, so an epoch contains many iterations. Dataset size divided by batch size.
- Order changes each time. Data is normally shuffled between epochs so the model does not learn the sequence.
- Schedules step at epoch boundaries. Learning rate decay and early stopping are usually measured in epochs.
Typical values
- Training from scratch on images: commonly 10 to 100 epochs.
- Fine-tuning a language model: often 1 to 5. More usually makes it worse.
- Transfer learning: 5 to 20.
- Early stopping patience: stop after 5 to 20 epochs with no validation improvement.
Common mistakes
- "An epoch is an iteration." An iteration is one update on one batch. An epoch is a full pass containing many.
- "More epochs means a better model." Past a point it means overfitting. Validation loss tells you when to stop.
- "Pick the epoch count up front." Set a maximum and let early stopping decide the real number.
- "Pretrained models need many epochs." One or two is frequently enough, and more can undo what the model already knew.
