Random Forest is an ensemble method that trains many decision trees on different subsets of data and features, then averages their predictions to produce a more stable and accurate result.
Random Forest builds many decision trees in parallel. Each tree trains on a bootstrap sample of the data and considers only a random subset of features at each split. Final predictions are the majority vote for classification or the average for regression.
That dual randomness is what makes the ensemble robust. Bootstrapping makes trees different because they see different examples. Feature subsampling makes them different because they cannot rely on the same dominant feature at every split. Averaging cancels out the individual mistakes that any single deep tree would memorize.
Think of it like this. Think of asking many doctors independently for a diagnosis rather than following one specialist. Individual opinions may be wrong, but the consensus is usually steadier and less biased.
Bootstrap sampling draws random subsets with replacement for each tree. At each split, only a random subset of features is considered, which decorrelates the trees. Leaf predictions are aggregated by majority vote or mean. Out-of-bag samples provide an internal validation estimate without a separate holdout set.
"More trees always help." After enough trees, gains plateau and inference cost rises. "Random Forest is always better than gradient boosting." Gradient boosting often wins on accuracy; random forest wins on speed and robustness. "Feature importance is exact." It is a useful heuristic but can be biased toward high-cardinality features.
Strong baseline with minimal tuning, robust to overfitting, and easy to parallelize. Training cost scales linearly with tree count, and interpretability is lower than a single tree, but feature importance still provides useful signal.