A decision tree is a flowchart-like model that splits data on feature values, making a series of simple if-then decisions that end in a prediction.
A decision tree recursively partitions the feature space into rectangles. At each node, the algorithm picks the feature and threshold that best separates the target. Leaves store the prediction, either a class label or a numeric value.
That visual structure is why decision trees are the most interpretable model family in common use. You can trace any prediction from root to leaf and explain exactly which conditions led to it. The downside is that a single deep tree memorizes training patterns and overfits aggressively, which is why practitioners almost always prefer ensembles of shallow trees.
Think of it like this. Think of a doctor following a diagnostic flowchart. Each box asks one yes-no question and sends you to the next box until you reach a conclusion. The chart is easy to read, but if every possible patient detail branches into its own path, the chart becomes brittle.
The algorithm evaluates every feature and possible split at each node. It selects the split that maximizes impurity reduction, using metrics such as Gini impurity or entropy for classification and mean squared error for regression. Splitting continues until a stopping condition is met, such as minimum samples per leaf or maximum depth.
"Decision trees are the best model because they are interpretable." A single deep tree is rarely the best predictor. "Trees handle all data types equally well." They are robust to feature scaling, but high-cardinality categorical features can create spurious splits. "More depth is always better." Deep trees overfit and generalize poorly.
Fast, visual, and robust to outliers and scaling, but single trees are unstable and prone to overfitting. Best used as base learners in ensembles or for quick exploratory analysis before more complex models.