A decision tree asks a series of yes/no questions about your data and follows the answers to a prediction.
Complete lesson & earn 250 PX
EXERCISE
1A decision tree asks a series of yes/no questions about your data and follows the answers down a branching path to a prediction.
EXERCISE
2At each branching point, the tree picks the split that creates the most uniform groups — measured by something called the Gini index.
EXERCISE
3Once built, a decision tree predicts by walking a new data point from the root to a leaf — following yes/no answers at each branch.
Think of a like a game of 20 Questions. "Is the comedian ranked above 6.5?" Yes → "Are they from the UK?" No → "Go to the show!" Each question splits the data into smaller, more uniform groups.
The concept:
Is Rank ≤ 6.5?
├── YES → Don't Go (all low-ranked shows were bad)
└── NO → Is Nationality UK?
├── YES → Is Age ≤ 35.5?
│ ├── YES → Go!
│ └── NO → Check Experience...
└── NO → Go! (non-UK, high rank = always good)
Building one in Python:
import pandas as pd
from sklearn.tree import DecisionTreeClassifier
df = pd.read_csv("shows.csv")
# Convert strings to numbers (trees need numerical data)
d = {"UK": 0, "USA": 1, "N": 2}
df["Nationality"] = df["Nationality"].map(d)
d = {"YES": 1, "NO": 0}
df["Go"] = df["Go"].map(d)
# Separate features and target
features = ["Age", "Experience", "Rank", "Nationality"]
X = df[features]
y = df["Go"]
# Build the tree
dtree = DecisionTreeClassifier()
dtree = dtree.fit(X, y)
print("Tree built successfully!")
# Output: Tree built successfully!
Pro Tip: Decision trees are one of the few ML models that are human-readable. You can look at the tree and understand exactly WHY it made a decision. This "explainability" makes decision trees popular in fields where you need to justify predictions — like medicine, finance, and law.
Imagine sorting a bag of mixed red and blue marbles. The best split is one that puts all reds in one group and all blues in the other. The Gini index measures how "pure" each group is — 0.0 means perfectly pure, 0.5 means maximally mixed.
The Gini formula:
Gini = 1 - (proportion_yes)² - (proportion_no)²
# Example: 13 comedians, 6 "NO" and 7 "GO"
total = 13
go = 7
no = 6
gini = 1 - (go/total)**2 - (no/total)**2
print(f"Gini: {round(gini, 3)}")
# Output: Gini: 0.497
# Very mixed — close to 0.5
# After splitting by Rank ≤ 6.5:
# Left branch: 5 comedians, all "NO"
gini_left = 1 - (0/5)**2 - (5/5)**2
print(f"Left Gini: {gini_left}")
# Output: Left Gini: 0.0
# Perfectly pure! All same class.
# Right branch: 8 comedians, 1 "NO" and 7 "GO"
gini_right = 1 - (7/8)**2 - (1/8)**2
print(f"Right Gini: {round(gini_right, 3)}")
# Output: Right Gini: 0.219
# Mostly pure — good split!
What the tree looks for at each step:
| Gini Value | Meaning |
|---|---|
| 0.0 | Pure — all samples have the same class |
| 0.25 | Mostly pure — one class dominates |
| 0.5 | Maximum impurity — 50/50 split |
Pro Tip: The tree tries every possible split on every feature and picks the one with the lowest weighted Gini across both branches. This greedy search is why decision trees are fast to build but not always globally optimal — they make the best choice at each step without looking ahead.
Using a trained decision tree is like following a flowchart. You start at the top, answer each question about your input, and follow the arrow to the next question until you reach a leaf — that leaf is your .
Predicting with the trained tree:
import pandas as pd
from sklearn.tree import DecisionTreeClassifier
df = pd.read_csv("shows.csv")
d = {"UK": 0, "USA": 1, "N": 2}
df["Nationality"] = df["Nationality"].map(d)
d = {"YES": 1, "NO": 0}
df["Go"] = df["Go"].map(d)
features = ["Age", "Experience", "Rank", "Nationality"]
X = df[features]
y = df["Go"]
dtree = DecisionTreeClassifier()
dtree = dtree.fit(X, y)
# Predict: 40yr old American, 10yrs experience, rank 7
result = dtree.predict([[40, 10, 7, 1]])
print(f"Should I go? {'YES' if result[0] == 1 else 'NO'}")
# Output: Should I go? YES
# What if rank drops to 6?
result2 = dtree.predict([[40, 10, 6, 1]])
print(f"Rank 6? {'YES' if result2[0] == 1 else 'NO'}")
# Output: Rank 6? NO
Important caveat — inconsistent results:
# Decision trees are NON-DETERMINISTIC by default!
# Run the same code twice and you might get different trees
# Fix it with random_state:
dtree = DecisionTreeClassifier(random_state=42)
dtree = dtree.fit(X, y)
# Now the tree is reproducible
Pro Tip: Decision trees are non-deterministic — run the same code twice and you might get a different tree (and different predictions). This happens because when two splits are equally good, the algorithm randomly picks one. Always set
random_statefor reproducible results. In production, this is not optional.
Decision trees split data by to outcomes. Use scikit-learn's DecisionTreeClassifier to build one from a CSV.