Set your Publisher ID in src/config/ads.ts to display live ads.
Decision Trees & Random Forest Ensembles
Gini impurity, tree depth, bagging ensembles, and feature importances
Decision Trees split data hierarchically based on feature threshold rules. Random Forests aggregate hundreds of diverse trees (Bagging) to eliminate overfitting.
From Single Trees to Random Forests
Decision Tree: Easy to visualize and explain, but prone to severe overfitting if depth is unconstrained.
Gini Impurity: Measures how mixed classes are at a node (0.0 = pure node where all samples belong to one class).
Random Forest: Trains N trees on random bootstrap subsets of data with random feature subsets. Predictions are averaged across all trees, yielding exceptional accuracy and resilience.
Feature Importance: Random forests automatically rank features by how effectively they reduce node impurity.
from sklearn.ensemble import RandomForestClassifier
import numpy as np
# Synthetic features: [age, salary, credit_score]
X = np.array([[25, 40000, 650], [45, 90000, 780], [35, 60000, 710], [22, 25000, 580]])
y = np.array([0, 1, 1, 0])
rf = RandomForestClassifier(n_estimators=100, max_depth=3, random_state=42)
rf.fit(X, y)
# Print feature importance ranking
feature_names = ["Age", "Salary", "Credit Score"]
for name, importance in zip(feature_names, rf.feature_importances_):
print(f"Feature '{name}': {importance * 100:.1f}%")Set your Publisher ID in src/config/ads.ts to display live ads.
Test Yourself With Exercise
Why does a Random Forest generalize significantly better to unseen data than an individual unconstrained Decision Tree?