Set your Publisher ID in src/config/ads.ts to display live ads.
Overfitting, Regularization (L1/L2) & Cross-Validation
Bias-Variance tradeoff, Ridge, Lasso, and K-Fold Cross Validation
Generalization is the core objective of machine learning. Overfitting occurs when a model memorizes training noise rather than true underlying patterns.
Combating Overfitting with Regularization
Bias-Variance Tradeoff: High bias models underfit (too simple); high variance models overfit (too complex).
L2 Regularization (Ridge): Penalizes large weights by adding λ * Σ(w^2) to the loss, shrinking coefficients smoothly toward zero.
L1 Regularization (Lasso): Penalizes absolute weights by adding λ * Σ|w|, forcing non-essential weights to exactly zero for automatic feature selection.
K-Fold Cross-Validation: Splits data into K folds, iteratively training on K-1 and testing on 1, producing an unbiased estimate of model performance across the full dataset.
from sklearn.model_selection import cross_val_score
from sklearn.linear_model import Ridge
import numpy as np
X = np.random.randn(100, 5)
y = np.random.randn(100)
# Ridge regression with L2 regularization
model = Ridge(alpha=1.0)
# Evaluate model across 5 distinct cross-validation splits
scores = cross_val_score(model, X, y, cv=5, scoring="r2")
print("Scores per fold:", np.round(scores, 3))
print(f"Mean CV R² Score: {scores.mean():.3f} (± {scores.std():.3f})")Set your Publisher ID in src/config/ads.ts to display live ads.
Test Yourself With Exercise
What unique mathematical property distinguishes L1 Regularization (Lasso) from L2 Regularization (Ridge)?