Set your Publisher ID in src/config/ads.ts to display live ads.
Data Preprocessing & Train/Test Splitting
train_test_split, StandardScaler, OneHotEncoder, and preventing data leakage
To evaluate how a model generalizes to unseen real-world data, datasets are split into distinct training and testing partitions, with features scaled and encoded.
Avoiding Data Leakage in Preprocessing
train_test_split(X, y, test_size=0.2, random_state=42): Holds out 20% of data for unbiased validation.
StandardScaler: Scales features to zero mean and unit variance (z = (x - u) / s).
CRITICAL RULE: Call .fit_transform() on training data ONLY, and .transform() on test data. Never fit a scaler on the entire dataset, which causes data leakage!
OneHotEncoder: Converts categorical text columns (e.g. "Country": ["USA", "CAN"]) into numeric binary indicator vectors.
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
import numpy as np
# Simulated dataset: 100 samples, 2 features
X = np.random.randn(100, 2) * 50 + 100
y = np.random.randint(0, 2, size=100)
# 80% train, 20% test split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Fit scaler strictly on training data
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
# Transform test data using the parameters learned from train
X_test_scaled = scaler.transform(X_test)
print("Scaled train mean:", np.round(X_train_scaled.mean(axis=0), 2))Set your Publisher ID in src/config/ads.ts to display live ads.
Test Yourself With Exercise
Why should you NEVER call `scaler.fit()` on your test dataset?