ADVERTISEMENT
Google AdSense Slot (headerLeaderboard)

Set your Publisher ID in src/config/ads.ts to display live ads.

Machine Learning with Python Tutorial• 7 min read

Linear Regression: Ordinary Least Squares (OLS)

Slope, intercept, residuals, Mean Squared Error, and R² score

Linear Regression models the linear relationship between continuous dependent variables and one or more independent predictor features by minimizing the sum of squared residuals.

The Linear Equation & Performance Metrics

Model equation: y_pred = w1*x1 + w2*x2 + ... + b.

Mean Squared Error (MSE): Measures average squared difference between predictions and actual values: MSE = (1/n) * Σ(y - y_pred)^2.

R² Score (Coefficient of Determination): The proportion of variance in y explained by features X (1.0 = perfect model, 0.0 = baseline mean predictor).

Evaluating Linear Regression with Scikit-Learn
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error, r2_score
import numpy as np

X = np.array([[1], [2], [3], [4], [5]])
y = np.array([2.1, 3.9, 6.2, 7.8, 10.1])

model = LinearRegression()
model.fit(X, y)
predictions = model.predict(X)

print(f"Slope (w): {model.coef_[0]:.3f}")
print(f"Intercept (b): {model.intercept_:.3f}")
print(f"MSE: {mean_squared_error(y, predictions):.4f}")
print(f"R² Score: {r2_score(y, predictions):.4f}")
R² score close to 1.0 indicates strong explanatory power.
ADVERTISEMENT
Google AdSense Slot (inContentBanner)

Set your Publisher ID in src/config/ads.ts to display live ads.

Test Yourself With Exercise

+15 XP

What does an R² (R-squared) score of 0.85 indicate for a linear regression model?