Handling Missing Data: isna, dropna & fillna

Detecting NaN values, imputation strategies, and forward/backward filling

« Data Selection: loc, iloc & the query() Method« Previous GroupBy & The Split-Apply-Combine Pattern »Next »

Real-world data is plagued with missing values represented as np.nan or None. Pandas offers rich tools to detect, remove, or impute them.

Missing Data Toolset

df.isna().sum(): Counts missing null values per column.

df.dropna(subset=[...], thresh=...): Drops rows or columns containing missing values.

df.fillna(value): Replaces missing values with a scalar, mean, median, or mode.

df.ffill() and df.bfill(): Forward fills or backward fills missing values, indispensable for financial time-series.

Cleaning and Imputing Missing Data

Try It Yourself »

import pandas as pd
import numpy as np

df = pd.DataFrame({
    "user": ["Alice", "Bob", "Charlie", "David"],
    "score": [92.0, np.nan, 78.0, np.nan],
    "department": ["Eng", "Sales", np.nan, "Eng"]
})

print("Null count per column:
", df.isna().sum())

# Impute missing scores with the column median
median_score = df["score"].median()
df["score"] = df["score"].fillna(median_score)

# Fill missing categorical departments with "Unassigned"
df["department"] = df["department"].fillna("Unassigned")
print("
Cleaned DataFrame:
", df)

Imputing medians avoids skewing distributions with outliers.

Google AdSense Slot (inContentBanner)

Set your Publisher ID in src/config/ads.ts to display live ads.

Test Yourself With Exercise

+15 XP

Which method fills missing NaN values with the last valid preceding observation in the column?

« Previous: Data Selection: loc, iloc & the query() Method Next: GroupBy & The Split-Apply-Combine Pattern »