Text Processing Utilities: grep, awk & sed

Filtering with grep, column extraction with awk, and stream editing with sed

« I/O Redirection, Pipes & File Descriptors« Previous Defensive Shell Scripting: set -euo pipefail »Next »

DevOps and system administrators process megabytes of log files daily using the classic Unix trifecta: grep for searching, awk for columnar manipulation, and sed for stream find/replace.

The Big Three Text Utilities

grep -rn "ERROR" /var/log: Recursively searches files for matching regular expressions.

awk '{print $1, $4}' access.log: Slices text into whitespace-delimited columns, printing specific fields.

sed 's/foo/bar/g' file.txt: Stream editor that substitutes text globally (s/find/replace/g).

Extracting IP Addresses from Web Server Logs

Try It Yourself »

# Sample access log line: 192.168.1.45 - - [10/Oct/2024] "GET /api/status HTTP/1.1" 200

# Extract Column 1 (IP address), count occurrences, and find top 5 visitors
cat access.log | awk '{print $1}' | sort | uniq -c | sort -nr | head -n 5

Pipelines combine small tools to create powerful analytical reports.

Google AdSense Slot (inContentBanner)

Set your Publisher ID in src/config/ads.ts to display live ads.

Test Yourself With Exercise

+15 XP

Which utility is specifically optimized for printing specific whitespace-delimited columns from text logs (e.g. `{print $1, $3}`)?

« Previous: I/O Redirection, Pipes & File Descriptors Next: Defensive Shell Scripting: set -euo pipefail »