Retrieval-Augmented Generation (RAG) Architecture

Chunking, Vector Embeddings, Cosine Similarity, and Context Injection

« Advanced Prompt Engineering & In-Context Learning« Previous Fine-Tuning, LoRA & Alignment (RLHF / DPO) »Next »

RAG bridges the gap between static foundation model weights and dynamic private enterprise data by retrieving relevant document chunks from a vector database and injecting them into the prompt.

The 4 Stages of a Production RAG Pipeline

1Ingestion & Chunking: Raw documents (PDFs, docs, HTML) are split into semantic chunks (e.g., 500 tokens with 50-token overlap).

2Vector Embedding: An embedding model (like text-embedding-3-small) transforms each chunk into a high-dimensional vector (e.g. 1536 floats) representing semantic meaning.

3Vector Retrieval: When a user asks a question, their query is converted to an embedding vector. Cosine similarity or HNSW indexing searches the vector database for the top-k most relevant chunks.

4Augmented Generation: The retrieved chunks are formatted into the LLM system prompt as verified context, eliminating hallucinations and grounding answers in verified sources.

Cosine Similarity Retrieval in Python

Try It Yourself »

import numpy as np

def cosine_similarity(v1, v2):
    return np.dot(v1, v2) / (np.linalg.norm(v1) * np.linalg.norm(v2))

# Mock 3D embeddings
query_vec = np.array([0.9, 0.2, -0.4])
chunk_a = np.array([0.88, 0.25, -0.38]) # Highly relevant
chunk_b = np.array([-0.5, 0.8, 0.3])    # Irrelevant

print("Relevance Chunk A:", round(cosine_similarity(query_vec, chunk_a), 4))
print("Relevance Chunk B:", round(cosine_similarity(query_vec, chunk_b), 4))

Vectors with high cosine similarity (close to 1.0) share deep semantic meaning.

Pro Tip

Always provide chunk overlap (e.g., 10-20%) during ingestion to ensure that boundary sentences are not severed across two disjoint chunks.

Google AdSense Slot (inContentBanner)

Set your Publisher ID in src/config/ads.ts to display live ads.

Test Yourself With Exercise

+15 XP

What is the primary role of a Vector Database in a Retrieval-Augmented Generation (RAG) system?

« Previous: Advanced Prompt Engineering & In-Context Learning Next: Fine-Tuning, LoRA & Alignment (RLHF / DPO) »