Why this matters

LangChain is the most widely used RAG framework in production Python codebases. Understanding it deeply matters for two reasons: (1) you will encounter it in existing codebases and need to debug it, and (2) it provides high-quality implementations of document loaders, text splitters, and retrievers that would take days to write from scratch. Knowing which abstractions are worth using and which ones add unnecessary complexity is the judgment that separates engineers who use frameworks well from those who fight them.

This lesson covers the LangChain components that are genuinely useful — document loaders, text splitters, and retrievers — and shows you a complete RAG pipeline built with them. It also covers when to abandon LangChain abstractions and use direct API calls instead.

Core concepts

Document loaders

LangChain's document loaders handle the messy work of reading content from different sources and normalizing it into Document objects (text + metadata). The most useful ones in practice:

PyPDFLoader — extracts text from PDFs page by page. Each page becomes a Document with page and source metadata. Handles multi-page PDFs automatically.

WebBaseLoader — fetches a URL and strips HTML to plain text using BeautifulSoup. Useful for indexing documentation sites or web content.

CSVLoader — each row becomes a Document. Columns become metadata fields. Essential for structured data ingestion.

DirectoryLoader — recursively loads all files in a directory matching a glob pattern. Useful for batch ingestion of a document corpus.

The key advantage of loaders: they produce a consistent Document interface regardless of source format, so your chunking and embedding pipeline does not need to handle format-specific parsing.

Text splitters

Text splitters divide Document objects into chunks. The two most important splitters:

RecursiveCharacterTextSplitter — the default choice for most text. Tries to split on paragraph breaks (\n\n), then line breaks (\n), then sentences (. ), then spaces, then characters — always trying the coarsest split first. Produces naturally-bounded chunks that respect sentence and paragraph structure.

SemanticChunker — uses embedding similarity between consecutive sentences to detect topic boundaries. Splits where similarity drops significantly. Produces chunks that each contain a single coherent topic. Higher quality but much more expensive: requires an embedding call per sentence.

The critical parameters for RecursiveCharacterTextSplitter: chunk_size (target size in characters), chunk_overlap (characters shared between adjacent chunks), and length_function (character count by default, token count if you pass a tokenizer).

Retrievers

LangChain's retriever interface makes it easy to swap search strategies without changing the rest of your pipeline. Every retriever exposes get_relevant_documents(query: str) -> list[Document].

VectorStoreRetriever — wraps any LangChain vector store (FAISS, Chroma, Pinecone, etc.) and does embedding-based similarity search. The default.

BM25Retriever — keyword-based search using the BM25 algorithm. No embeddings, no API calls. Deterministic and fast. Excellent at exact matches and rare terms that embeddings miss.

EnsembleRetriever — combines multiple retrievers with weighted score fusion. The standard setup is 70% vector store + 30% BM25, which consistently outperforms either alone on diverse query sets. This is the hybrid search pattern.

MultiQueryRetriever — generates multiple phrasings of the original query using an LLM, retrieves independently for each, and deduplicates results. Increases recall at the cost of extra LLM calls.

LangChain chains vs custom implementation

LangChain's LCEL (LangChain Expression Language) chains compose the retriever + prompt + LLM into a callable pipeline. They handle orchestration boilerplate and make the pipeline declarative.

Use LangChain chains for: rapid prototyping, standard retrieve-then-generate patterns, built-in tracing via LangSmith.

Write a custom implementation when: you need fine-grained control over prompt formatting, you want to add confidence checking or multi-hop logic (which chains make awkward), or LangChain abstractions obscure what is happening in a way that makes debugging harder.

Working example

from __future__ import annotations

from langchain_community.document_loaders import PyPDFLoader, WebBaseLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_community.vectorstores import FAISS
from langchain_openai import OpenAIEmbeddings, ChatOpenAI
from langchain.retrievers import BM25Retriever, EnsembleRetriever
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables import RunnablePassthrough


# 1. Load documents

def load_pdf(path: str) -> list:
    return PyPDFLoader(path).load()  # list[Document], one per page


def load_url(url: str) -> list:
    return WebBaseLoader(url).load()


# 2. Split into chunks

def split_documents(docs: list, chunk_size: int = 800, overlap: int = 100) -> list:
    splitter = RecursiveCharacterTextSplitter(
        chunk_size=chunk_size,
        chunk_overlap=overlap,
        separators=["\n\n", "\n", ". ", " ", ""],
    )
    return splitter.split_documents(docs)


# 3. Build indexes

def build_faiss_index(chunks: list, embeddings=None) -> FAISS:
    if embeddings is None:
        embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
    return FAISS.from_documents(chunks, embeddings)


def build_ensemble_retriever(chunks: list, faiss_index: FAISS, k: int = 5) -> EnsembleRetriever:
    """
    Combine FAISS (semantic) and BM25 (keyword) with 70/30 weighting.
    EnsembleRetriever uses Reciprocal Rank Fusion to merge results.
    """
    vector_retriever = faiss_index.as_retriever(search_kwargs={"k": k})
    bm25_retriever = BM25Retriever.from_documents(chunks)
    bm25_retriever.k = k
    return EnsembleRetriever(retrievers=[vector_retriever, bm25_retriever], weights=[0.7, 0.3])


# 4. Build RAG chain using LCEL

RAG_PROMPT = ChatPromptTemplate.from_template(
    """Answer the question using only the provided context. If the context does not contain enough information, say "I don't have enough information."

Context:
{context}

Question: {question}

Answer:"""
)


def format_docs(docs: list) -> str:
    return "\n\n---\n\n".join(
        f"[Source: {doc.metadata.get('source', 'unknown')}, page {doc.metadata.get('page', '?')}]\n{doc.page_content}"
        for doc in docs
    )


def build_rag_chain(retriever, model: str = "gpt-4o-mini"):
    llm = ChatOpenAI(model=model, temperature=0.1)
    # LCEL: retriever -> format -> prompt -> LLM -> parse output
    return (
        {"context": retriever | format_docs, "question": RunnablePassthrough()}
        | RAG_PROMPT
        | llm
        | StrOutputParser()
    )


# 5. Full pipeline: load -> split -> index -> query

def build_rag_pipeline(pdf_path: str) -> dict:
    raw_docs = load_pdf(pdf_path)
    chunks = split_documents(raw_docs, chunk_size=800, overlap=100)
    print(f"Loaded {len(raw_docs)} pages -> {len(chunks)} chunks")

    faiss_index = build_faiss_index(chunks)
    retriever = build_ensemble_retriever(chunks, faiss_index, k=5)
    chain = build_rag_chain(retriever)

    return {"chain": chain, "retriever": retriever, "faiss_index": faiss_index, "chunk_count": len(chunks)}


# 6. Usage

if __name__ == "__main__":
    pipeline = build_rag_pipeline("your_document.pdf")
    chain = pipeline["chain"]

    for q in ["What are the main findings?", "What methodology was used?"]:
        print(f"\nQ: {q}")
        print(f"A: {chain.invoke(q)}")

    # Retrieve without generating to inspect what gets surfaced:
    retriever = pipeline["retriever"]
    for i, chunk in enumerate(retriever.get_relevant_documents("main findings")):
        print(f"\nChunk {i+1} (page {chunk.metadata.get('page', '?')}):")
        print(chunk.page_content[:200] + "...")

Common mistakes

1. Using LangChain for everything. LangChain chains handle the simple case well. For self-RAG loops, multi-hop reasoning, or custom confidence logic, you will spend more time working around LangChain abstractions than implementing the logic directly. Know when to drop down to direct API calls.

2. Default chunk size without measuring. LangChain's default chunk_size=4000 is very large. For most retrieval use cases, 512-1000 characters produces better precision. Always measure retrieval quality at a few chunk sizes before settling on one.

3. Ignoring metadata during splitting. When you split a Document, LangChain preserves the parent document's metadata in each chunk. But if you need chunk-level metadata (chunk index, character offset), add it manually after splitting — LangChain does not add it by default.

4. EnsembleRetriever with untuned weights. The 70/30 split is a reasonable default, but optimal weights depend on your query distribution. If your users mostly ask exact-match questions (product codes, names, IDs), increase the BM25 weight. For conceptual questions, increase the vector weight. Evaluate both before shipping.

5. PyPDFLoader on scanned PDFs. PyPDFLoader uses pypdf for text extraction, which fails on scanned images. Scanned PDFs need OCR (Tesseract, AWS Textract, Google Document AI) before loading. Check your document source carefully.

Try it yourself

Pick any PDF you have access to (a paper, a technical document, a report). Build the full LangChain RAG pipeline from this lesson using FAISS and RecursiveCharacterTextSplitter. Write 10 test questions of varying difficulty: some that require a single chunk, some that require synthesizing information from different sections. Measure whether the ensemble retriever (FAISS + BM25) returns better top-5 results than FAISS alone for your question set. Document your findings — which question types benefit most from hybrid search?