AI Engineer Portal

Your personal operating system for career transition.

Private mode

Exercise

Implement semantic chunking with overlap

Implement Semantic Chunking with Overlap

Fixed-size chunking is fast but blind to sentence boundaries — it cuts paragraphs in half, splits mid-sentence, and produces chunks where neither half carries the full context. Semantic chunking detects topic shifts using embedding similarity, then splits at the boundaries where meaning changes. Combined with overlap, this produces index-ready chunks that each contain a coherent unit of thought.

What you are building

Create a SemanticChunker class that:

Splits text into sentences using a simple rule-based splitter (sentence-ending punctuation followed by whitespace and a capital letter, or end-of-string).
Groups consecutive sentences into a window of size window_size and computes an embedding for each window.
Detects topic shift boundaries where the cosine similarity between consecutive windows drops below a configurable threshold.
Merges sentences into chunks between detected boundaries, up to a max_chunk_size character limit.
Adds overlap by copying the last overlap_sentences sentences from each chunk into the start of the next chunk.
Attaches metadata to each chunk: source identifier, chunk index, character start/end, and the sentences it contains.

Why this matters

Semantic chunking is the approach used in production systems where chunk quality directly affects answer quality — legal documents, medical content, technical specifications. Understanding how it works means you can tune the threshold and window size for your corpus rather than accepting a library default.

Constraints

Mock the embedding call with a simple hash-based fake that returns a deterministic vector — the exercise is about the chunking logic, not the API.
All type hints required.
The chunk method should accept a plain string and return list[Chunk].
Test with a multi-paragraph document where topic shifts are visible.

Rag Systems / medium / Step 1 of 6

Practice stage

General drill

Hint

Keep the solution explicit and reviewable.

Next drill

Build a hybrid retrieval pipeline

Success criteria

Make the solution explicit, debuggable, and easy to explain.

Review checklist

Review where the boundary is, what gets validated, and what would be hard to debug later.

from __future__ import annotations

import math
import re
from dataclasses import dataclass, field
from typing import Callable

@dataclass
class Chunk:
    """A text chunk with provenance metadata."""
    text: str
    metadata: dict = field(default_factory=dict)

def cosine_similarity(a: list[float], b: list[float]) -> float:
    """Compute cosine similarity between two equal-length float vectors."""
    # TODO: implement dot product / (|a| * |b|)
    raise NotImplementedError

def split_into_sentences(text: str) -> list[str]:
    """
    Split text into sentences on '.', '!', '?' followed by whitespace + capital,
    or end of string. Returns non-empty stripped sentences.
    """
    # TODO: use re.split with a lookahead pattern
    raise NotImplementedError

class SemanticChunker:
    def __init__(
        self,
        embed_fn: Callable[[str], list[float]],
        similarity_threshold: float = 0.85,
        window_size: int = 2,
        max_chunk_size: int = 800,
        overlap_sentences: int = 1,
    ):
        self.embed_fn = embed_fn
        self.similarity_threshold = similarity_threshold
        self.window_size = window_size
        self.max_chunk_size = max_chunk_size
        self.overlap_sentences = overlap_sentences

def chunk(self, text: str, source: str = "") -> list[Chunk]:
        """
        Chunk text using semantic boundary detection.
        Returns a list of Chunk objects with metadata.
        """
        sentences = split_into_sentences(text)
        if not sentences:
            return []

# TODO: compute a sliding window embedding for each sentence position
        # window at index i covers sentences[i : i + window_size]
        # embed the joined window text

# TODO: compute cosine similarity between consecutive window embeddings

# TODO: identify boundary positions where similarity < threshold

# TODO: group sentences into chunks between boundaries
        # respect max_chunk_size by splitting further if needed

# TODO: add overlap_sentences from end of previous chunk to start of next

# TODO: build Chunk objects with metadata
        raise NotImplementedError

Practice

Generate a variation

Generate a new exercise variation to deepen understanding or practice a related concept.

Attempt history

Recent submissions

Before you submit, decide what a strong answer should make obvious to the reviewer.

No attempts yet.