Why this matters

Personally Identifiable Information flowing through AI systems creates legal exposure under GDPR, CCPA, and sector-specific regulations. When a user submits a support ticket containing their SSN and your LLM echoes it back in the response, that is both a privacy failure and potentially a regulatory violation.

The problem compounds in AI systems because the model may surface PII from one user in a response to another user if the data ends up in the training pipeline, in a shared cache, or in retrieved documents. Defense requires detecting PII at input, stripping it before it reaches the model, and re-checking outputs.

Core concepts

Regex-based PII detection

Regex catches well-formatted PII: structured patterns like SSNs, credit card numbers, emails, and US phone numbers.

import re
from dataclasses import dataclass


@dataclass
class PIIMatch:
    pii_type: str
    value: str
    start: int
    end: int


PII_PATTERNS = {
    "email": re.compile(
        r"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}"
    ),
    "ssn": re.compile(
        r"\b(?!000|666|9\d{2})\d{3}-(?!00)\d{2}-(?!0{4})\d{4}\b"
    ),
    "credit_card": re.compile(
        r"\b(?:4[0-9]{12}(?:[0-9]{3})?|"   # Visa
        r"5[1-5][0-9]{14}|"                   # Mastercard
        r"3[47][0-9]{13}|"                    # Amex
        r"6(?:011|5[0-9]{2})[0-9]{12})\b"   # Discover
    ),
    "us_phone": re.compile(
        r"\b(?:\+1[-.\s]?)?\(?([2-9][0-9]{2})\)?[-.\s]?([2-9][0-9]{2})[-.\s]?([0-9]{4})\b"
    ),
    "ipv4": re.compile(
        r"\b(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}"
        r"(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\b"
    ),
}


def detect_pii(text: str) -> list[PIIMatch]:
    matches = []
    for pii_type, pattern in PII_PATTERNS.items():
        for m in pattern.finditer(text):
            matches.append(PIIMatch(
                pii_type=pii_type,
                value=m.group(),
                start=m.start(),
                end=m.end(),
            ))
    return sorted(matches, key=lambda x: x.start)

Named entity recognition for unstructured PII

Regex misses PII embedded in natural language: "My name is John Smith and I live at 123 Main Street." NER models catch person names, locations, and organizations.

# Using spaCy (install: pip install spacy && python -m spacy download en_core_web_sm)
import spacy

nlp = spacy.load("en_core_web_sm")

NER_PII_LABELS = {"PERSON", "GPE", "LOC", "ORG", "DATE", "CARDINAL"}


def detect_ner_pii(text: str) -> list[PIIMatch]:
    doc = nlp(text)
    matches = []
    for ent in doc.ents:
        if ent.label_ in NER_PII_LABELS:
            matches.append(PIIMatch(
                pii_type=f"ner_{ent.label_.lower()}",
                value=ent.text,
                start=ent.start_char,
                end=ent.end_char,
            ))
    return matches

Note: en_core_web_sm is fast but not precise. For production, use en_core_web_trf (transformer-based) or a dedicated PII model like Microsoft Presidio.

Redaction strategies

Three options for handling detected PII:

from enum import Enum


class RedactionStrategy(str, Enum):
    MASK = "mask"        # Replace with type label: [EMAIL]
    REPLACE = "replace"  # Replace with realistic fake: [email protected]
    REMOVE = "remove"    # Delete entirely


REPLACEMENTS = {
    "email": "[email protected]",
    "ssn": "XXX-XX-XXXX",
    "credit_card": "XXXX-XXXX-XXXX-XXXX",
    "us_phone": "(555) 000-0000",
    "ipv4": "0.0.0.0",
}


def redact_text(
    text: str,
    matches: list[PIIMatch],
    strategy: RedactionStrategy = RedactionStrategy.MASK,
) -> str:
    if not matches:
        return text

    # Process in reverse order to preserve character positions
    result = list(text)
    for match in sorted(matches, key=lambda x: x.start, reverse=True):
        if strategy == RedactionStrategy.MASK:
            replacement = f"[{match.pii_type.upper()}]"
        elif strategy == RedactionStrategy.REPLACE:
            replacement = REPLACEMENTS.get(match.pii_type, f"[{match.pii_type.upper()}]")
        else:  # REMOVE
            replacement = ""
        result[match.start:match.end] = list(replacement)

    return "".join(result)

The PIIProtector pipeline

@dataclass
class PIIProtectionResult:
    original_length: int
    redacted_text: str
    pii_found: list[PIIMatch]
    strategy_used: RedactionStrategy


class PIIProtector:
    def __init__(
        self,
        strategy: RedactionStrategy = RedactionStrategy.MASK,
        use_ner: bool = False,
    ) -> None:
        self.strategy = strategy
        self.use_ner = use_ner

    def protect(self, text: str) -> PIIProtectionResult:
        matches = detect_pii(text)
        if self.use_ner:
            matches += detect_ner_pii(text)
            # Deduplicate overlapping matches
            matches = _deduplicate_matches(matches)

        redacted = redact_text(text, matches, self.strategy)
        return PIIProtectionResult(
            original_length=len(text),
            redacted_text=redacted,
            pii_found=matches,
            strategy_used=self.strategy,
        )


def _deduplicate_matches(matches: list[PIIMatch]) -> list[PIIMatch]:
    sorted_m = sorted(matches, key=lambda x: x.start)
    result = []
    last_end = -1
    for m in sorted_m:
        if m.start >= last_end:
            result.append(m)
            last_end = m.end
    return result

Compliance awareness

GDPR (EU): Data minimization principle — collect only what you need. If a user submits PII in a query, you must have a lawful basis to process it. Logging raw user inputs containing PII may require explicit consent or a legitimate interest assessment.

CCPA (California): Users have the right to know what personal data you collect and to opt out of sale. AI systems that log prompts are data processors — treat logged PII under the same retention policies as your database.

Practical rule: Apply PIIProtector with MASK strategy before logging any user input. Store the redacted version. If you need the original for debugging, encrypt it at rest with a key managed separately from the log store.

Common mistakes

Running PII detection only on outputs. If PII enters the LLM prompt, it is in your logs, your provider's request logs, and potentially in fine-tuning data. Detect and redact at input, not just at output.

Regex without NER for person names. "Please help John Smith with his account" contains PII that zero regexes will catch. NER is not optional for unstructured support-style inputs.

Logging the original text alongside the redacted version in the same record. If both are in the same log line, you have not protected anything. Log only the redacted version in operational logs.

Treating GDPR/CCPA as a legal team problem. Engineers make the daily decisions about what gets logged, how long it is retained, and who can query it. The legal team sets policy; you implement it.

Try it yourself

1. Run detect_pii against a paragraph containing one email, one SSN, one credit card number, and one phone number embedded in natural language sentences. Verify all four are detected.
2. Apply all three redaction strategies to the same text and compare the outputs. Note which strategy is most appropriate for: (a) sending to an LLM, (b) storing in a debug log, (c) displaying back to the user.
3. Add a DATE_OF_BIRTH regex pattern (format: MM/DD/YYYY) to PII_PATTERNS and test it against five date strings — make sure it does not match dates that are not DOB-formatted.