Portfolio slice: ship one narrow assistant well

Why this matters

The most common portfolio mistake for AI engineers transitioning from full-stack work is the vague chatbot. A "general purpose assistant" demonstrates that you can make API calls. A narrow assistant that does one thing well demonstrates that you can build a product.

In interviews, a narrow, polished LLM feature is more impressive than a broad but shallow one. It shows product judgment (you chose a workflow worth automating), engineering discipline (you handled edge cases and failures), and evaluation maturity (you know what "working well" actually means).

Core concepts

Choosing the right first LLM feature. The best first feature has three properties:

1. Narrow scope — the input and output are well-defined. "Summarize this PR description into bullet points" is narrow. "Help me with code" is not.
2. Clear success criteria — you can tell whether the output is good without expert judgment. "Did the summary capture the three main changes?" is testable. "Is this a good response?" is not.
3. Existing user workflow — it fits into something users already do, so adoption does not require behavior change. Drafting a response to a customer email fits into a workflow the support agent is already doing.

Good first features:

• PR summary generator (input: diff/description, output: 3-bullet summary)
• Meeting notes → action items extractor
• Code review comment generator (input: diff, output: review comments in a standard format)
• Customer email classifier + draft responder
• Job description → resume tailoring suggestions

Evaluation before launch. Before shipping, build a small eval set:

• 20–50 representative inputs
• For each: what is the expected output? What makes a response good or bad?
• Run the eval set before launch, and again after every prompt change

For a PR summarizer, good metrics are:

• Coverage: did the summary mention the key changes?
• Precision: did the summary avoid hallucinating changes that are not in the diff?
• Format compliance: did it produce exactly 3 bullets?
• Length: are the bullets appropriately concise?

You can automate most of these checks.

UX for AI features. Three UX patterns that separate professional AI features from demos:

1. Loading states with streaming. Users tolerate 10+ seconds of generation if they see text appearing. Silence for 5 seconds kills trust. Use streaming with a visible cursor or progress indicator.

2. Confidence signals. If your feature produces a result with low confidence, show that. A subtle "review carefully" indicator or confidence badge helps users calibrate trust.

3. Edit + accept pattern. Never auto-apply AI output. Show the suggestion, let the user edit it, then let them accept. This makes errors recoverable and builds trust over time.

Working example

Design doc for a narrow PR summary assistant:

# PR Summary Assistant — Design Doc

## Problem
Engineers spend 2–5 minutes writing PR descriptions that reviewers often ignore.
Reviewers spend time understanding what changed without a clear summary.

## Scope (narrow)
Input: PR title + diff (up to 200 lines)
Output: exactly 3 bullets covering: what changed, why it changed, what to review carefully

## Not in scope
- Suggesting code improvements
- Checking for bugs
- Generating test cases
- Summarizing long diffs (>200 lines shows a warning instead)

## Evaluation plan
Golden set: 30 real PRs from team history, manually labeled
Metrics:
  - Coverage score: do bullets mention the key files/features changed? (automated via keyword check)
  - Hallucination flag: does the summary mention things NOT in the diff? (automated via string matching)
  - Format pass: exactly 3 bullets, each <80 chars? (automated)
  - Human quality score: 1-3 rating from 3 team members (sampled weekly)
Target: >85% format pass, >80% coverage, 0 hallucination flags, avg quality >2.2

## Failure modes and mitigations
- Diff too long: show a warning, offer to summarize the first 200 lines only
- No meaningful changes (formatting only): detect and show "minor cleanup" summary
- Model refuses (unlikely): fall back to extracting the PR title + first 3 commit messages

## UX
- Auto-generate on PR creation, shown in a draft state
- User can edit before submitting
- Show token count and regenerate button in dev mode
- No streaming needed (output is short, <1s on Haiku)

## Cost estimate
- Model: claude-haiku-4-5
- Avg tokens: ~800 input (diff) + ~150 output (3 bullets)
- Cost per summary: ~$0.00064
- At 200 PRs/day: ~$0.13/day — negligible

This design doc is the portfolio artifact. It shows:

• You chose a specific problem worth solving
• You defined success criteria before building
• You thought about failure modes
• You did the cost math

Common mistakes

1. Starting with the prompt before defining success. If you do not know what good looks like before you write the prompt, you will iterate in circles. Define your eval criteria first, then write the prompt to satisfy them.

2. Building for the demo, not for edge cases. The demo uses a clean 50-line diff. Production users will paste a 2000-line diff. Build for the 95th percentile input, not the median.

3. Shipping without a feedback loop. The first version of any LLM feature will have issues you did not anticipate. Build a way to collect user feedback (thumbs up/down, edit rate, regeneration rate) from day one.

4. Too much scope for the first version. "Code review assistant" is too broad. "Generate a comment pointing out any missing error handling in a Python function" is shippable. Narrow scope lets you evaluate quality precisely.

5. Treating the first prompt as final. Plan to iterate on the prompt at least 5 times in the first two weeks based on real usage. The first prompt is a hypothesis, not a solution.

Try it yourself

Write the design doc for one narrow LLM feature you could build in a weekend. Pick something you genuinely need or would use at work. Define the success criteria before writing any code or prompts. What are your 20 golden-set examples? How will you measure format compliance, hallucination rate, and quality? Share the design doc in a code review before you start building — the review feedback will shape the feature more than the first version of the prompt.