Your personal operating system for career transition.

Generate Kubernetes Manifests for an LLM Service

Deploying an LLM service to Kubernetes requires careful manifest design. The key differences from a typical web service:

• Slow startup: Even API-gateway-style LLM services can be slow to initialize (loading embedding models, warming caches). The startupProbe must allow enough time.
• Memory sensitivity: LLM services hold large in-memory caches. Pod memory limits need to be generous.
• Config via secrets: API keys must come from Kubernetes Secrets, never plain env vars in the manifest.
• Graceful shutdown: In-flight LLM calls can take 10-30 seconds. terminationGracePeriodSeconds must accommodate this.

What to build

Write Python functions that generate Kubernetes manifest dictionaries:

1. build_deployment(config: ServiceConfig) -> dict — a Deployment manifest with startup/liveness/readiness probes, resource limits, secret env vars, and terminationGracePeriodSeconds: 45.
2. build_service(config: ServiceConfig) -> dict — a ClusterIP Service manifest.
3. build_hpa(config: ServiceConfig) -> dict — a HorizontalPodAutoscaler (autoscaling/v2) targeting 70% CPU utilization.

The startup probe must use failureThreshold: 30 + periodSeconds: 10 (5 minutes total).

Constraints

• Return plain Python dicts (JSON-serializable).
• API keys injected via secretKeyRef from config.secret_name.
• Use apps/v1 for Deployment, autoscaling/v2 for HPA.