Glossary
Every MLOps term and abbreviation used across the fundamentals track, grouped by theme — from health checks and HPA to drift, gateways, and governance
Glossary
How to use this page
Every term and abbreviation used anywhere in this track, expanded and explained in plain language, grouped by theme rather than alphabetically. Where a concept has a full explanation elsewhere, the entry links to it.
| Property | Value |
|---|---|
| Level | All levels — reference |
| Reading time | Reference; skim or search |
| Prerequisites | Designing an MLOps System |
| You will understand | Every piece of MLOps vocabulary used across the fundamentals track |
Serving & Packaging
| Term | Meaning |
|---|---|
| Health check | An endpoint confirming a service — and ideally its model and dependencies — is actually ready, not just alive. See Model Serving Patterns |
| Synchronous serving | One request in, one response out, on the calling connection — simple, latency-sensitive |
| Async / queue-based serving | Requests are queued and processed independently of the caller's connection — fits long-running inference or smoothing bursts |
| Request batching | Grouping several requests into one model call for better throughput per unit of compute |
| Cold start | The latency cost of starting a new instance — often dominated by loading model weights, which is why GPU-backed services scale differently |
| gRPC | A binary RPC protocol, an alternative to REST, often used for lower-latency internal service-to-service calls |
| Multi-stage build | A Docker pattern keeping build-time tools out of the final shipped image |
| GPU container | A container image with CUDA/GPU driver support baked in, distinct from a lightweight CPU-only image |
| Docker Compose | A tool for defining and running multi-container setups locally, useful for a gateway + cache + model stack |
Orchestration & Scaling
| Term | Meaning |
|---|---|
| Kubernetes (K8s) | The standard orchestration platform for running, scheduling, and scaling containers |
| Pod | The smallest deployable unit in Kubernetes — one or more containers running together |
| Deployment | A Kubernetes object managing a set of replica Pods and their rollout |
| Service | A stable network endpoint routing traffic to a set of Pods |
| HPA (Horizontal Pod Autoscaler) | Scales replica count based on CPU/memory utilization or a custom metric |
| KEDA | Event-driven autoscaling — scales on signals like queue depth or request rate, useful for bursty ML workloads |
| Scale-to-zero | Scaling a workload down to zero replicas when idle — risky for GPU workloads due to cold-start cost |
| Warm minimum replicas | Keeping a baseline of always-on replicas to avoid cold-start latency during traffic spikes |
CI/CD & Registry
| Term | Meaning |
|---|---|
| Validation gate | A CI/CD check that blocks deployment unless a model meets a quality bar, distinct from ordinary code tests. See CI/CD for ML |
| Blue-green deploy | Running two full environments (old and new) and switching traffic over at once, with the old environment available for instant rollback |
| Canary deploy | Releasing a new version to a small fraction of traffic first, monitored before a full rollout |
| Automated rollback | Reverting to the previous version automatically when a canary or post-deploy metric regresses |
| Model registry | The versioned, staged source of truth for trained models. See Model Registry & Versioning |
| Stage (Staging / Production / Archived) | A registry's lifecycle labels for a model version, controlling what's actually serving traffic |
| Artifact storage | Where the actual model files (weights, config) live, referenced by the registry's metadata |
Monitoring & Drift
| Term | Meaning |
|---|---|
| p50 / p95 / p99 latency | Percentile latency — p99 being the value 99% of requests are faster than, which surfaces tail problems averages hide |
| Structured logging | Logging in a consistent, machine-parseable format (e.g. JSON) rather than free-text messages |
| Distributed tracing | Tracking one request's path across multiple services, to see where time and errors actually occur |
| Data drift | The live input distribution diverging from the training distribution. See Data & Model Drift |
| Concept drift | The relationship between inputs and the correct output changing, even if inputs look the same |
| KS-test / Population Stability Index (PSI) | Statistical tests commonly used to quantify how much a distribution has shifted |
| Retraining trigger | A defined condition (a drift threshold, a schedule, an accuracy drop) that initiates retraining, rather than retraining on an ad-hoc basis |
Cost & Experimentation
| Term | Meaning |
|---|---|
| Exact-match cache | Caching keyed on identical requests |
| Semantic cache | Caching keyed on meaning/similarity, so differently-worded but equivalent requests hit the same entry. See Caching & Cost Optimization |
| Similarity threshold | The cutoff deciding whether two requests are "close enough" for a semantic cache hit — too loose returns wrong answers |
| Cost per request | The cost metric that matters for pay-as-you-go inference, tracked explicitly rather than only in aggregate |
| Cache hit rate | The fraction of requests served from cache instead of a real model call |
| A/B test | Comparing two versions via a randomized real-traffic split. See A/B Testing & Experimentation |
| Traffic splitting | Routing a defined fraction of real traffic to each variant under test |
| Statistical significance / p-value | The measure of whether an observed difference is likely real or explainable by chance |
| Multi-armed bandit | An adaptive alternative to a fixed A/B split, shifting traffic toward the better arm as evidence accumulates |
| Exploration vs exploitation | The bandit trade-off between trying under-tested arms and favoring the current best-known one |
LLM Ops & Data
| Term | Meaning |
|---|---|
| LLM gateway | A centralized layer for routing, reliability, and cost-tracking across LLM providers. See LLM Gateway & Routing |
| Rate limiting | Enforcing a request/token ceiling before it's exceeded, rather than discovering it from a provider error |
| Exponential backoff | Spacing retries out increasingly (e.g. 1s, 2s, 4s) instead of retrying immediately and repeatedly |
| Circuit breaker | Stops calling a failing provider for a cooldown period after repeated failures, protecting both sides |
| Fallback chain | An ordered list of alternate providers/models to try when the primary fails |
| Feature store | A system ensuring training and serving compute the same feature the same way. See Feature Stores & Data Pipelines |
| Training-serving skew | A mismatch between offline (training) and online (serving) feature computation |
| Online / offline store | The low-latency serving-time store and the historical batch store, both derived from one feature definition |
| Point-in-time correctness | Training features must reflect their value at the historical moment of each example, not today's value |
| Feature versioning | Tracking which definition of a feature a deployed model depends on, as definitions change over time |
Security & Governance
| Term | Meaning |
|---|---|
| Access control | Rules governing who can deploy, promote, or query what. See Security & Governance |
| PII redaction | Detecting and removing personal information from free text before it reaches a model or third-party provider |
| Audit trail | A record of who did what, when — deploys, promotions, data access, and configuration changes |
| Model governance | Approval workflows for high-stakes model changes, beyond what automated CI/CD gates check |
Terms Most Often Confused
Pairs worth keeping straight
Data drift vs concept drift
Data drift is the input distribution itself changing — different kinds of requests arriving. Concept drift is subtler: the relationship between a given input and the correct output changes, even when the inputs look statistically the same. Data drift is easier to detect from inputs alone; concept drift usually needs a feedback signal on actual outcomes.
Blue-green vs canary deploy
Blue-green switches all traffic to the new version at once, with the old environment kept live for instant rollback. Canary sends only a small fraction of traffic to the new version first, observing it before a full rollout. Canary catches problems with less blast radius; blue-green gives a faster, simpler full cutover once you're confident.
HPA vs KEDA
HPA scales on resource metrics — CPU and memory utilization. KEDA scales on event-driven signals — queue depth, request rate, or custom metrics — which fits bursty, queue-based ML workloads better than CPU alone often does. They can be used together, KEDA driving the signal HPA's mechanism ultimately acts on.
A/B testing vs a multi-armed bandit
An A/B test splits traffic in a fixed ratio for a fixed duration, then makes one decision at the end. A bandit adjusts the traffic split continuously as evidence accumulates, favoring the better-performing arm sooner. A/B testing gives a cleaner statistical answer; a bandit reduces the cost of serving a losing variant during the test itself.
Retries with backoff vs a circuit breaker
Retries with backoff handle a single request's transient failure by trying again with increasing delay. A circuit breaker looks at the pattern across many requests — after enough consecutive failures, it stops sending requests to a provider entirely for a cooldown period. Backoff is a per-request tactic; a circuit breaker is a system-level protection against hammering something that's already down.
Training-serving skew vs model drift
Training-serving skew is a bug — the same feature computed differently in two pipelines, present from day one, that just hadn't been noticed. Drift is a real change over time in the world the model was trained on. They can look identical from a slowly-declining-accuracy chart, which is exactly why the failure modes page treats "check for skew" as a distinct diagnostic step from "check for drift."
Next
That completes the MLOps fundamentals track — when you're ready to put it to work, the Project Ideas brief lays out the gateway project this glossary's terms all belong to. For a fast one-page refresher on any of it, revisit the MLOps Crash Course.