Model Registry & Versioning
What a model registry actually stores, how models move through stages, and why "which model is in production" must be a queryable fact
Model Registry & Versioning
TL;DR
A model registry is the versioned, staged source of truth for every trained model — the artifact, its evaluation metrics, its training metadata, and which stage it's in (Staging, Production, Archived). Without one, "which model is actually live" is tribal knowledge that evaporates the moment the person who deployed it is unavailable.
| Property | Value |
|---|---|
| Level | Intermediate |
| Reading time | ~17 minutes |
| Prerequisites | CI/CD for ML |
| You will understand | What a registry tracks, how stage transitions work, and why this discipline prevents a specific, expensive kind of incident |
What a Registry Actually Stores
A registry entry is not just a file — it's a bundle of everything you'd need to answer "is this model trustworthy, and what exactly is it" without asking a person.
| Field | Why it's there |
|---|---|
| The model artifact (or a pointer to it) | The actual weights/serialized model |
| Evaluation metrics | The numbers from its validation run — accuracy, F1, latency, whatever the CI/CD validation gate checked |
| Training metadata | Training data version, hyperparameters, code/commit version that produced it |
| Stage | Where this version currently sits in its lifecycle |
| Lineage | Which run produced it, and which previous version it's being compared against |
A registry entry with no metrics or training metadata is barely better than a filename. The value of a registry isn't storage — it's that every version carries enough context to answer "should I trust this" and "what changed since the last one" without archaeology through old commits and Slack messages.
Stage Transitions as a Real Workflow
A model's life in the registry
import mlflow
with mlflow.start_run():
mlflow.log_params({"lr": 2e-5, "epochs": 3})
mlflow.log_metrics({"eval_f1": 0.91, "eval_latency_p95_ms": 84})
mlflow.pytorch.log_model(model, artifact_path="model", registered_model_name="ticket-classifier")
client = mlflow.MlflowClient()
client.transition_model_version_stage(
name="ticket-classifier", version=12, stage="Staging",
)
# ... validation gates pass ...
client.transition_model_version_stage(
name="ticket-classifier", version=12, stage="Production",
archive_existing_versions=True, # v11 moves to Archived, not deleted
)MLflow is used here as a concrete, representative example — the pattern (log metrics and metadata alongside the artifact, move through named stages, keep the previous version instead of deleting it) is what matters, not the specific tool.
archive_existing_versions=True is doing real work, not just tidying up. The moment a new version becomes Production, the previous one becomes the fastest possible rollback path — reverting is "point traffic back at v11," not "find the old weights file someone has on their laptop and hope it's the right one."
Artifact Storage Is a Different Problem Than Metadata Storage
Where things actually live
The registry database
RecommendedStores metadata: metrics, parameters, stage, lineage, and a pointer to where the artifact lives. Small, fast to query, this is what a "registry" mostly means in practice.
Object storage (S3, GCS, or equivalent)
RecommendedStores the actual multi-gigabyte model file. A database is the wrong tool for this — object storage is built for large binary blobs and cheap at scale.
Confusing the two — trying to put model weights directly into a database row, or trying to make object storage answer "what's the eval F1 of this version" — is a common early mistake. Keep metadata queryable and artifacts in blob storage, linked by a pointer.
The Incident This Discipline Prevents
A concrete scenario: A new model version is deployed Friday afternoon. Monday morning, a data scientist notices a quality regression in a downstream report. Nobody currently working can say with certainty which model version was live before Friday's deploy — the person who pushed it is out, the deploy script didn't log a version tag, and three candidate weight files sit in a shared drive with names like model_final_v2_USE_THIS.pt. The rollback that should take two minutes takes most of a day, spent reconstructing what "before" even was.
A registry makes this a non-event: the previous Production version is sitting in Archived, its exact identity and metrics are one query away, and rolling back is pointing traffic at a known artifact rather than reconstructing history under pressure.
Concept Checks
Check yourself
Why does a registry entry need evaluation metrics and training metadata, not just the model file itself?
Because the value of a registry is answering "should I trust this version, and what makes it different from the last one" without manual archaeology. A bare model file with no metrics or metadata provides no way to judge trustworthiness or compare it to alternatives — it's just a filename with extra steps.
Why archive the previous Production version instead of deleting it once a new version is promoted?
Because the archived version is the fastest available rollback target — if the new version regresses, reverting means pointing traffic back at a known-good, already-validated artifact rather than trying to reconstruct or retrain what was previously live. Deleting it trades a small storage saving for a potentially very costly incident-response delay.
Why shouldn't large model weight files be stored directly in the registry's database?
Because a database is built for fast queries over structured metadata, not for storing and serving multi-gigabyte binary blobs efficiently. Object storage is purpose-built for that, so the registry pattern splits the two: metadata and a pointer live in the queryable registry, and the actual artifact lives in object storage.
Key Concepts Recap
| Concept | One-line summary |
|---|---|
| Registry entry | Artifact pointer + metrics + training metadata + stage, not just a file |
| Stage workflow | New version → Staging → Production → Archived |
| Archiving, not deleting | The previous Production version is kept as an instant rollback target |
| MLflow example | log_metrics, log_model, transition_model_version_stage — representative of the pattern |
| Metadata vs artifact storage | Registry database for queryable metadata; object storage for the large binary file |
| The incident this prevents | A slow, uncertain rollback because nobody can confirm what was previously live |
Next
With versioning solved, the next question is how to know a deployed model is actually still working: Monitoring & Observability.
CI/CD for ML
What ordinary software CI/CD checks, what a model validation gate adds, and blue-green vs canary deploys for a model change
Monitoring & Observability
System metrics, model quality signals, and drift — the three layers of watching a production model, and why only some of them catch a silent failure