Containerization & Packaging
A full worked multi-stage Dockerfile, GPU-enabled images, image size discipline, and why floating version tags are a production risk
Containerization & Packaging
TL;DR
A container bundles a model, its code, and every dependency into one reproducible artifact — "works on my machine" becomes "works everywhere this image runs." Multi-stage builds keep the image small by leaving build-time tools out of what actually ships, and pinning every version (base image, dependencies) is what stops a rebuild six months from now silently producing a different image.
| Property | Value |
|---|---|
| Level | Beginner |
| Reading time | ~17 minutes |
| Prerequisites | Model Serving Patterns |
| You will understand | How to package an ML service reproducibly, and why image size and version pinning matter more than they seem to |
A Full Worked Dockerfile
# Stage 1: builder — has the compilers and headers needed to install dependencies
FROM python:3.12.7-slim AS builder
WORKDIR /build
COPY requirements.txt .
RUN pip install --no-cache-dir --target=/build/deps -r requirements.txt
# Stage 2: runtime — only what's needed to actually run the service
FROM python:3.12.7-slim
WORKDIR /app
COPY --from=builder /build/deps /usr/local/lib/python3.12/site-packages
COPY . .
EXPOSE 8000
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]Why the builder stage exists at all: some Python packages need to compile native code during installation, which requires compilers and headers that add hundreds of megabytes and aren't needed once the compiled package exists. The builder stage pays that cost once; the final stage only copies the result — the installed packages — leaving the compilers behind entirely.
GPU-Enabled Containers
A GPU-backed service needs a base image with CUDA already installed and matched to the host's driver version, plus the NVIDIA Container Toolkit on the host so the container can actually see the GPU:
FROM nvidia/cuda:12.4.1-cudnn-runtime-ubuntu22.04
RUN apt-get update && apt-get install -y python3.12 python3-pip
# ...docker run --gpus all my-model-service:latestGPU images are typically several times larger than CPU-only ones, because CUDA and cuDNN libraries alone add gigabytes before any application code is added. Budget for this explicitly — it directly affects the pull-time cost discussed below, and it's a real reason to pick the minimal CUDA base tag (-runtime, not -devel) that still has what your dependencies need.
Image Size Discipline
Image size isn't just a storage line item — it's cold-start latency. Every time a new pod starts (a deploy, a scale-up event, a node replacement), the container runtime has to pull the image before anything can run. A 4GB image pulling over a constrained network can add tens of seconds to exactly the moment you need capacity fastest — during a traffic spike.
| Technique | Effect |
|---|---|
| Multi-stage builds | Leaves compilers and build artifacts out of the final image entirely |
--no-cache-dir on pip install | Skips caching downloaded wheels inside the image layer |
| Slim/minimal base images | python:3.12-slim instead of the full python:3.12, saves hundreds of MB of tools you don't need at runtime |
| Remove build tools from the final stage | If a stage genuinely needs gcc or build-essential, that stage shouldn't be the one that ships |
.dockerignore | Keeps local artifacts (.git, __pycache__, local venvs) from bloating the build context and layers |
Docker Compose for Local Development
Production is rarely one service — it's an API plus a cache, a vector database, or a feature store. Compose lets local development mirror that shape instead of mocking it away:
services:
api:
build: .
ports: ["8000:8000"]
environment:
REDIS_URL: redis://cache:6379
depends_on: [cache]
cache:
image: redis:7.4-alpine
ports: ["6379:6379"]Developing against a mocked-out dependency and only meeting the real one in production is a common source of "works locally, fails in prod" bugs. Compose costs almost nothing to set up and closes that gap — the same multi-service shape runs on a laptop as in the deployed environment.
Pinning Versions Is Not Optional
# Risky — "latest" and unpinned dependencies mean this build isn't reproducible
FROM python:latest
RUN pip install fastapi
# Reproducible — the same Dockerfile produces the same image, today or in six months
FROM python:3.12.7-slim
RUN pip install fastapi==0.115.6This is the same "main moves" lesson that shows up throughout these tracks, applied to container images. python:latest and an unpinned pip install mean the exact same Dockerfile can produce a materially different image next month, when a new Python release or a new library version ships. A build that isn't reproducible is a debugging session waiting to happen — "it worked in CI yesterday" with no code change to explain why it doesn't today.
Concept Checks
Check yourself
Why does a multi-stage Dockerfile end up smaller than a single-stage one installing the exact same dependencies?
Because the compilers, headers, and other build-time tools some dependencies need to install are only present in the builder stage — the final stage copies over just the installed result (the compiled packages), never the tools that produced them. A single-stage build has no way to discard those build-time tools once they've served their purpose, so they ship in the final image whether anything still needs them or not.
Why does image size matter for latency, not just storage cost?
Because every new pod — from a deploy, a scale-up event, or a node replacement — has to pull the image before it can start serving, and a larger image takes longer to pull. That pull time adds directly to cold-start latency at exactly the moment capacity is needed fastest, such as during a traffic spike, which is a very different cost than the comparatively minor expense of storing a larger image at rest.
A Dockerfile uses `FROM python:latest` and `RUN pip install fastapi` with no version pin. What breaks, and when?
Nothing breaks immediately — the risk is that rebuilding the exact same Dockerfile at a later date can silently produce a different image, because latest and an unpinned install resolve to whatever versions exist at build time. Months later, a routine rebuild can pull in a new Python version or a new FastAPI release with different behavior, producing a "why did this change, nothing in the code changed" debugging session with no actual code change to point to.
Key Concepts Recap
| Concept | One-line summary |
|---|---|
| Multi-stage builds | Keep build-time tools out of the image that actually ships |
| GPU images | Need a CUDA-matched base and the NVIDIA Container Toolkit on the host; typically much larger |
| Image size = cold-start latency | Every new pod pays the pull-time cost, especially painful during a scale-up |
| Docker Compose | Mirrors production's multi-service shape locally, closing a common "works locally" gap |
| Version pinning | The same reproducibility discipline as pinning a model revision, applied to images and dependencies |
.dockerignore | Keeps local artifacts out of the build context and final layers |
Next
With the service packaged reproducibly, the next question is running many of them reliably: Orchestration & Scaling.