HuggingFace Crash Course
All of the HuggingFace ecosystem on one page — the Hub, transformers, tokenizers, datasets, fine-tuning, quantization, alignment, distributed training, and shipping a model
HuggingFace Crash Course
Start here
This single page covers all of the HuggingFace ecosystem at working depth. Read it start to finish in about 30 minutes and you will understand what the Hub is, how the core libraries fit together, which decisions actually matter, and what to reach for at each stage from "load a model" to "serve it in production."
Every section ends with a Go deeper link to a full page on that topic. Read this first, then follow the links for whatever you need in detail.
| Property | Value |
|---|---|
| Level | Everyone — starts from zero, ends at production |
| Reading time | ~30 minutes |
| Prerequisites | None. Basic Python helps; PyTorch familiarity helps for the training sections. |
| You will understand | The whole HuggingFace stack, well enough to pick the right library for a task and know why |
1. What HuggingFace Actually Is
HuggingFace is two things that are easy to conflate:
| Part | What it is |
|---|---|
| The Hub | A hosted platform — git-backed repositories for models, datasets, and Spaces (running apps). Over a million models, hundreds of thousands of datasets and Spaces. |
| The libraries | A dozen-odd open-source Python packages that load, train, evaluate, and serve things stored on the Hub — transformers, datasets, tokenizers, peft, trl, accelerate, diffusers, evaluate, huggingface_hub, gradio, safetensors, smolagents, trackio. |
The one-sentence definition
HuggingFace is the package manager for pretrained models. The Hub is the registry; the libraries are the tooling that installs, runs, adapts, and ships what's in it — the same relationship npm has to the packages on it, or PyPI to Python packages.
None of these libraries require the Hub — you can point transformers at a local folder — but the two are designed together, and almost every workflow below touches the Hub at some point: pulling a model, streaming a dataset, or pushing a fine-tuned adapter back up.
The ecosystem map
Where each library sits
Storage
Load & run
Adapt
Measure & ship
These libraries are independent, not a monolith. You can use transformers without ever touching peft, or datasets without transformers at all. Pull in a library when a specific problem calls for it — do not install the whole stack because a tutorial did.
Go deeper: What Is HuggingFace? — the Hub, the libraries, model/dataset/Space repos, and how they relate to PyTorch itself.
2. Loading and Running a Model
The entry point for almost everything is transformers, currently at major version 5 — a rewrite that unified how models load, how tokenization backends work, and how pipelines handle mixed text/image/audio input.
pipeline() — the fastest path to a working model
from transformers import pipeline
classifier = pipeline("sentiment-analysis")
classifier("HuggingFace crash courses are surprisingly dense.")
# [{'label': 'POSITIVE', 'score': 0.9998}]One function call: downloads the default model and tokenizer for the task, builds the right preprocessing/postprocessing, and gives you a callable. It covers dozens of tasks — text, vision, audio, and now unified any-to-any multimodal chat.
The Auto* classes — when you need control
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3.8-27B")
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.8-27B", dtype="bfloat16", device_map="auto")Auto* classes read the repo's config and load the right architecture class automatically — you never need to know it's actually a Qwen3ForCausalLM under the hood. device_map="auto" hands GPU placement to accelerate without you writing any placement code.
| Choose | When |
|---|---|
pipeline() | Prototyping, a known task, you don't need custom pre/post-processing |
Auto* classes | You need the raw model, custom generation logic, or to feed it into peft/trl |
dtype, not torch_dtype. Transformers v5 renamed the load-time precision argument; the old name still works but is deprecated. Pin your weight format deliberately — bfloat16 for training and most inference, float16 only where bfloat16 isn't supported, full precision rarely needed for inference.
Go deeper: Transformers & Pipelines.
3. Tokenizers
Before a model sees text, a tokenizer turns it into integers. Get this step wrong and everything downstream is silently corrupted — the model still runs, it just runs on garbage.
| Algorithm | Used by | Idea |
|---|---|---|
| BPE | GPT-family, Llama | Merge the most frequent adjacent byte/character pairs repeatedly |
| WordPiece | BERT-family | Like BPE, but merges chosen to maximize training-data likelihood |
| Unigram | T5, most SentencePiece models | Start with a huge vocabulary, prune down to the most useful pieces |
tokenizer("Tokenization is not optional.")
# {'input_ids': [101, 19204, 3989, ...], 'attention_mask': [1, 1, ...]}The tokenizer and the model are a matched pair. They were trained together against one vocabulary; using bert-base-uncased's tokenizer with a Llama model produces input IDs the model was never trained on. This looks exactly like a working system that returns nonsense — always load both from the same repo.
Fast tokenizers (Rust-backed, from the tokenizers library) are the default — orders of magnitude faster than the old pure-Python ones, and the only ones that support offset mapping back to the original string, which you need for span-based tasks like NER.
Go deeper: Tokenizers — BPE vs WordPiece vs Unigram, special tokens, chat templates, training your own.
4. Datasets
The datasets library loads, transforms, and streams data at any scale — from a 50-row CSV to a multi-terabyte web-scale corpus — through one API.
from datasets import load_dataset
ds = load_dataset("HuggingFaceFW/finewiki", split="train", streaming=True)| Mode | What happens | Use when |
|---|---|---|
| Regular | Downloads and caches as typed Arrow tables on disk | The dataset fits comfortably on disk |
| Streaming | Reads examples as your code consumes them, no download | The dataset is huge or you want to start training now |
Streaming is fast now, not just memory-cheap. Recent releases made HuggingFace's streaming path up to 100× more efficient than it used to be, reaching throughput on par with local SSDs when training across many workers — the old advice that "streaming is for when you have no choice" is out of date.
.map() is the workhorse for preprocessing — tokenizing, filtering, reformatting — and runs multiprocessed and cached, so re-running a script doesn't redo work that already happened.
Go deeper: Datasets — loading, streaming, .map(), splits, and publishing your own.
5. Finding and Trusting a Model on the Hub
Every model repo carries a model card (README.md with YAML frontmatter): task, license, languages, intended use, and — increasingly — evaluation numbers.
| Signal | What it tells you |
|---|---|
| License | Whether you can actually use it — Apache-2.0 and MIT are unrestricted; many "open" model licenses (Llama, Gemma) carry use restrictions |
| Downloads / likes | A rough popularity signal, not a quality one |
safetensors present | The weights use a format that cannot execute code on load — prefer it over legacy .bin pickle files |
| Gated | You must accept terms before downloading — common for the largest or most capable releases |
| Revision / commit | Every repo is git — pin a commit hash in production so an upstream update can't silently change your model underneath you |
model = AutoModelForCausalLM.from_pretrained(
"meta-llama/Llama-4-Scout",
revision="a1b2c3d", # pin it — "main" moves
)"main" is not a version number. A repo owner can push new weights to the same branch at any time. Anything running in production should pin a commit hash or a tagged revision, the same discipline you'd apply to a container image tag.
Downloads and uploads run through Xet — HuggingFace's chunk-based storage layer that replaced Git LFS, deduplicating at the byte-chunk level so a small weight change re-transfers only the changed chunks, not the whole multi-gigabyte file.
Go deeper: The Hub, Model Cards & Trust.
6. Fine-Tuning: Full, LoRA, or QLoRA
Full fine-tuning updates every weight. For anything past a few billion parameters that means multiple GPUs and a full copy of the optimizer state per parameter — expensive, and usually unnecessary.
PEFT (Parameter-Efficient Fine-Tuning) freezes the base model and trains a small number of extra weights instead.
Picking a fine-tuning strategy
Full fine-tuning
Updates every parameter. Best final quality ceiling, by far the most expensive in GPU memory and time. Reserve it for smaller models or when LoRA has measurably fallen short.
LoRA (Low-Rank Adaptation)
RecommendedFreezes the base weights, injects small trainable low-rank matrices into the attention and MLP layers. Trains a fraction of a percent of the parameters, matches full fine-tuning quality on most tasks, and produces a tiny adapter file you can swap in and out.
QLoRA
LoRA on top of a 4-bit quantized frozen base model. Cuts memory further — a 27B model that needs ~54GB in bfloat16 fits on a single consumer GPU. The quality cost is small; the memory win is not.
from peft import LoraConfig, get_peft_model
config = LoraConfig(r=16, lora_alpha=32, target_modules=["q_proj", "v_proj"], lora_dropout=0.05)
model = get_peft_model(model, config)
model.print_trainable_parameters()
# trainable params: 8,388,608 || all params: 27,xxx,xxx,xxx || trainable%: 0.03%The adapter is the product, not the merged model. Ship the small LoRA weights (a few tens of MB) alongside the public base model, rather than re-uploading a full multi-gigabyte merged checkpoint for every fine-tune. Merge only when you need a single self-contained artifact.
Go deeper: Fine-Tuning with PEFT — LoRA math, QLoRA, the Trainer API, and picking hyperparameters.
7. Making Models Smaller and Faster
Quantization stores weights (and sometimes activations) at lower numerical precision, trading a little accuracy for large memory and speed wins.
| Format | Precision | Typical use |
|---|---|---|
| bfloat16 / float16 | 16-bit | The default for training and most GPU inference |
| bitsandbytes 4-bit (NF4) | 4-bit | Load-time quantization for training (QLoRA) or inference, no separate conversion step |
| GGUF | Mixed, typically 4–8 bit | The format for llama.cpp-family runtimes — CPU and edge inference |
| AWQ / GPTQ | 3–4 bit | Calibrated post-training quantization, strong accuracy retention at very low bit-widths |
| FP8 | 8-bit float | Increasingly used at both training and inference time on recent GPUs, less accuracy loss than integer quantization at the same size |
from transformers import BitsAndBytesConfig
bnb_config = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype="bfloat16")
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.8-27B", quantization_config=bnb_config, device_map="auto")Quantize for the runtime you'll actually deploy on. bitsandbytes is convenient inside transformers but is a training/experimentation tool; production serving usually wants a format the serving engine was built for — GGUF for llama.cpp, AWQ/FP8 for vLLM and TGI. Converting once for the right target beats re-quantizing per environment.
Go deeper: Quantization & Optimization — bitsandbytes, GGUF, AWQ/GPTQ, and optimum for ONNX/hardware-specific export.
8. Evaluation
evaluate is the metrics library — accuracy, F1, BLEU, ROUGE, perplexity, and hundreds of task-specific and community metrics behind one interface.
import evaluate
metric = evaluate.load("f1")
metric.compute(predictions=preds, references=labels, average="macro")A leaderboard number is not your number. A model's published benchmark score describes its performance on that benchmark's distribution. The only score that tells you whether a fine-tune actually helped your task is one computed on your held-out data, with your metric.
For generative tasks — summarization, chat, agents — exact-match metrics stop being meaningful, and evaluation increasingly means either a task-specific benchmark harness or an LLM-as-judge pass, both of which need a curated test set you control just as much as a training set.
Go deeper: Evaluation — metrics, benchmark harnesses, LLM-as-judge, and building a test set that won't lie to you.
9. Alignment: Beyond Next-Token Prediction
TRL (Transformer Reinforcement Learning), now at major version 1, provides trainers for every stage past plain supervised fine-tuning:
| Trainer | Teaches the model to | Needs |
|---|---|---|
SFTTrainer | Follow a format/instruction style | Instruction–response pairs |
RewardTrainer | Score which of two responses is better | Preference pairs (chosen vs rejected) |
DPOTrainer | Prefer the chosen response directly | Preference pairs — no separate reward model needed |
GRPOTrainer | Improve at a verifiable task via RL | A reward function, e.g. "did the code pass its tests" |
from trl import DPOConfig, DPOTrainer
trainer = DPOTrainer(model=model, args=DPOConfig(output_dir="out", report_to="trackio"), train_dataset=pref_ds)
trainer.train()DPO replaced classic RLHF for most teams. The original recipe — train a separate reward model, then run PPO against it — is expensive and unstable to tune. DPO optimizes the same preference objective directly against the policy model, with one trainer and no reward model to babysit. Reach for reward-model + RL only when you need a verifiable, programmatic signal DPO's static preference pairs can't express — which is exactly what GRPO is for.
TRL integrates with peft directly, so DPO/GRPO on top of a LoRA adapter is the common path rather than the exception — full-parameter alignment training is rarely necessary.
Go deeper: Alignment with TRL — SFT, reward modeling, DPO, GRPO, and when each one earns its cost.
10. Training Across Multiple GPUs
accelerate is the layer that takes ordinary PyTorch training code and runs it across however many devices you point it at — without you rewriting the loop for each hardware target.
from accelerate import Accelerator
accelerator = Accelerator()
model, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)| Strategy | Idea | Use when |
|---|---|---|
| DDP | Full model copy per GPU, gradients synced | Model fits on one GPU, you want more throughput |
| FSDP | Model, gradients, and optimizer state sharded across GPUs | The model doesn't fit on one GPU |
| DeepSpeed ZeRO | Same idea as FSDP, different implementation, deeper optimization knobs | Very large models, or you need its specific features (e.g. offload) |
accelerate launch reads a config file (from accelerate config) and handles process spawning, device placement, and mixed precision — the same training script runs unmodified on one GPU, eight GPUs, or a multi-node cluster.
Don't reach for distributed training first. A single modern GPU with LoRA/QLoRA and gradient checkpointing fine-tunes surprisingly large models. Move to FSDP or DeepSpeed when you've actually hit a memory or throughput wall, not by default — and consider hf jobs, which rents the GPU for the duration of the job rather than requiring you to own one.
Go deeper: Distributed Training — DDP vs FSDP vs DeepSpeed, mixed precision, and running jobs on rented hardware with hf jobs.
11. Shipping It
A trained model is only useful once something can call it. HuggingFace gives you three shapes of "running," in increasing order of effort:
From weights to a callable endpoint
demo.launch(mcp_server=True)That single argument turns any Gradio app into an MCP server — every function becomes a callable tool, with descriptions generated from its docstring, reachable by Claude, Cursor, or any other MCP client. Thousands of Spaces on the Hub already run this way.
Inference Providers is the fastest path when you don't want to run anything yourself: it's OpenAI-client compatible, so base_url is often the only line that changes in existing code.
Pick the shape that matches the traffic. Inference Providers for prototyping and variable/bursty traffic you don't want to provision for. A Space for a demo, an internal tool, or an MCP-callable tool. A dedicated Inference Endpoint once you have steady, latency-sensitive traffic that justifies owning the deployment.
Go deeper: Deployment & Inference — Spaces, Gradio, Inference Providers, Inference Endpoints, and picking between them.
12. Beyond Text: Diffusion and Multimodal
diffusers is the library for image, video, and audio generation models — Stable Diffusion, FLUX, and the current wave of unified any-to-any models.
from diffusers import DiffusionPipeline
pipe = DiffusionPipeline.from_pretrained("black-forest-labs/FLUX.1-dev", dtype="bfloat16").to("cuda")
image = pipe("a fox reading documentation, watercolor").images[0]The same quantization ideas from section 7 apply here, component-by-component — you can quantize just the transformer backbone and leave the text encoder full precision, trading memory for quality exactly where it matters most for a given pipeline.
Go deeper: Diffusers & Multimodal — image/video/audio pipelines, ControlNet-style conditioning, and mixed-precision component quantization.
13. Production Essentials
| Area | The thing you must get right |
|---|---|
| Pin revisions | main moves. Pin a commit hash for anything running unattended. |
Prefer safetensors | Legacy pickle checkpoints (.bin) can execute arbitrary code on load — never load one from a source you don't trust. |
| Track experiments | Use trackio (or another tracker) from the first run, not after the tenth one you can no longer reproduce. |
| Respect licenses | "Open weight" is not "unrestricted." Read the license before you ship, not after legal asks. |
| Cache deliberately | The local Hub cache dedupes by content hash via Xet — clear it with hf cache tooling, don't just rm -rf and re-download everything. |
| Right-size the deployment shape | Match section 11's three shapes to your actual traffic pattern, and revisit the choice as traffic changes. |
| Gate destructive Hub actions | Deleting or overwriting a shared repo affects everyone using it — treat Hub write access with the same care as production database access. |
Go deeper: Production & Operations.
14. When It Breaks
Most "the model is broken" bugs are a mismatch, not a bad model. Wrong tokenizer, wrong dtype, wrong chat template, wrong revision. Check the pairing before you question the weights.
| Symptom | Usual cause |
|---|---|
| Output is fluent nonsense | Tokenizer/model mismatch — loaded from two different repos |
CUDA out of memory | No quantization, no gradient checkpointing, or batch size too large for available VRAM |
| Model behaves differently than yesterday | main moved — pin a revision |
| Fine-tune "forgets" the base model's general ability | Learning rate too high, or too many epochs on a narrow dataset — catastrophic forgetting |
| Chat model ignores the system prompt | Chat template mismatch — raw text sent instead of the model's expected template |
pip install pulls in a huge dependency tree unexpectedly | Installed the full extras (transformers[torch,sentencepiece,...]) instead of just what the task needs |
| Streaming dataset is slower than expected | Too few shards for the number of workers, or transformations that block on network I/O per-example |
| A Space works locally, fails on the Hub | Missing entry in requirements.txt, or hardware tier too small for the model's memory footprint |
Go deeper: Failure Modes & Debugging.
15. Vocabulary You Need
| Term | Meaning |
|---|---|
| Hub | HuggingFace's hosted platform for model, dataset, and Space repositories |
| Model card | A repo's README.md with structured metadata — license, task, languages, evaluation |
| Pipeline | The high-level transformers API: task in, prediction out |
Auto* class | A class that resolves to the right architecture automatically from a repo's config |
| Checkpoint | A saved set of model weights at a point in training |
safetensors | The safe, fast weight-serialization format — no arbitrary code execution on load |
| Xet | The chunk-based storage backend behind Hub uploads/downloads, replacing Git LFS |
| LoRA | Low-Rank Adaptation — fine-tuning by training small injected matrices instead of the full model |
| QLoRA | LoRA on top of a 4-bit quantized frozen base model |
| PEFT | Parameter-Efficient Fine-Tuning — the library and the general technique family |
| DPO | Direct Preference Optimization — aligns a model to preference pairs without a separate reward model |
| GRPO | Group Relative Policy Optimization — RL fine-tuning against a programmatic reward |
| Quantization | Storing weights/activations at lower numerical precision to save memory and increase speed |
accelerate | The library that runs one training script across any number of devices |
| FSDP / DeepSpeed ZeRO | Two implementations of sharding model, gradient, and optimizer state across GPUs |
| Inference Providers | HuggingFace's routed, multi-vendor inference API |
| Inference Endpoint | A dedicated, autoscaling deployment of one model |
| Space | A hosted app on the Hub — usually Gradio or Streamlit, can be an MCP server |
| MCP | Model Context Protocol — a standard way to expose tools (including a Space) to an agent |
hf jobs | Serverless compute rented by the job, run via the hf CLI, Python client, or HTTP API |
Go deeper: Glossary — every term and abbreviation, expanded.
16. The Rules
If you remember nothing else:
| # | Rule |
|---|---|
| 1 | Pull in a library when a problem calls for it. The stack is modular, not a bundle to install all at once. |
| 2 | Tokenizer and model are a matched pair. Always load both from the same repo. |
| 3 | Pin a revision for anything unattended. main moves; a commit hash doesn't. |
| 4 | Prefer safetensors. Legacy pickle checkpoints can execute code on load. |
| 5 | Try LoRA/QLoRA before full fine-tuning. Match full-FT quality at a fraction of the memory, most of the time. |
| 6 | Evaluate on your data, not the leaderboard's. A published benchmark score is not your score. |
| 7 | DPO before RLHF-with-a-reward-model. Reach for a reward model and RL only when you need a verifiable programmatic signal. |
| 8 | Don't distribute training by default. A single GPU with LoRA and gradient checkpointing goes further than expected. |
| 9 | Match the deployment shape to the traffic. Inference Providers, a Space, or a dedicated Endpoint — in that order of effort. |
| 10 | Track experiments from run one. Not after the run you can no longer reproduce. |
| 11 | Read the license before you ship. "Open weight" is not "unrestricted." |
| 12 | Check the pairing before you blame the model. Most weird output is a tokenizer, dtype, or template mismatch. |
17. Where To Go Next
You now have the whole picture. The full track goes deeper on each part:
Loading and running
| Page | Covers |
|---|---|
| What Is HuggingFace? | The Hub, the libraries, and how the ecosystem fits together |
| Transformers & Pipelines | pipeline(), Auto* classes, generation, chat templates |
| Tokenizers | BPE, WordPiece, Unigram, special tokens, training your own |
| Datasets | Loading, streaming, .map(), splits, publishing |
| The Hub, Model Cards & Trust | Licenses, revisions, safetensors, Xet storage |
Adapting a model
| Page | Covers |
|---|---|
| Fine-Tuning with PEFT | LoRA, QLoRA, the Trainer API, hyperparameters |
| Quantization & Optimization | bitsandbytes, GGUF, AWQ/GPTQ, optimum |
| Evaluation | Metrics, benchmark harnesses, LLM-as-judge |
| Alignment with TRL | SFT, reward modeling, DPO, GRPO |
| Distributed Training | DDP, FSDP, DeepSpeed, hf jobs |
Shipping it
| Page | Covers |
|---|---|
| Deployment & Inference | Spaces, Gradio, Inference Providers, Inference Endpoints |
| Diffusers & Multimodal | Image, video, and audio generation pipelines |
| Production & Operations | Revisions, caching, licensing, cost |
| Failure Modes & Debugging | Symptom to cause, across every stage |
| Designing a HuggingFace-Based System | A full worked design with real numbers |
| Glossary | Every term and abbreviation |
Then build something
Reading takes you only so far. When you're ready to build, Project Ideas lays out one end-to-end project that ties the whole stack together — fine-tuning, evaluation, tracking, and a deployed, agent-callable Space — as a starting brief rather than a finished build.