Vector Databases & Indexes
How approximate nearest neighbour search works — HNSW, IVF, quantization — plus metadata filtering, and how to choose a database
Vector Databases & Indexes
TL;DR
A vector database stores your embeddings and finds the closest ones to a query fast. The trick that makes it fast is approximate search. Rather than compare your query against every stored vector, an index narrows the search to a promising region. You give up a tiny amount of accuracy and gain an enormous amount of speed. This page explains how those indexes work, what you give up, and how to choose.
| Property | Value |
|---|---|
| Level | Intermediate → Advanced |
| Reading time | ~22 minutes |
| Prerequisites | Embeddings Explained |
| You will understand | ANN indexes, the recall/latency trade-off, filtering, and database selection |
The Problem
You have 10 million chunks, each a 1536-dimension vector. A query arrives. Finding the closest vectors by brute force means 10 million cosine similarity calculations, each over 1536 numbers — roughly 15 billion multiplications, per query.
That is too slow. Vector indexes exist to avoid it.
The core trade
Exact search compares everything and is guaranteed correct. Approximate Nearest Neighbour (ANN) search checks a small fraction and is usually correct. In practice ANN returns 95–99% of the true nearest neighbours while being 100–1000× faster. For RAG that trade is almost always worth taking — a reranker downstream will fix the ordering anyway.
Vocabulary
| Term | Expansion | Meaning |
|---|---|---|
| ANN | Approximate Nearest Neighbour | Fast search that may miss a few true matches |
| kNN | k Nearest Neighbours | The k closest vectors to a query |
| Recall@k | — | Of the true top-k, what fraction did the index actually return? |
| Flat index | — | No index — brute-force compare everything. Exact and slow |
| HNSW | Hierarchical Navigable Small World | A graph-based index. Fast and accurate; the common default |
| IVF | Inverted File Index | Clusters vectors, searches only the nearest clusters |
| PQ | Product Quantization | Compresses vectors to save memory, losing some precision |
| Sharding | — | Splitting an index across machines |
| Filtering | — | Restricting search by metadata (date, type, permissions) |
How the Indexes Work
Flat — No Index at All
Compare the query against every vector. Exact, simple, and perfectly fine below roughly 10,000 vectors, where the whole thing takes a few milliseconds.
| Recall | Speed | Memory | Use when |
|---|---|---|---|
| 100% | Slow, scales linearly | Lowest | Small collections, or ground truth for evaluating an ANN index |
Do not build a sophisticated index for 5,000 chunks. Flat search is exact, has no tuning parameters, and is fast enough. Complexity should follow scale, not precede it.
HNSW — Hierarchical Navigable Small World
The most widely used ANN index. The mental model is a road network with layers.
Imagine finding a specific house in a country. You do not check every house. You take the motorway to the right region, then A-roads to the right town, then local streets to the right house. HNSW builds exactly that: a top layer of long-range links for coarse jumps, and progressively denser lower layers for fine navigation.
How an HNSW search descends
| Parameter | Controls | Raising it means |
|---|---|---|
M | Links per node | Better recall, more memory, slower build |
ef_construction | Effort at build time | Better index quality, slower indexing (build-time only) |
ef_search | Effort at query time | Better recall, slower queries — tune this one live |
ef_search is your runtime quality dial and the one worth knowing. It costs nothing to change — no rebuild — so it is the right knob when you need to trade a little latency for a little recall in production.
| Recall | Speed | Memory | Use when |
|---|---|---|---|
| 95–99% | Very fast | High — the graph itself is large | Default choice up to tens of millions of vectors |
IVF — Inverted File Index
Cluster all vectors into groups during indexing. At query time, identify which few clusters the query falls near and search only those.
The trade is easy to see. Search 10 of 1,000 clusters and you look at 1% of the data, so you go about 100× faster. But if the true nearest neighbour happens to sit in cluster 11, you miss it.
| Parameter | Controls |
|---|---|
nlist | How many clusters to create |
nprobe | How many clusters to search — the runtime recall/speed dial |
| Recall | Speed | Memory | Use when |
|---|---|---|---|
| 90–98% | Fast | Lower than HNSW | Very large collections where HNSW memory is prohibitive |
Product Quantization — Compression
PQ shrinks vectors by splitting them into segments and replacing each segment with a code from a learned codebook. A 1536-dimension float vector (6 KB) can compress to under 100 bytes — a 60× reduction.
The cost is precision: you are storing an approximation of an approximation. Standard practice is to use PQ for a fast first pass, then re-score the survivors against the full-precision vectors.
| Recall | Speed | Memory | Use when |
|---|---|---|---|
| 80–95% alone | Very fast | Dramatically lower | Billions of vectors, or memory-constrained deployment |
Index Comparison
| Index | Recall | Query speed | Memory | Build time | Best for |
|---|---|---|---|---|---|
| Flat | 100% | Slow | Low | None | < 10K vectors |
| HNSW | 95–99% | Very fast | High | Medium | Most systems |
| IVF | 90–98% | Fast | Medium | Medium | Millions of vectors |
| IVF + PQ | 80–95% | Very fast | Very low | Slow | Billions of vectors |
Recall is not accuracy. Recall@k measures how well your index approximates brute-force search. An index with 99% recall over badly chunked, badly embedded data still returns bad results very efficiently. Index quality and retrieval quality are separate problems, and tuning the index will not fix the other one.
Metadata Filtering
Real queries are rarely "find similar text" alone. They are "find similar text that this user may read, from 2025, in the policy collection."
Why Filtering Is Harder Than It Sounds
Three ways to combine filtering and vector search
Post-filtering — search, then filter
Retrieve the top 100 by similarity, then discard everything failing the filter. Simple. Fails badly when the filter is selective: if only 1% of documents match, your 100 candidates may yield two results, or zero. You asked for 10 and got 2, with no error.
Pre-filtering — filter, then search
Find everything matching the filter, then vector-search only that subset. Always returns enough results. Expensive when the filter is broad — you may be brute-forcing over millions of vectors, losing the benefit of the index entirely.
Filtered search — integrated
RecommendedThe index itself is filter-aware, skipping non-matching nodes during graph traversal. Gets both correctness and speed. Support and quality vary by database — this is one of the most important things to evaluate when choosing one.
Permission filtering must be pre- or integrated filtering, never post-filtering. With post-filtering, a highly relevant confidential document consumes a candidate slot and is then dropped — degrading results for legitimate queries — and any bug in the discard step leaks it. Security filters belong inside the search, not after it.
Choosing a Database
| Database | Model | Strengths | Consider when |
|---|---|---|---|
| ChromaDB | Embedded / local | Trivial setup, zero infrastructure | Learning, prototypes, small local apps |
| FAISS | Library, not a server | Extremely fast, every index type | You want raw performance and will build the service layer yourself |
| Qdrant | Self-host or cloud | Excellent filtered search, strong performance | Filtering matters and you want control |
| Weaviate | Self-host or cloud | Built-in hybrid search, GraphQL API | You want hybrid search without assembling it |
| Pinecone | Managed only | Nothing to operate, scales transparently | You want to avoid running infrastructure |
| Milvus | Self-host or cloud | Built for billion-scale, many index types | Very large corpora |
| pgvector | PostgreSQL extension | Vectors live beside your relational data; one system to operate | You already run Postgres — start here |
| Elasticsearch / OpenSearch | Search engine | Mature keyword search plus vectors | You already run it for keyword search |
The advice most teams need
If you already run PostgreSQL, try pgvector first. Below a few million vectors it is entirely adequate, and it eliminates an entire additional system from your architecture — no second database to back up, secure, monitor, and keep consistent. Teams routinely adopt a specialised vector database before they have a scale problem that justifies it.
Selection Criteria
| Criterion | Question |
|---|---|
| Scale | Thousands, millions, or billions of vectors? |
| Filtering | How selective are your filters, and is filtered search integrated? |
| Hybrid search | Is keyword search built in, or must you run a second system? |
| Updates | Do documents change constantly? Some indexes handle deletes poorly |
| Operations | Do you have the team to run it? |
| Cost | Managed services charge by vector count and dimensions — model this early |
| Consistency | Must a newly indexed document be searchable immediately? |
Operational Realities
| Concern | What to know |
|---|---|
| Deletes | Graph indexes often "soft delete" — marking nodes as removed without reclaiming space. Recall degrades over time and periodic reindexing is required |
| Updates | Usually delete plus insert. Frequent updates fragment the index |
| Backups | An index can be rebuilt from source documents plus embeddings. Keep the chunk text, not only the vectors, or a rebuild means paying for embeddings again |
| Memory | HNSW generally wants the graph in RAM. 10M × 1536 dimensions ≈ 60 GB of vectors before graph overhead — size this before committing |
| Warm-up | Cold indexes are slow on the first queries as data pages in |
| Versioning | Store the embedding model name and version alongside vectors, so an accidental model change is detectable |
Always store the chunk text next to its vector. A vector cannot be converted back into text. If you keep only vectors, then rebuilding, debugging, or switching models means re-processing every source document. You also lose the ability to read what was retrieved — the single most useful debugging step in RAG.
Concept Checks
Check yourself
Your filter matches 0.5% of documents and you use post-filtering. What breaks?
You silently return too few results. Vector search fetches the top 100 by similarity, then the filter discards ~99.5% of them, leaving perhaps zero or one. No error is raised — the system just answers "I don't know" for questions it should handle. Selective filters require pre-filtering or integrated filtered search.
Recall@10 measures 99% but users say results are irrelevant. Where is the problem?
Not in the index. Recall@10 of 99% means the index is faithfully reproducing what brute-force search would return — it is doing its job almost perfectly. The problem is upstream: chunking, embedding quality, or the query itself. Tuning ef_search here would be wasted effort.
You have 8,000 chunks and are choosing between HNSW and IVF+PQ. Which?
Neither — use a flat index. At 8,000 vectors brute force takes single-digit milliseconds, returns exact results, and has no parameters to tune or rebuild. Both ANN options add tuning surface and approximation error to solve a performance problem you do not have.
Why store chunk text alongside vectors when the vector is what you search?
Because embedding is one-way — you cannot recover text from a vector. Without the text you cannot read what was retrieved, which is the most valuable debugging step available. You also cannot send the chunk to the model. And after a model change you cannot rebuild the index without re-processing and re-paying for every source document.
Key Concepts Recap
| Concept | One-line summary |
|---|---|
| ANN | Approximate search: ~99% of the accuracy, ~1% of the work |
| Flat | Exact brute force; correct choice under ~10K vectors |
| HNSW | Layered graph; fast and accurate; the default |
| IVF | Cluster-based; searches only nearby clusters |
| PQ | Compresses vectors dramatically at some precision cost |
| ef_search / nprobe | Runtime recall-vs-latency dials |
| Recall ≠ accuracy | Index fidelity is separate from retrieval quality |
| Filtered search | Integrated beats post-filtering, especially for permissions |
| pgvector first | If you already run Postgres, you may not need a new database |
| Store the text | Vectors are one-way; keep chunks for debugging and rebuilds |
Next
Your index is built. Now use it well — dense, sparse, hybrid, and query transformation: Retrieval Strategies.
Embeddings Explained
What a vector actually is, how cosine similarity works, how to choose an embedding model, and the mistakes that silently destroy retrieval
Retrieval Strategies
Dense, sparse, and hybrid search; BM25 and reciprocal rank fusion; query rewriting, expansion, HyDE, decomposition, and routing