The index hiding under your vector store
Every vector store you've ever used — Qdrant, Weaviate, pgvector, FAISS — has an index underneath it. When you call a similarity search, something has to decide which vectors to even look at. That something is an Approximate Nearest Neighbor algorithm, and which one you're using determines your recall, your memory footprint, and how fast your queries run at scale.
Most developers treat the index as a configuration detail, something to set once and forget. That works fine at small scale. But push past a few million vectors and the index choice stops being a detail — it becomes a system constraint. A setup that runs beautifully at one million vectors can become unbuildable at one billion.
There are four index structures worth understanding in depth: HNSW, IVF-PQ, scalar quantization, and DiskANN. Each makes a different tradeoff between recall, memory, and query speed. By the end of this lesson you'll know which to reach for and why.
The graph that made ANN practical
HNSW — Hierarchical Navigable Small World — is the dominant ANN algorithm in production today, and it earned that position by solving a genuinely hard problem elegantly. The core insight is borrowed from small-world network theory: if you build a graph where each node connects to a few long-range neighbors and a few short-range neighbors, you can navigate from any starting point to any destination in a logarithmic number of hops.
HNSW makes this hierarchical. At the top layer, there are very few nodes with very long-range connections — these let you traverse large distances quickly. Each lower layer adds more nodes with shorter-range connections. A query starts at the top and greedily descends, refining its neighborhood at each level until it reaches the bottom layer, where the true approximate nearest neighbors live.
The result is the best recall-per-query-per-second ratio of any general-purpose ANN algorithm, with one more crucial property: you can add new vectors without rebuilding. This is why pgvector, Qdrant, Weaviate, and Milvus all use HNSW as their default index.
The parameters that matter: ef_construction controls how thoroughly the graph is built — higher values improve recall but slow indexing. M is the number of edges per node — higher values mean better recall and more memory (typically 8–64). ef_search is the query-time exploration factor — you can tune recall vs. latency after the fact by adjusting this alone, without rebuilding the index. The memory cost is real: HNSW stores all vectors in RAM, which becomes the binding constraint as you scale.
Trading recall for memory you don't have
The problem HNSW doesn't solve is memory. At a hundred million 768-dimensional float32 vectors, you need roughly 300 GB of RAM just for the raw vectors, before any index overhead. Most systems don't have that. IVF-PQ is the answer when HNSW's memory requirement is simply out of reach.
IVF stands for Inverted File. The idea is to first cluster all your vectors into K centroids using k-means. At query time, you only search the clusters closest to the query vector — instead of searching everything, you search a small fraction. The number of clusters to probe is a tunable parameter.
PQ — Product Quantization — is a compression layer stacked on top. Each vector is split into sub-vectors, and each sub-vector is quantized to the nearest entry in a small codebook. A 768-dimensional float32 vector that would normally take 3 KB can be compressed to 64 bytes or less. FAISS is the canonical implementation, developed at Meta and used at billion-vector scale in production.
The recall tradeoff: IVF-PQ achieves roughly 90–95% recall compared to HNSW's 98%+. That 5–10% gap means some genuinely relevant results won't be returned. Whether that's acceptable depends entirely on your use case. For a recommendation system serving millions of users, the memory savings are worth it. For a high-stakes document retrieval system where missing a result has real consequences, you need to think carefully about that gap.
Compress the vectors, not the search
Scalar quantization takes a simpler approach than IVF-PQ: don't change how you search, just compress what you're searching over. No clustering, no codebooks, no architectural changes — just make each vector smaller.
SQ8 compresses each float32 dimension — four bytes — down to a single int8 byte. That's a 4x reduction in storage, and the recall loss is surprisingly small: roughly one to two percent. The compression works because most embedding dimensions don't actually need 32-bit floating point precision. An int8 value representing a value in the range of that dimension is nearly as accurate for the distance computations that matter.
Binary quantization takes this further, compressing each dimension to a single bit. A 768-dimensional float32 vector drops from 3 KB to 96 bytes — 32x compression. The recall loss climbs to 5–15%, and binary search uses Hamming distance rather than cosine or dot product. Qdrant supports both SQ8 and binary quantization natively, and it's common practice to use scalar quantization as a first pass to reduce memory, then rerank with the original float32 vectors for the top candidates.
The reranking trick: Because scalar and binary quantization operate on compressed vectors, you can use them for fast approximate recall and then rerank the top-K results against the full-precision originals. This recovers most of the recall loss at a fraction of the memory cost of keeping everything in full float32. Many production deployments store quantized vectors in RAM for fast search and keep full-precision vectors on disk for reranking.
When a billion vectors won't fit in RAM
HNSW and IVF-PQ both assume most of your data lives in memory. DiskANN challenges that assumption directly. Developed by Microsoft Research, it's a graph-based ANN algorithm designed from the ground up to work with most of the index on SSD and only a small working set in RAM.
The key insight is that SSD random read latency — while much slower than RAM — is predictable and parallelizable. DiskANN carefully constructs the graph so that each hop in a query traversal requires only a small, bounded number of SSD reads, and those reads can be issued in parallel. The result is billion-scale ANN search with RAM requirements measured in gigabytes rather than terabytes.
For most teams, DiskANN is a horizon technology — relevant when you're operating at a scale most applications never reach. But it matters to know it exists, because it reframes what "too large to index" means. Collections that were previously considered impossible to search with ANN — full web crawls, large-scale media libraries, genome databases — become feasible. Azure AI Search offers DiskANN as a managed option, so you don't need to run the research code directly.
When to reach for it: DiskANN is the right call when your vector collection won't fit in RAM and you still need the recall characteristics of a graph-based index (as opposed to the recall tradeoffs of IVF-PQ). The operational complexity is higher, and the managed options narrow, but for billion-scale corpora it's often the only path that doesn't require a hardware budget measured in six figures.
Match the index to the scale
The choice of ANN index is really a choice about which constraint you're willing to hit first. HNSW is the right default — it gives you the best recall, supports live inserts, and is available everywhere. When its memory requirements become the binding constraint, IVF-PQ trades some recall for a dramatically smaller footprint. When even IVF-PQ's memory cost is too high, scalar quantization compresses in-place. And when you've genuinely outgrown RAM as a medium for your index, DiskANN points the way to what comes next.
One thing these structures don't handle on their own is filtering — searching only the vectors that satisfy a metadata predicate. A naive pre-filter (apply the filter, then do ANN on the remainder) breaks down badly on small filtered subsets, and a naive post-filter (do ANN globally, then discard mismatches) wastes work. Getting filtering right requires the index to be filter-aware during traversal, not before or after it.
In the next lesson, we'll look at exactly that problem — metadata and filtered retrieval — including how modern indexes like Qdrant and Weaviate implement filter-aware ANN traversal, and how self-querying retrievers let LLMs extract structured filters automatically from natural language queries.