Vector Databases 101: The Core Principles Behind Storing and Retrieving Enterprise Knowledge

Once you've converted text into embedding vectors, a more practical question immediately arises: where should millions of vectors be stored, and how do you find the few closest to a user's query in milliseconds? This is precisely the core problem that vector databases are designed to solve.

What Exactly Is a Vector Database

Here's an analogy to understand it: a traditional relational database is like a row of labeled filing cabinets—you ask for the "open" folder by name, and it hands it to you precisely. A vector database is more like a librarian who understands semantics—even without an exact matching label, they can find the book that "means the closest thing" based on your intent.

According to the original author Suresh Pallapothu's series of articles, relational databases excel at exact matching, such as WHERE status = 'open'. They lack the ability to efficiently answer "which stored vectors are closest to this query vector," because that's a geometric problem. Purpose-built vector databases like Milvus, Qdrant, and Pinecone are designed from the ground up to store and index massive embeddings and efficiently perform nearest-neighbor queries.

ANN: Trading a Little Precision for a Lot of Speed

The most naive approach is to compare the query vector against every single vector in the database. But when documents reach the millions, this brute-force comparison becomes unacceptably slow. The vector database solution is to pre-build an index, and HNSW (Hierarchical Navigable Small World) is one of the most popular index structures.

HNSW connects vectors into a layered graph: the search starts from a coarse high-level layer, "jumps" progressively closer to the target layer by layer, and finally locks onto the nearest neighbors at the bottom layer. The original article explicitly states that this is an approximate nearest neighbor (ANN) strategy—sacrificing a bit of recall precision in exchange for an order-of-magnitude speed improvement. For most enterprise retrieval scenarios, this trade-off is very worthwhile.

Embedding vectors and metric spaces are the mathematical foundation of all vector retrieval. An embedding model maps a piece of text into a fixed-length real-valued vector (common dimensions include 384, 768, 1536, etc.), with the core assumption that semantically similar texts are also close in vector space. There are three main ways to measure "distance"—cosine similarity (comparing the angle between directions, insensitive to text length, most commonly used for semantic retrieval), Euclidean distance (L2, comparing absolute spatial distance), and inner product (IP, related to vector magnitude). In pgvector, <=>, <->, and <#> correspond to these three respectively; the choice of distance function must align with the embedding model's training objective, otherwise retrieval quality will drop significantly. Note that ANN indexes are themselves bound to a distance metric: HNSW builds its neighbor graph according to a particular metric at construction time, and switching to another metric typically requires rebuilding the index.

pgvector: No Need to Introduce a New System

Not every team is willing to operate yet another database just for vector retrieval. If your team is already running PostgreSQL, the pgvector extension can directly add vector column types and similarity search capabilities to it. Embeddings and regular relational data live side by side, so a single query can simultaneously filter by customer and sort by semantics.

The original article provides a complete pgvector example (the example uses 3-dimensional vectors just to save space; real embeddings usually have hundreds of dimensions, and vector(3) should be replaced with the model's actual output dimension, such as vector(1536)):

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE docs (
  id        serial PRIMARY KEY,
  body      text,
  embedding vector(3)
);

CREATE INDEX ON docs USING hnsw (embedding vector_cosine_ops);

SELECT body
FROM docs
ORDER BY embedding <=> '[0.80, -0.10, 0.49]'
LIMIT 2;

Here <=> represents cosine distance, where smaller values indicate closer semantics. With the HNSW index added, this query can still remain fast as data scales up.

HNSW and IVFFlat are the two mainstream index types supported by pgvector, and their trade-offs differ. HNSW is based on a multi-layer skip-list-style graph structure; queries quickly approach from the sparse upper layers and then converge precisely in the dense bottom layer. It offers high recall and low latency, but slow construction and high memory usage. IVFFlat first uses k-means to cluster vectors into several clusters (lists), and at query time only scans the few clusters (probes) closest to the query vector—low memory overhead and fast construction, but recall is usually lower than HNSW, and the index should be built after data has reached a certain volume for reasonable clustering. For selection, consider: if the data is small and high precision is required, choose HNSW; if the data is extremely large, memory is tight, and slightly lower recall is tolerable, consider IVFFlat. Either way, index parameters (such as HNSW's m and ef_construction) directly affect the balance between speed and recall.

What a Complete Retrieval Looks Like

Putting the pipeline together: the user's question is first converted into a vector by the embedding model, the vector database finds the closest stored vectors, and then the corresponding passages are returned sorted by closeness. The original article summarizes this as three steps: "embed the question → find the nearest vectors → get back matching passages." This is exactly the foundation for subsequent RAG (Retrieval-Augmented Generation)—the original article previews that Day 13 will discuss how to use RAG to converse with enterprise private documents.

Three Paths for Selection

The original article summarizes storage placement into three routes:

• Purpose-built vector databases: Milvus, Qdrant, Pinecone—suitable for large-scale, pure vector retrieval scenarios;
• PostgreSQL + pgvector: suitable for teams that already have a PG stack and want vectors to coexist with business data;
• Local libraries: suitable for small-scale experimentation and prototype validation.

The key to selection isn't which is "strongest," but the balance between your data scale, existing tech stack, and operational costs. For most enterprise knowledge base projects starting from scratch, validating first with pgvector and then deciding whether to migrate based on scale is a pragmatic path.

The engineering advantages of purpose-built vector databases often only become apparent once scale picks up. Beyond providing indexes like HNSW, systems such as Milvus, Qdrant, and Pinecone also include production-oriented features: distributed sharding and horizontal scaling, quantization compression (such as product quantization PQ, scalar quantization SQ) to reduce memory usage, joint execution of metadata filtering and vector retrieval, multi-tenant isolation, and batch writes plus consistency control. When vector counts reach tens of millions or even billions, or when high-concurrency, low-latency online service is required, these capabilities are difficult for a single-node pgvector to match. Conversely, if the data scale is under a million and query QPS is not high, the simplicity of pgvector living in the same database as business data is often more valuable than extreme performance.

Key Takeaways

The essence of a vector database is upgrading "exact matching" to "semantic nearest-neighbor retrieval." It solves the scale problem through ANN indexes (such as HNSW), lowers the barrier to adoption through solutions like pgvector, and ultimately serves semantic search and RAG applications for enterprise knowledge. Understanding these three layers—storage, approximate retrieval, and integration approach—captures all the key aspects of vector databases in enterprise scenarios.

One-line Summary

Vector databases use ANN indexes to upgrade exact matching into semantic nearest-neighbor retrieval, and choose between purpose-built databases and pgvector based on scale.

Paragraph Summary

The article explains the core problem of vector databases: how to store massive embedding vectors and find the few closest to a query in milliseconds. It uses the analogy of a "librarian who understands semantics" to illustrate how vector databases differ from relational databases—the latter excel at exact matching, while the former excel at semantic nearest-neighbor retrieval. To address the retrieval bottleneck caused by millions of vectors, vector databases use approximate nearest neighbor (ANN) indexes such as HNSW, trading a small amount of recall precision for an order-of-magnitude speed improvement. For teams already using PostgreSQL, the pgvector extension can directly add vector columns and similarity search without introducing a new system, and the article provides a complete SQL example. The complete retrieval pipeline can be summarized as "embed the question → find the nearest vectors → return matching passages," which also forms the foundation of RAG. For selection, it outlines three paths: purpose-built vector databases, pgvector, and local libraries, with the key being the balance among data scale, tech stack, and operational costs.