IVFFlat
An approximate nearest neighbor indexing method based on k-means clustering. It groups vectors into a number of clusters and scans only the nearest clusters at query time. It has low memory overhead and fast build times, but recall is generally lower than that of HNSW.
Timeline (last 90 days)
The official announcement of pg_vectorsearch did not give the dimensionality of the benchmark dataset, the throughput (QPS), or a direct IVFFlat comparison, and multi-client throughput testing is listed as a to-be-completed item.
pgvector provides two index types, IVFFlat and HNSW, respectively aimed at different recall and speed trade-off scenarios.
IVFFlat uses two parameters, lists (the number of cluster buckets) and probes (the number of buckets actually scanned during a query); lists is often taken as rows/1000 (when the number of rows does not exceed one million) or sqrt(rows), and probes is usually tuned starting from sqrt(lists).
When the data distribution is skewed or the number of clusters is set improperly, IVFFlat will face the dilemma of recall collapsing because too few buckets are scanned, or speed degrading because too many buckets are scanned.
According to benchmark data provided by the project, Pg_vectorsearch surpasses pgvector's IVFFlat and HNSW in query throughput.
pgvector supports two mainstream types of indexes, HNSW and IVFFlat; HNSW has high recall and low latency but is slow to build and uses a lot of memory; IVFFlat has low memory overhead and builds quickly but usually has lower recall.
At very large scale datasets, pgvector's performance based on IVFFlat or HNSW indexes is often constrained by single-machine memory and CPU resources, making horizontal scaling difficult.
All Facts (7)
The official announcement of pg_vectorsearch did not give the dimensionality of the benchmark dataset, the throughput (QPS), or a direct IVFFlat comparison, and multi-client throughput testing is listed as a to-be-completed item.
50%Unverifiedpgvector provides two index types, IVFFlat and HNSW, respectively aimed at different recall and speed trade-off scenarios.
50%UnverifiedIVFFlat uses two parameters, lists (the number of cluster buckets) and probes (the number of buckets actually scanned during a query); lists is often taken as rows/1000 (when the number of rows does not exceed one million) or sqrt(rows), and probes is usually tuned starting from sqrt(lists).
50%UnverifiedWhen the data distribution is skewed or the number of clusters is set improperly, IVFFlat will face the dilemma of recall collapsing because too few buckets are scanned, or speed degrading because too many buckets are scanned.
50%UnverifiedAccording to benchmark data provided by the project, Pg_vectorsearch surpasses pgvector's IVFFlat and HNSW in query throughput.
50%Unverifiedpgvector supports two mainstream types of indexes, HNSW and IVFFlat; HNSW has high recall and low latency but is slow to build and uses a lot of memory; IVFFlat has low memory overhead and builds quickly but usually has lower recall.
50%UnverifiedAt very large scale datasets, pgvector's performance based on IVFFlat or HNSW indexes is often constrained by single-machine memory and CPU resources, making horizontal scaling difficult.
50%