Vector database benchmarks compare recall and queries-per-second on static, pre-loaded datasets. That's not what breaks in production. What breaks is filtered search under concurrent writes, index rebuild time when you change embedding models, and metadata query patterns the benchmark never tested. Pick a vector store based on how your system actually behaves, not the leaderboard.
Filtered search is the real test, not raw ANN speed
Almost every production retrieval query isn't a pure similarity search — it's similarity search plus filters: tenant ID, document permissions, date range, document type. Approximate nearest-neighbor indexes (HNSW especially) don't naturally combine with filters; a naive implementation either filters after retrieval (and returns too few results when the filter is selective) or filters before search (and loses the speed benefit of the index). Ask vendors specifically how they handle pre-filtering versus post-filtering, and benchmark with your filter selectivity, not an unfiltered query.
# Post-filtering: fast but can starve results
results = index.search(query_vector, top_k=10)
filtered = [r for r in results if r.tenant_id == current_tenant] # may return 0-2 usable results
# Pre-filtering aware index: slower per-query, correct result count
results = index.search(query_vector, top_k=10, filter={"tenant_id": current_tenant})Write throughput and index staleness
Read-heavy benchmarks ignore what happens when documents update frequently. Some vector databases handle upserts efficiently with incremental index updates; others require periodic full rebuilds, which means either accepting staleness between rebuilds or paying for a blue-green index swap. If your corpus changes hourly (support tickets, chat logs, live inventory), write-path behavior matters more than query latency at rest.
Multi-tenancy: isolation vs. cost
| Approach | Isolation | Cost at scale |
|---|---|---|
| Separate index per tenant | Strong | Expensive — index overhead multiplies |
| Shared index, metadata filter | Weak (relies on filter correctness) | Efficient |
| Namespace/partition-aware store | Strong | Moderate — purpose-built for this |
If you're building a multi-tenant SaaS product, this decision alone can determine your infrastructure bill more than embedding dimensionality or model choice. Purpose-built namespace support (rather than bolting isolation on with metadata filters) avoids the failure mode where a bug in filter logic leaks one tenant's documents into another's search results.
Operational maturity matters more than feature checklists
A vector database with fewer exotic features but mature backup/restore, monitoring, and a clear consistency model will cause fewer 2am pages than one with every ANN algorithm variant and no operational tooling. Concretely, check:
- Can you restore to a point in time, and how long does it take on your data size?
- What's the consistency model between a write and that write being searchable — immediate, eventual, or configurable?
- Does it expose per-query latency and recall metrics, or do you have to build that instrumentation yourself?
- What's the migration path if you need to re-embed the entire corpus with a new model — can you build a new index alongside the old one and cut over, or does it require downtime?
Don't over-index on scale you don't have yet
Below a few million vectors, most managed vector databases and even pgvector on a well-tuned Postgres instance perform comparably for typical RAG workloads — the differentiator at that scale is operational simplicity and how well it integrates with your existing stack, not raw throughput. Save the specialized, horizontally-scaled vector store for when you've actually measured a bottleneck, not because a blog post said you'd need it eventually.