Vector database benchmarks compare recall and queries-per-second on static, pre-loaded datasets. That's not what breaks in production. What breaks is filtered search under concurrent writes, index rebuild time when you change embedding models, and metadata query patterns the benchmark never tested. Pick a vector store based on how your system actually behaves, not the leaderboard.

Filtered search is the real test, not raw ANN speed

Almost every production retrieval query isn't a pure similarity search — it's similarity search plus filters: tenant ID, document permissions, date range, document type. Approximate nearest-neighbor indexes (HNSW especially) don't naturally combine with filters; a naive implementation either filters after retrieval (and returns too few results when the filter is selective) or filters before search (and loses the speed benefit of the index). Ask vendors specifically how they handle pre-filtering versus post-filtering, and benchmark with your filter selectivity, not an unfiltered query.

python
# Post-filtering: fast but can starve results
results = index.search(query_vector, top_k=10)
filtered = [r for r in results if r.tenant_id == current_tenant]  # may return 0-2 usable results

# Pre-filtering aware index: slower per-query, correct result count
results = index.search(query_vector, top_k=10, filter={"tenant_id": current_tenant})

Write throughput and index staleness

Read-heavy benchmarks ignore what happens when documents update frequently. Some vector databases handle upserts efficiently with incremental index updates; others require periodic full rebuilds, which means either accepting staleness between rebuilds or paying for a blue-green index swap. If your corpus changes hourly (support tickets, chat logs, live inventory), write-path behavior matters more than query latency at rest.

Multi-tenancy: isolation vs. cost

Approach Isolation Cost at scale
Separate index per tenant Strong Expensive — index overhead multiplies
Shared index, metadata filter Weak (relies on filter correctness) Efficient
Namespace/partition-aware store Strong Moderate — purpose-built for this

If you're building a multi-tenant SaaS product, this decision alone can determine your infrastructure bill more than embedding dimensionality or model choice. Purpose-built namespace support (rather than bolting isolation on with metadata filters) avoids the failure mode where a bug in filter logic leaks one tenant's documents into another's search results.

Operational maturity matters more than feature checklists

A vector database with fewer exotic features but mature backup/restore, monitoring, and a clear consistency model will cause fewer 2am pages than one with every ANN algorithm variant and no operational tooling. Concretely, check:

  • Can you restore to a point in time, and how long does it take on your data size?
  • What's the consistency model between a write and that write being searchable — immediate, eventual, or configurable?
  • Does it expose per-query latency and recall metrics, or do you have to build that instrumentation yourself?
  • What's the migration path if you need to re-embed the entire corpus with a new model — can you build a new index alongside the old one and cut over, or does it require downtime?

Don't over-index on scale you don't have yet

Below a few million vectors, most managed vector databases and even pgvector on a well-tuned Postgres instance perform comparably for typical RAG workloads — the differentiator at that scale is operational simplicity and how well it integrates with your existing stack, not raw throughput. Save the specialized, horizontally-scaled vector store for when you've actually measured a bottleneck, not because a blog post said you'd need it eventually.

A worked example

Requirements: 20M vectors, dim 768, filter on tenant_id, p95 query < 30ms, ops team knows Postgres. You prototype pgvector vs a dedicated engine on the same embeddings. Measure recall vs ef/nprobe. Backup/restore drill. Cost at 5× growth.

If filters are selective, check whether the engine applies metadata before or after ANN (recall cliff).

Failure modes

Ignoring filters. No recall measurement (only latency). Single-node RAM that will not fit. Vendor lock-in of a proprietary index without export. Treating the vector DB as the system of record. Under-sharding. HNSW build time surprise.

Mixing cosine/dot/L2 across writers.

When this is the wrong tool

<100k vectors: numpy / FAISS local / even SQL. Exact KNN may be fine. If lexical search suffices, skip vectors. A vector DB will not replace Redis for sessions. Do not buy a cluster for a demo. When updates are the product (stock), a transactional store plus embeddings as derived data is the architecture — the ANN index is not the ledger.

A worked failure mode

A startup puts 80k embeddings in a dedicated vector database because the architecture diagram had one. They then need metadata filters (tenant, ACL, created_at) and discover the filter runs after ANN, returning 2 results that pass ACL while the true neighbors were dropped. They add overfetch, latency triples, and they still leak a cross-tenant neighbor on a misconfigured namespace. Postgres with pgvector would have been boring and correct at this scale. The failure is buying ANN operations before you have a filtering and tenancy story. Measure recall with the same filters production uses, not a naked top-k on a research dump.

A specialized vector database is the wrong tool under a few million vectors with strong SQL filters, or when the bottleneck is bad chunking. It is the wrong tool if you cannot operate backups and rebuilds. Do not pick a vendor for a demo of 3D plots. Stay in the operational database until recall-at-filter, SLA, and ops cost say you have outgrown it.