Vector DB — the memory
of RAG and AI agents
Every RAG system and every agent with long-term memory relies on a vector database — the place where "knowledge" is stored as vectors and retrieved by meaning similarity. In this guide: how it really works, a comparison of Pinecone, Qdrant, Weaviate and pgvector, and how to choose the right one for your project.
What a vector database is and why you need one
A regular database searches for an exact match: "give me all records where city = Tel Aviv". A vector database searches for meaning similarity: "give me the documents closest in meaning to this question". This is exactly whatRAG and agent memory need — to find relevant knowledge even when it was not written in the same words.
The magic works through embeddings: a model turns each piece of text into a vector of numbers (a list of, say, 1536 values) that represents the meaning. Texts similar in meaning get vectors close together in space. The vector DB stores these vectors and can quickly retrieve the ones closest to a query.
How similarity search works — briefly
The process splits into two stages: indexing andquerying.
Searching for the exact nearest vector among a million vectors is too slow. HNSW (Hierarchical Navigable Small World) builds a layered graph that lets you find the nearest neighbors approximately at enormous speed, with 95%+ accuracy. Almost every modern vector DB uses it.
Comparison table — at a glance
| Store | Model | Best for |
|---|---|---|
| Pinecone | Managed only | Fast setup with no DevOps |
| Qdrant | Open source + cloud | Performance & self-host |
| Weaviate | Open source + cloud | Hybrid search & modules |
| pgvector | A Postgres extension | You already have Postgres |
| Chroma | Open source, embedded | Prototyping & learning |
The leading tools — who each suits
- Pinecone — a fully managed service. You do not manage a server, it just works and scales. The price is for the convenience. Excellent for a fast start and production without a DevOps team. See the Pinecone guide in full.
- Qdrant — written in Rust, a leader in performance and speed, with rich payload filtering. Open source for self-host or a managed cloud. The choice of many AI agents in 2026.
- Weaviate — strong at Hybrid Search (vector + keyword together) and built-in embedding modules. Open source + cloud.
- pgvector — a PostgreSQL extension. If you already have Postgres — you get vector search without adding a new system. Supabase is built on it too. Ideal for medium projects.
- Chroma — lightweight, runs embedded inside the application. Perfect for prototyping, learning and small local projects.
How to choose — a decision tree
Tips for retrieval quality
- Proper chunking — chunks not too large (they lose precision) and not too small (they lose context). Start at 300–800 tokens with a small overlap, and tune by the results.
- Metadata filtering — store metadata (source, date, category) and filter by it before the vector search. It cuts noise and improves relevance dramatically.
- Hybrid Search — combining vector search with keyword search (BM25) captures both exact terms and meaning. Weaviate and Qdrant support it built in.
- Reranking — after an initial retrieval, re-rank the results with a reranker model (Cohere, Jina) for higher precision at the top.
- The same embedding model — make sure to use the same model for indexing and querying. Mixing models = random results.
RAG quality is determined far more by chunking, the choice of embedding model and reranking — less by which vector DB you chose. Almost all the tools use the same HNSW and give similar results. Start simple (pgvector/Chroma), and upgrade only when scale demands it. See the full RAG guide.