Vector database or knowledge graph — which should you use for AI retrieval?
Use a vector database when relevance is about meaning — finding passages similar to a question across unstructured text. Use a knowledge graph when the answer depends on explicit relationships and facts — who connects to what, and how. They're complementary, not rival: vectors find the right neighborhood, a graph enforces the right facts, and pairing them is increasingly how teams cut hallucination.
Both power retrieval-augmented generation — giving a model the right context at answer time — but they retrieve on different principles. The choice follows the shape of your data and your tolerance for a confidently wrong answer.
1. Vector databases retrieve by meaning
A vector database stores text as embeddings and finds passages whose meaning is closest to the query, even with no shared keywords. Weaviate’s Bob van Luijt makes the case that this similarity search is the backbone of most RAG systems: it’s the fastest way to make a large pile of unstructured documents searchable by intent. Where it’s weak is precision on facts — “close in meaning” can surface a passage that’s relevant but not correct.
2. Knowledge graphs retrieve by relationship
A knowledge graph stores entities and the explicit links between them — facts, hierarchies, who-relates-to-what. On the show, Neo4j’s Sudhir Hasbe argues that hallucination is a data-architecture problem: when answers must respect real relationships and constraints, a graph gives the model grounded, structured facts instead of fuzzy nearest-neighbors. The cost is that you have to model and maintain that structure.
3. The pattern is “and,” not “or”
The strongest setups use both — often called GraphRAG. Vectors cast a wide net to find the relevant neighborhood; the graph then constrains and enriches it with the exact entities and relationships that make the answer factual. Vectors handle recall over messy text; the graph handles precision over facts.
How to choose
Mostly unstructured text, relevance by topic, speed to stand up → vector database. Answers that hinge on explicit relationships, compliance, or traceable facts → knowledge graph. Hallucination is biting and your domain has real structure → combine them.
Why it matters
Picking one because it’s the default — usually the vector database — leaves precision on the table for problems that are really about relationships. Let the data decide: meaning-similarity and fact-structure are different jobs, and the best retrieval stacks do both.
From the conversation
This explainer is drawn from these episodes — each carries its full transcript.
