Overview
Standard RAG systems retrieve the most semantically similar document chunks to a query, then pass them to an LLM for generation. This works well for factual lookups but struggles when the answer requires understanding relationships between entities — departments, products, systems, policies.
This project enhanced an internal knowledge assistant by combining document retrieval with graph database context.
Problem
Enterprise documents are dense with implicit relationships. A question like "What approval process does Team A follow for System B?" might require connecting three documents that individually mention approval processes, Team A's responsibilities, and System B's configuration. Cosine similarity alone won't surface that connection reliably.
Approach
The pipeline adds a graph context layer between retrieval and generation:
1. Standard retrieval — vector similarity search over indexed document chunks 2. Entity extraction — identify key entities mentioned in the query (teams, products, systems) 3. Graph traversal — query the graph database for relationships between extracted entities 4. Context merging — combine document chunks with graph-derived relational context 5. Generation — the LLM generates an answer grounded in both sources
The graph database holds curated relationships between internal entities: which teams own which systems, how products connect to workflows, approval chains, etc.
Challenge
More context isn't always better. When graph traversal returns too broad a neighborhood, the generation gets noisy and unfocused. Too narrow, and relevant relationships are missed.
The key tuning problem was: how many hops from each entity, and how to rank the resulting subgraph before passing it to the LLM.
We settled on shallow traversal (1–2 hops) with relevance scoring that prioritized edges directly mentioned in the original query, plus a hard token cap on injected graph context.
Results
Internal benchmark performance improved by approximately 15% compared to vector-only retrieval. The most significant gains were on multi-entity questions that required connecting information spread across different document sections.
The system runs as an internal knowledge assistant used by teams for querying product documentation, operational guidelines, and process knowledge.