Graph RAG: Transforming AI with Knowledge Graphs

Exploring how Graph RAG is reshaping AI applications in various industries.

Hi there!

This week on Eximius Echo, we delve into the transformative potential of Graph RAG in the world of AI. As enterprises grapple with the challenges of deploying generative AI at scale, solutions like Retrieval-Augmented Generation (RAG) and model fine-tuning have emerged to address issues of accuracy and hallucination. Graph RAG stands out by incorporating knowledge graphs, offering a more accurate, context-aware, and secure approach to AI outputs.

If you’ve not heard of us, Eximius is a pre-seed stage fund focusing on FinTech, SaaS, Consumer, Online Media & Gaming, and HealthTech. You can find out more here. This newsletter is an attempt to share ideas, insights, and context within the realms of our chosen sectors. Let’s dive in.

The adoption of generative AI in enterprises is still in its nascent stages. Despite the excitement, few LLM projects have made it to production, primarily due to issues with model output accuracy and hallucinations. To address these challenges, Retrieval-Augmented Generation (aka, RAG) and model fine-tuning have emerged as the key solutions, helping customise these off-the-shelf large models. These methods are not mutually exclusive and can be used together or separately, depending on the use case. However, RAG is expected to emerge as the preferred choice due to the cost-effectiveness in updating retrieval sources, granular optimisation potential, and the ease of auditing responses.

The State of RAG Systems Today

Current RAG systems predominantly use vector-based approaches. These systems store information (such as documents or entities) in a high-dimensional space within vector databases. The vectors represent the semantic content of the papers, allowing those with similar meanings or topics to be positioned closer together. When a user query is made, the system retrieves documents with vectors nearest to the query, providing thematically relevant information.

Despite their usefulness, vector-based RAGs have intrinsic limitations (incomplete capturing of semantic intent, opaque ranking process, etc.), becoming increasingly evident as queries become more complex, revealing the systems' inability to reason across retrieved content. This has led to exploring knowledge graphs as a promising enhancement for RAG systems.

What are Knowledge Graphs?

Knowledge graphs are collections of nodes and edges where each node represents entities (such as people, places, or events), and edges define their relationships. This structure provides a flexible and intuitive way to model complex relationships and dependencies within data, closely mimicking human cognitive processes.

Enhancing RAG with Knowledge Graphs: Graph RAG 

Graph RAG leverages knowledge graphs as its data store. Upon receiving a query, Graph RAG traverses the knowledge graph to identify relevant entities and their relationships, aligning closely with the user’s intent. This process allows for a deeper understanding of the context and connections within the information, facilitating more sophisticated reasoning and inference.

Why Graph RAG?

• Higher Accuracy and More Useful Answers: The primary and most tangible benefit of Graph RAG is its enhanced accuracy. Practical applications and academic studies, like those by Microsoft, have shown that Graph RAG vastly improves the retrieval portion of RAG, populating the context window with higher relevance content and resulting in better answers with clear evidence provenance. Similarly, LinkedIn observed that Graph RAG improved answers by 77.6% in Mean Reciprocal Rank (MRR) and by 0.32 in BLEU scores compared to the vector-only approach.

• Enhanced Security: Knowledge graphs allow for easy permissions management, where data access is governed by the database, ensuring users only see data they are authorised to view. This is a challenge for both LLMs and vector databases.

• Intuitive and Visually Insightful: Knowledge graphs help users gain new insights and assist in building and debugging GenAI applications. They provide a clear trace of answers to the data and the causal chain.

Current Adoption and Future Challenges 

Enterprises are increasingly adopting these advanced RAG systems for various applications. For instance, LinkedIn has implemented Graph RAG in its customer service platform, achieving a 30% reduction in median issue resolution time. Similarly, Deutsche Telekom utilises knowledge graphs to enhance the accuracy of its coding assistant co-pilot.

Businesses are also investigating knowledge graphs as a long-term moat. Since it acts as a scalable structured representation of a company's domain expertise, it can be used as a foundational building block for both RAG and even for fine-tuning a model using completed graphs in the future.

However, the widespread adoption of Graph RAG systems poses several obstacles. Scalability remains a significant concern, as constructing graphs and generating embeddings are computationally intensive processes. Additionally, robust entity disambiguation modules are essential for accurate information retrieval. These challenges represent some of the next frontiers for research and development in the field.

Despite these challenges, the potential of GraphRAG to revolutionise our interaction with unstructured data is undeniable, promising holistic, context-aware AI assistants and staggering possibilities for augmenting human intelligence.

---

If you're an entrepreneur in this space seeking support in the early stages, kindly fill up a short form here, and we would love to hear more about what you are building!