Graph retrieval augmented generation (Graph RAG) is emerging as a powerful technique for generative AI applications to use domain-specific knowledge and relevant information. Graph RAG is an alternative to vector search methods that use a vector database. Knowledge graphs are knowledge systems where graph databases such as Neo4j or Amazon Neptune can represent structured data. In a knowledge graph, the relationships between data points, called edges, are as meaningful as the connections between data points, called vertices or sometimes nodes. A knowledge graph makes it easy to traverse a network and process complex queries about connected data. Knowledge graphs are especially well suited for use cases involving chatbots, identity resolution, network analysis, recommendation engines, customer 360 and fraud detection.

A Graph RAG approach leverages the structured nature of graph databases to give greater depth and context of retrieved information about networks or complex relationships. When a graph database is paired with a large language model (LLM), a developer can automate significant parts of the graph creation process from unstructured data like text. An LLM can process text data and identify entities, understand their relationships and represent them in a graph structure.

There are many ways to create a Graph RAG application, for instance Microsoft’s GraphRAG, or pairing GPT4 with LlamaIndex. For this tutorial you’ll use Memgraph, an open source graph database solution to create a rag system by using Meta’s Llama-3 on watsonx. Memgraph uses Cypher, a declarative query language. It shares some similarities with SQL but focuses on nodes and relationships rather than tables and rows. You’ll have Llama 3 both create and populate your graph database from unstructured text and query information in the database.

While you can choose from several tools, this tutorial walks you through how to set up an IBM account to use a Jupyter Notebook.

Log in to watsonx.ai™ using your IBM Cloud® account.

Create a watsonx.ai project.

You get your project ID from within your project. Click the Manage tab. Then, copy the project ID from the Details section of the General page. You need this Project ID for this tutorial.

Next, associate your project with the watsonx.ai Runtime

a.  Create a watsonx.ai Runtime service instance (choose the Lite plan, which is a free instance).

b.  Generate an API Key in watsonx.ai Runtime. Save this API key for use in this tutorial.

c.  Go to your project and select the Manage tab

d.  In the left tab, select Services and Integrations

e.  Select IBM services

f.  Select Associate service and pick waxtsonx data.

g.  Associate the waxtsonx™ data service to the project that you created in watsonx.ai

Now, you’ll need to install Docker from https://www.docker.com/products/docker-desktop/

Once you’ve installed Docker, install Memgraph using their Docker container. On OSX or Linux, you can use this command in a terminal:

curl https://install.memgraph.com | sh

On a Windows computer use:

iwr https://windows.memgraph.com | iex

Follow the installation steps to get the Memgraph engine and Memgraph lab up and running.

On your computer, create a fresh virtualenv for this project:

virtualenv kg_rag –python=python3.12

In the Python environment for your notebook, install the following Python libraries:

./kg_rag/bin/pip install langchain langchain-openai langchain_experimental langchain-community==0.3.15 neo4j langchain_ibm jupyterlab json-repair getpass4

Now you’re ready to connect to Memgraph.

If you’ve configured Memgraph to use a username and password, set them here, otherwise you can use the defaults of having neither. It’s not good practice for a production database but for a local development environment that doesn’t store sensitive data, it’s not an issue.

import os
from langchain_community.chains.graph_qa.memgraph import MemgraphQAChain
from langchain_community.graphs import MemgraphGraph

url = os.environ.get(“MEMGRAPH_URI”, “bolt://localhost:7687”)
username = os.environ.get(“MEMGRAPH_USERNAME”, “”)
password = os.environ.get(“MEMGRAPH_PASSWORD”, “”)

#initialize memgraph connection
graph = MemgraphGraph(
    url=url, username=username, password=password, refresh_schema=True
)

Now create a sample string that describes a dataset of relationships that you can use to test the graph generating capabilities of your LLM system. You can use more complex data sources but this simple example helps us demonstrate the algorithm.

graph_text = “””
John’s title is Director of the Digital Marketing Group.
John works with Jane whose title is Chief Marketing Officer.
Jane works in the Executive Group.
Jane works with Sharon whose title is the Director of Client Outreach.
Sharon works in the Sales Group.
“””

Enter the watsonx API key that you created in the first step:

from getpass import getpass

watsonx_api_key = getpass()
os.environ[“WATSONX_APIKEY”] = watsonx_api_key

watsonx_project_id = getpass()
os.environ[“WATSONX_PROJECT_ID”] = watsonx_project_id

Now configure a WatsonxLLM instance to generate text. The temperature should be fairly low and the number of tokens high to encourage the model to generate as much detail as possible without hallucinating entities or relationships that aren’t present.

from langchain_ibm import WatsonxLLM
from ibm_watsonx_ai.metanames import GenTextParamsMetaNames

graph_gen_parameters = {   
    GenTextParamsMetaNames.DECODING_METHOD: “sam