Table of Contents
GraphRAG using Ollama
GraphRAG using Ollama we build a Graph-based Retrieval Augmented Generation (GraphRAG) system using a local LLM (Ollama) and Python, without any cloud APIs.
We will:
• Extract relationships using an LLM
• Build a knowledge graph using NetworkX
• Retrieve facts from the graph
• Answer questions strictly from graph context
This entire setup runs offline.
Why GraphRAG (Instead of Normal RAG)?
Traditional RAG retrieves text chunks.
GraphRAG retrieves structured facts like:
Google → developed → Flutter Flutter → integrates_with → Firebase
This reduces hallucinations and improves factual accuracy.
Tech Stack Used
- Python
- Ollama (Local LLM)
- NetworkX (Graph engine)
- Regex + JSON parsing
Step 1: Install Dependencies
pip install requests networkx
Make sure Ollama is running locally:
ollama run amplifyabhi
Step 2: Define Documents (Input Knowledge)
These are the facts we want to convert into a graph.
DOCUMENTS = [
"Flutter is a UI toolkit developed by Google.",
"Flutter integrates with Firebase.",
"Firebase is owned by Google."
]
Step 3: Ollama Configuration
We call Ollama directly using its HTTP API.
OLLAMA_URL = "http://localhost:11434/api/generate" MODEL = "amplifyabhi"
Step 4: Relationship Extraction Prompt
We force the LLM to return structured JSON only.
GRAPH_PROMPT = """
Extract relationships as JSON
Format:
{
"relationships": [
{ "source": "", "relation": "", "target": ""}
]
}
Text:
"""
This prompt is critical.
Bad prompts = broken JSON.
Step 5: Call Ollama Safely
def ollama(prompt: str) -> str:
response = requests.post(
OLLAMA_URL,
json={
"model": MODEL,
"prompt": prompt,
"stream": False
},
timeout=60
)
return response.json().get("response", "")
We use .get(“response”, “”) to avoid crashes if Ollama returns unexpected output.
Step 6: Extract Relationships (JSON-Safe)
LLMs sometimes return extra text.
So we extract only the JSON block.
def extract_relationships(text: str):
raw = ollama(GRAPH_PROMPT + text)
start = raw.find("{")
end = raw.rfind("}") + 1
if start == -1 or end == -1:
return []
json_text = raw[start:end].strip()
try:
data = json.loads(json_text)
except json.JSONDecodeError:
return []
relationships = data.get("relationships", [])
return [r for r in relationships if r.get("source") and r.get("relation") and r.get("target")]
This step prevents:
- JSONDecodeError
- Partial JSON
- Hallucinated keys
Step 7: Build Knowledge Graph
We store extracted facts in a directed graph.
def build_graph(docs):
graph = nx.DiGraph()
for doc in docs:
relations = extract_relationships(doc)
for r in relations:
graph.add_edge(
r["source"],
r["target"],
relation=r["relation"]
)
return graph
Step 8: Query Normalization
Used to match questions with graph nodes.
def normalize(text):
return re.sub(r"[^a-z0-9 ]", "", text.lower())
Step 9: Retrieve Facts From Graph
This replaces vector search.
def retrieve_from_graph(query, graph):
q = normalize(query)
facts = []
for u, v, d in graph.edges(data=True):
if normalize(u) in q or normalize(v) in q:
facts.append(f"{u} {d['relation']} {v}")
return facts
GraphRAG using Ollama Part 1
GraphRAG using Ollama explained in detail
Step 10: GraphRAG Answer Generation
LLM is strictly bounded by graph facts.
def graphrag_answer(query, graph):
facts = retrieve_from_graph(query, graph)
if not facts:
return "I don't know based on the graph."
context = "\n".join(facts)
prompt = f"""
You are a technical assistant.
Rules:
- Answer in 1–2 factual sentences
- Use ONLY the context
- No assumptions
Context:
{context}
Question:
{query}
Answer:
"""
return ollama(prompt)
Step 11: Run the System
if __name__ == "__main__":
graph = build_graph(DOCUMENTS)
print("Graph Edges:\n")
for u, v, d in graph.edges(data=True):
print(f"{u} - [{d['relation']}]-> {v}")
while True:
q = input("Question: ")
if q.lower() == "exit":
break
print(graphrag_answer(q, graph))
GraphRAG using Ollama Part 2
In this video, we go deep into GraphRAG (Graph-based Retrieval Augmented Generation)
and show how to STOP large language models from adding fake or extra information.
Key Takeaways
Implementing GraphRAG using Ollama these are the key takeaways
• GraphRAG reduces hallucinations
• Local LLMs can power serious AI systems
• JSON enforcement is mandatory
• Graphs > embeddings for factual queries
