GraphRAG using Ollama

GraphRAG using Ollama we build a Graph-based Retrieval Augmented Generation (GraphRAG) system using a local LLM (Ollama) and Python, without any cloud APIs.

We will:
• Extract relationships using an LLM
• Build a knowledge graph using NetworkX
• Retrieve facts from the graph
• Answer questions strictly from graph context

This entire setup runs offline.

Why GraphRAG (Instead of Normal RAG)?

Traditional RAG retrieves text chunks.
GraphRAG retrieves structured facts like:

Google → developed → Flutter
Flutter → integrates_with → Firebase

This reduces hallucinations and improves factual accuracy.

Tech Stack Used

Python
Ollama (Local LLM)
NetworkX (Graph engine)
Regex + JSON parsing

Step 1: Install Dependencies

pip install requests networkx

Make sure Ollama is running locally:

ollama run amplifyabhi

Step 2: Define Documents (Input Knowledge)

These are the facts we want to convert into a graph.

DOCUMENTS = [
    "Flutter is a UI toolkit developed by Google.",
    "Flutter integrates with Firebase.",
    "Firebase is owned by Google."
]

Step 3: Ollama Configuration

We call Ollama directly using its HTTP API.

OLLAMA_URL = "http://localhost:11434/api/generate"
MODEL = "amplifyabhi"

Step 4: Relationship Extraction Prompt

We force the LLM to return structured JSON only.

GRAPH_PROMPT = """
Extract relationships as JSON

Format: 
{
    "relationships": [
        { "source": "", "relation": "", "target": ""}
    ]
}
Text:
"""

This prompt is critical.

Bad prompts = broken JSON.

Step 5: Call Ollama Safely

def ollama(prompt: str) -> str:
    response = requests.post(
        OLLAMA_URL,
        json={
            "model": MODEL,
            "prompt": prompt,
            "stream": False
        },
        timeout=60
    )
    return response.json().get("response", "")

We use .get(“response”, “”) to avoid crashes if Ollama returns unexpected output.

Step 6: Extract Relationships (JSON-Safe)

LLMs sometimes return extra text.
So we extract only the JSON block.

def extract_relationships(text: str):
    raw = ollama(GRAPH_PROMPT + text)

    start = raw.find("{")
    end = raw.rfind("}") + 1

    if start == -1 or end == -1:
        return []
    
    json_text = raw[start:end].strip()
    
    try:
        data = json.loads(json_text)
    except json.JSONDecodeError:
        return []
    
    relationships = data.get("relationships", [])
    return [r for r in relationships if r.get("source") and r.get("relation") and r.get("target")]

This step prevents:

JSONDecodeError
Partial JSON
Hallucinated keys

Step 7: Build Knowledge Graph

We store extracted facts in a directed graph.

def build_graph(docs):
    graph = nx.DiGraph()
    for doc in docs:
        relations = extract_relationships(doc)
        for r in relations:
            graph.add_edge(
                r["source"],
                r["target"],
                relation=r["relation"]
            )
    return graph

Step 8: Query Normalization

Used to match questions with graph nodes.

def normalize(text):
    return re.sub(r"[^a-z0-9 ]", "", text.lower())

Step 9: Retrieve Facts From Graph

This replaces vector search.

def retrieve_from_graph(query, graph):
    q = normalize(query)
    facts = []
    for u, v, d in graph.edges(data=True):
        if normalize(u) in q or normalize(v) in q:
            facts.append(f"{u} {d['relation']} {v}")
    return facts