Artificial Intelligence

GraphRAG using Ollama

GraphRAG using Ollama we build a Graph-based Retrieval Augmented Generation (GraphRAG) system using a local LLM (Ollama) and Python, without any cloud APIs.

We will:
• Extract relationships using an LLM
• Build a knowledge graph using NetworkX
• Retrieve facts from the graph
• Answer questions strictly from graph context

This entire setup runs offline.

Why GraphRAG (Instead of Normal RAG)?

Traditional RAG retrieves text chunks.
GraphRAG retrieves structured facts like:

Google → developed → Flutter
Flutter → integrates_with → Firebase

This reduces hallucinations and improves factual accuracy.

Tech Stack Used

• Python
• Ollama (Local LLM)
• NetworkX (Graph engine)
• Regex + JSON parsing

Step 1: Install Dependencies

pip install requests networkx

Make sure Ollama is running locally:

ollama run amplifyabhi

Step 2: Define Documents (Input Knowledge)

These are the facts we want to convert into a graph.

DOCUMENTS = [
    "Flutter is a UI toolkit developed by Google.",
    "Flutter integrates with Firebase.",
    "Firebase is owned by Google."
]

Step 3: Ollama Configuration

We call Ollama directly using its HTTP API.

OLLAMA_URL = "http://localhost:11434/api/generate"
MODEL = "amplifyabhi"

Step 4: Relationship Extraction Prompt

We force the LLM to return structured JSON only.

GRAPH_PROMPT = """
Extract relationships as JSON

Format: 
{
    "relationships": [
        { "source": "", "relation": "", "target": ""}
    ]
}
Text:
"""

This prompt is critical.

Bad prompts = broken JSON.

Step 5: Call Ollama Safely

def ollama(prompt: str) -> str:
    response = requests.post(
        OLLAMA_URL,
        json={
            "model": MODEL,
            "prompt": prompt,
            "stream": False
        },
        timeout=60
    )
    return response.json().get("response", "")

We use .get(“response”, “”) to avoid crashes if Ollama returns unexpected output.

Step 6: Extract Relationships (JSON-Safe)

LLMs sometimes return extra text.
So we extract only the JSON block.

def extract_relationships(text: str):
    raw = ollama(GRAPH_PROMPT + text)

    start = raw.find("{")
    end = raw.rfind("}") + 1

    if start == -1 or end == -1:
        return []
    
    json_text = raw[start:end].strip()
    
    try:
        data = json.loads(json_text)
    except json.JSONDecodeError:
        return []
    
    relationships = data.get("relationships", [])
    return [r for r in relationships if r.get("source") and r.get("relation") and r.get("target")]

This step prevents:

• JSONDecodeError
• Partial JSON
• Hallucinated keys

Step 7: Build Knowledge Graph

We store extracted facts in a directed graph.

def build_graph(docs):
    graph = nx.DiGraph()
    for doc in docs:
        relations = extract_relationships(doc)
        for r in relations:
            graph.add_edge(
                r["source"],
                r["target"],
                relation=r["relation"]
            )
    return graph

Step 8: Query Normalization

Used to match questions with graph nodes.

def normalize(text):
    return re.sub(r"[^a-z0-9 ]", "", text.lower())

Step 9: Retrieve Facts From Graph

This replaces vector search.

def retrieve_from_graph(query, graph):
    q = normalize(query)
    facts = []
    for u, v, d in graph.edges(data=True):
        if normalize(u) in q or normalize(v) in q:
            facts.append(f"{u} {d['relation']} {v}")
    return facts

GraphRAG using Ollama Part 1

GraphRAG using Ollama explained in detail

Step 10: GraphRAG Answer Generation

LLM is strictly bounded by graph facts.

def graphrag_answer(query, graph):
    facts = retrieve_from_graph(query, graph)

    if not facts:
        return "I don't know based on the graph."

    context = "\n".join(facts)

    prompt = f"""
        You are a technical assistant.

        Rules:
        - Answer in 1–2 factual sentences
        - Use ONLY the context
        - No assumptions

        Context:
        {context}

        Question:
        {query}

        Answer:
        """
    return ollama(prompt)

Step 11: Run the System

if __name__ == "__main__":
    graph = build_graph(DOCUMENTS)

    print("Graph Edges:\n")
    for u, v, d in graph.edges(data=True):
        print(f"{u} - [{d['relation']}]-> {v}")

    while True:
        q = input("Question: ")
        if q.lower() == "exit":
            break
        print(graphrag_answer(q, graph))

GraphRAG using Ollama Part 2

In this video, we go deep into GraphRAG (Graph-based Retrieval Augmented Generation)
and show how to STOP large language models from adding fake or extra information.

Key Takeaways

Implementing GraphRAG using Ollama these are the key takeaways
• GraphRAG reduces hallucinations
• Local LLMs can power serious AI systems
• JSON enforcement is mandatory
• Graphs > embeddings for factual queries