ChromaDB

LangChain RAG tutorial

Ollama RAG Tutorial for Beginners, This tutorial RAG with Ollama and ChromaDB demonstrates how to build a local Retrieval-Augmented Generation (RAG) system using Ollama for embeddings, ChromaDB for vector storage, and LangChain-style querying—without any cloud APIs.

Retrieval Augmented Generation Python

We designed this RAG model using Python to ensure wide adoption, as Python is easy to learn and is more accessible compared to many other programming languages.

Feel free to reach out if you are looking for any other programming language.

Ollama embeddings RAG

Used Ollama in generating embeddings.

ChromaDB vector database

Embeddings generated in above step are stored in chroma DB

This tutorial demonstrates a Local RAG system built using Python, combining an Offline RAG LLM, semantic search with embeddings, and efficient vector search in Python to deliver accurate, privacy-first AI responses without cloud dependencies.

Complete Code

import ollama
import chromadb

client = chromadb.Client()

collection = client.create_collection(name="my_embeddings")

sentences = [
    "ollama makes running LLMs locally easy.",
    "Sentence embeddings are useful for search.",
    "Java is great for machine learning."
]


for idx, sentence in enumerate(sentences):
    response = ollama.embeddings(
        model="nomic-embed-text",
        prompt = sentence
    )
    collection.add(
        ids = [str(idx)],
        documents=[sentence],
        embeddings=[response["embedding"]]
    )
print("Embeddings are stored")

query = "Which language is great for machine learning"

query_embedding = ollama.embeddings(
    model="nomic-embed-text",
    prompt = query
)["embedding"]

results = collection.query(
    query_embeddings = [query_embedding],
    n_results=1
)

print(results["documents"])

context = results["documents"][0][0]

prompt = f"""
Use the context to answer the question, make sure the answer is in 1 or 2 sentences and crisp

Question:
Which language is great for machine learning?
"""

response = ollama.generate(
    model="amplifyabhi",
    prompt=prompt
)

print(response["response"])

Ollama RAG Tutorial for Beginners Complete Playlist

For more interesting updates on RAG with Ollama and ChromaDB stay subscribed to Amplifyabhi