How to Build Production-Ready Local AI Agents with Ollama & Python (2026 Code Tutorial)

Building AI applications in 2026 no longer requires transmitting sensitive proprietary code or customer data to external cloud APIs. With modern local open-weight reasoning models like DeepSeek-R1 and Llama 3.3, developers can run autonomous AI agents 100% offline on standard workstation hardware.

This practical tutorial walks you through setting up a production-ready Local Retrieval-Augmented Generation (RAG) AI Agent using Python, Ollama, LangChain, and ChromaDB.

1. Prerequisites & Environment Setup

Ensure you have Python 3.10+ and Ollama installed.

# 1. Install Ollama & Pull Models
curl -fsSL https://ollama.com/install.sh | sh
ollama pull deepseek-r1:8b
ollama pull nomic-embed-text

# 2. Create Python Virtual Environment
python3 -m venv venv
source venv/bin/activate

# 3. Install Dependencies
pip install langchain langchain-community langchain-chroma ollama pypdf

2. Step 1: Ingesting & Embedding Documents

We will load PDF documents into a local vector store (ChromaDB) using Ollama's nomic-embed-text embeddings.

# ingest.py
import os
from langchain_community.document_loaders import PyPDFLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_chroma import Chroma
from langchain_community.embeddings import OllamaEmbeddings

DB_PATH = "./chroma_db"

def ingest_documents(pdf_path: str):
    print(f"📄 Loading document: {pdf_path}")
    loader = PyPDFLoader(pdf_path)
    docs = loader.load()

    # Split document into chunks
    splitter = RecursiveCharacterTextSplitter(chunk_size=600, chunk_overlap=100)
    chunks = splitter.split_documents(docs)
    print(f"✂️ Created {len(chunks)} text chunks.")

    # Generate vector embeddings
    embeddings = OllamaEmbeddings(model="nomic-embed-text")
    vector_store = Chroma.from_documents(
        documents=chunks,
        embedding=embeddings,
        persist_directory=DB_PATH
    )
    print("✅ Local Vector DB updated successfully!")

if __name__ == "__main__":
    # Create sample document if not exists
    if not os.path.exists("sample.pdf"):
        with open("sample_notes.txt", "w") as f:
            f.write("Learntrix system architecture runs on Next.js 16 with client-side WebAssembly tools.")
    ingest_documents("sample.pdf")

3. Step 2: Building the Reasoning Agent Engine

Now we combine the Vector DB with DeepSeek-R1 to perform localized chain-of-thought answering.

# agent.py
from langchain_chroma import Chroma
from langchain_community.embeddings import OllamaEmbeddings
from langchain_community.llms import Ollama
from langchain.chains import RetrievalQA

DB_PATH = "./chroma_db"

def get_agent():
    # 1. Initialize local embeddings & vector store
    embeddings = OllamaEmbeddings(model="nomic-embed-text")
    vector_store = Chroma(
        persist_directory=DB_PATH,
        embedding_function=embeddings
    )

    # 2. Initialize local DeepSeek-R1 reasoning LLM
    llm = Ollama(model="deepseek-r1:8b", temperature=0.1)

    # 3. Create RAG Chain
    retriever = vector_store.as_retriever(search_kwargs={"k": 3})
    qa_chain = RetrievalQA.from_chain_type(
        llm=llm,
        chain_type="stuff",
        retriever=retriever,
        return_source_documents=True
    )
    return qa_chain

def ask_agent(query: str):
    chain = get_agent()
    print(f"\n🔍 Querying Agent: '{query}'...")
    result = chain.invoke({"query": query})
    
    print("\n🧠 Agent Response:")
    print(result["result"])
    
    print("\n📚 Sources Used:")
    for doc in result["source_documents"]:
        print(f" - Page {doc.metadata.get('page', 1)}: {doc.page_content[:80]}...")

if __name__ == "__main__":
    ask_agent("What technology stack does Learntrix use?")

4. Performance Optimizations for Local Agents

1. GPU Acceleration: On macOS, Ollama automatically utilizes Metal (Unified Memory). On Linux/Windows, ensure CUDA 12+ is installed for GPU offloading.
2. Quantization Selection: Use 4-bit quantized models (q4_k_m) to reduce VRAM requirements by ~60% with virtually zero loss in reasoning accuracy.
3. Context Trimming: Set strict context windows (k=3 or k=4) in vector retrievers to prevent prompt bloat.

Conclusion

With this architecture, you have built a 100% offline, zero-cost, enterprise-grade AI Agent capable of analyzing complex documentation without relying on external cloud APIs or subscriptions.