Chroma

Field Guide: Production-Grade RAG Architectures

This technical guide maps Retrieval-Augmented Generation (RAG) as a design space and describes practical production patterns and failure modes. It defines three evolutionary paradigms — Naive RAG, Advanced RAG (pre/post-retrieval optimizations), and Modular RAG (composable pipelines) — and catalogs eight architectural patterns: Standard (Dense), Hybrid, GraphRAG, Corrective RAG (CRAG), Self-RAG, Adaptive RAG, Agentic/Multi-Agent RAG, and Multi-Modal RAG. The article explains common production failures (chunking, semantic drift, multi-hop needs, static top-k, hallucination) and recommends incremental upgrades — notably hybrid dense+sparse search with re-ranking — and routing by query complexity. It includes runnable Python examples for hybrid retrieval + re-ranking and a simple CRAG-style relevance gate, plus an architectural decision matrix comparing complexity, latency, cost, and best use cases.

Local RAG Evolved into Agentic AI with LangGraph

A developer describes converting a locally hosted RAG assistant (built with Ollama, ChromaDB, LangChain, Docker) into an agentic AI architecture using LangGraph. The author introduces a shared AgentState contract and implements three single-purpose agents — a RAG agent for documentation lookup, a Diagnostic agent with a fast known-error lookup and LLM fallback, and an Escalation agent that generates structured tickets when human intervention is required. An orchestrator uses a classifier to route queries conditionally through a state graph. The article discusses design lessons (classifier fragility, embedding initialization overhead, hardcoded escalation thresholds) and recommends starting with RAG and adding agents where needed.

Local RAG Assistant with Ollama, ChromaDB, LangChain

A Master's student built a local Retrieval-Augmented Generation (RAG) assistant to let technicians query private PDF manuals without sending data to cloud providers. The pipeline uses 300-character chunking, all-MiniLM-L6-v2 embeddings stored in ChromaDB, retrieval of the top 3 chunks, and local Llama 3 inference via Ollama. The system runs as four Docker Compose services (Ollama, ChromaDB, FastAPI, Streamlit). The author documents three practical failures and fixes: ChromaDB v2 silently storing data without an explicit HttpClient, LangChain refactoring into langchain_core, and slow Llama 3 CPU inference (mitigated by reducing retrieved chunks, capping responses with num_predict, and adding RAM). The project is open-source on GitHub and the author plans to evolve the pipeline toward an agentic architecture.