PostgreSQL

Ship a RAG Chatbot with Claude, pgvector, FastAPI

This technical how-to shows how to build a retrieval-augmented generation (RAG) chatbot in a weekend using three components: PostgreSQL with the pgvector extension as the vector store, FastAPI as a thin web layer, and Anthropic's Claude for text generation. Claude does not provide an embeddings endpoint, so an external embedding provider is required (examples used are Voyage AI, OpenAI, or local sentence-transformers). The guide explains practical schema choices (embedding dimension must match pgvector column), ingestion and chunking, HNSW indexing for fast nearest-neighbor search, and the single-SQL retrieval pattern (ORDER BY embedding <=> query LIMIT k). It also emphasizes grounding via a strict system prompt and lists production hardening tasks (connection pooling, evaluation, better chunking, streaming, and auth) that should follow the initial weekend prototype.

Kmemo: Semantic LLM Cache That Avoids Wrong Hits

Kmemo is an open-source semantic cache for LLM calls that supplements embedding similarity with a chain of lexical guards and optional verification to avoid serving incorrect cached answers. Published as kmemo-core 1.0.0 (Apache-2.0), it embeds prompts once, reuses vectors for lookup and writes, coalesces concurrent requests, and exposes explainability and metrics hooks. Kmemo ships integrations and store adapters (in-memory, Redis/RediSearch KNN, Postgres/pgvector, optional in-process HNSW) and provides configurable guard strictness, threshold calibration, and an optional Verifier model for world-knowledge near-misses. On a blind validation split, its guards reject 67% of near misses and retain 88% of paraphrases. The project is available on GitHub (NaCode-Studios/Kmemo) and Maven Central (io.github.nacode-studios:kmemo-core:1.0.0).

Production RAG at Scale: Lessons from 10,000+ Listings

A hands‑on account of running a production RAG pipeline that processes over 10,000 job listings daily. The author describes production decisions and tradeoffs around chunking strategy, embedding models (OpenAI vs self‑hosted Llama via Ollama), vector storage (Pinecone for prototyping, pgvector/PostgreSQL in production), LLM scoring cost controls (OpenAI Batch API, caching, model tiering, function calls), and observability (structured logging, correlation IDs, Sentry, LogRocket). The post emphasizes data normalization, designing chunkers to match document structure, and operational considerations that move RAG from demo to stable, affordable production service.