Baseten vs Qdrant

Goodfire Launches Internal AI Agent Monitors

Goodfire, a startup specializing in AI interpretability, launched on Thursday a new type of AI agent monitor that inspects a model's internal signals rather than reading its output, aiming to detect rogue behaviors more efficiently and at a fraction of the cost. The monitors are available to customers of Baseten, an AI model hosting platform. Baseten's Base Labs had previously announced a safety partnership with Goodfire and Hugging Face. Goodfire's approach uses small probes that scan a model's internal activations at each step, triggering a closer AI review only when flagged. In tests on the Kimi K3 model, Goodfire's monitors caught 94% of malicious hacking sessions and cost about $51 for 1,500 sessions, compared to $233 for a cheaper model and $10,000 for a top-tier one. The company positions the solution for open models, which can be stripped of safeguards, and sees it as critical for inference-time guardrails.

• Goodfire launched internal activation monitors for AI agents on October 8, 2026.
• Monitors are available to Baseten customers.
• In tests on Kimi K3, monitors caught 94% of malicious hacking sessions.

Announcing our partnership with OpenAI

Baseten announces a partnership with OpenAI, as featured on their blog.

Baseten, Hugging Face, Goodfire Partner for AI Safety

Baseten's research arm, Base Labs, announced a partnership with Hugging Face and Goodfire AI to establish a safety standard for open-weight AI models. The collaboration aims to develop and publish methods for training and monitoring these models, making safety an integrated feature rather than an afterthought. This comes amid rising concern over 'abliteration,' a technique that removes safeguards from open models, with Hugging Face listing over 6,000 such models. The companies have not detailed technical specifics, but Goodfire's expertise in AI interpretability is expected to play a key role. This initiative seeks to leverage openness as an advantage for safety, providing transparent controls and encouraging community contributions.

• Baseten launched a safety infrastructure standard via Base Labs.
• Partnership with Hugging Face and Goodfire AI announced.
• Hugging Face hosts over 6,000 abliterated models.

LLMOps for Compound AI Systems: Observability & Cost

The article argues that most GenAI pilots fail in production due to insufficient system-level engineering rather than poor models. It presents an LLMOps playbook for compound AI systems (embedders, retrievers, vector stores, re-rankers, validators, tool calls, and multiple LLMs) centered on five controls: a model gateway for routing and budgeting, pipeline-level traces for end-to-end observability, semantic caching keyed by query embeddings, lightweight eval gates for safety and quality, and tiered scaling of heavy infrastructure. A concrete engineering example reports a 38% reduction in token spend and 25% lower median latency after implementing a gateway, semantic cache, and tracing. The post includes a short pseudocode example (using qdrant-style vector operations) and an operational checklist for iterating LLMOps as an operating model.

• The article defines five LLMOps controls: model gateway, pipeline-level traces, semantic caching, eval gates, and tiered scaling.
• Author recommends using OpenTelemetry-compatible spans to instrument embed, search, rerank, prompt build, LLM call, and tool call stages.
• A cited engineering example achieved a 38% reduction in token spend and 25% lower median latency after implementing three LLMOps controls.

Vector Database vs Knowledge Graph for LLMs

A practical guide comparing vector databases and knowledge graphs as storage backends for LLM applications. The article explains that vector databases (e.g., Qdrant, pgvector, Pinecone) store embeddings and excel at semantic similarity queries, while knowledge graphs (e.g., Neo4j, RDF stores) model entities and relationships and support exact, multi-hop relational queries. The author argues most real-world LLM systems need a hybrid: vectors for retrieval and graphs for precise reasoning and auditability (a pattern exemplified by Microsoft’s GraphRAG). Tool-level trade-offs, operational costs, and a four-question decision rule are provided to help teams choose the right architecture for their queries.

• Vector databases store high-dimensional embeddings and are optimized for semantic similarity (examples: Qdrant, pgvector, Pinecone, Milvus, Weaviate, Chroma).
• Knowledge graphs store entities and relationships as first-class citizens and support exact traversals and multi-hop queries using languages like Cypher or SPARQL (examples: Neo4j, RDF stores).
• The author has deployed both approaches in production: vector solutions (Qdrant and pgvector) and Neo4j for graph workloads.

AI Agents Need Vector Databases for Memory

This technical blog post explains why retrieval-backed long-term memory for AI agents is best implemented with vector databases. It defines three memory types (working, long-term, episodic), outlines the memory stack (embedding model, vector store, chunking, metadata), recommends practical tooling (pgvector, Qdrant, Chroma) and embedding-dimension trade-offs, and provides a minimal Python example using pgvector and OpenAI embeddings. The author lists common production failure modes (stale memory, poor chunking, blind cosine similarity, context overflow, cost, privacy, and silent quality rot) and a practitioner's checklist for safe, private, and maintainable memory-enabled agents.

• Vector databases enable retrieval-backed long-term memory for agents by returning semantically similar chunks via embeddings and ANN search.
• Author recommends tooling trade-offs: pgvector (Postgres extension) for most production cases, Qdrant for heavy metadata filtering/scale, and Chroma for quick prototypes.
• Embedding dimension and model choice trade quality against cost; 768 dimensions is suggested as a sane production default.