OpenTelemetry vs SigNoz

OpenTelemetry Go Logs API and SDK reach release candidate status

OpenTelemetry Go v1.47.0-rc.1 is here. This release promotes the Logs API and SDK to release candidate (RC), the final stage before we provide stable v1 compatibility guarantees. We believe the design is ready, and now we need the community to test …

LLMOps for Compound AI Systems: Observability & Cost

The article argues that most GenAI pilots fail in production due to insufficient system-level engineering rather than poor models. It presents an LLMOps playbook for compound AI systems (embedders, retrievers, vector stores, re-rankers, validators, tool calls, and multiple LLMs) centered on five controls: a model gateway for routing and budgeting, pipeline-level traces for end-to-end observability, semantic caching keyed by query embeddings, lightweight eval gates for safety and quality, and tiered scaling of heavy infrastructure. A concrete engineering example reports a 38% reduction in token spend and 25% lower median latency after implementing a gateway, semantic cache, and tracing. The post includes a short pseudocode example (using qdrant-style vector operations) and an operational checklist for iterating LLMOps as an operating model.

• The article defines five LLMOps controls: model gateway, pipeline-level traces, semantic caching, eval gates, and tiered scaling.
• Author recommends using OpenTelemetry-compatible spans to instrument embed, search, rerank, prompt build, LLM call, and tool call stages.
• A cited engineering example achieved a 38% reduction in token spend and 25% lower median latency after implementing three LLMOps controls.

agent-cost: Measure LLM Usage, Separate Task Attribution

The author describes agent-cost, a small tooling primitive that reads local logs from LLM CLIs (e.g., Claude Code and Codex) to produce auditable, machine-readable usage facts (model, token kind, timestamp, count) and an estimated price. The tool is designed to run with no network calls at runtime, carry a versioned price catalog (with SHA-256 digest), and keep session measurement distinct from task attribution. Unknown or unsupported pricing and ambiguous session-to-task bindings are surfaced (labels like "unpriced" or "lower_bound") rather than silently allocated. The author re-ran the published coding-agent-cost 0.1.0 package and notes a catalog version 2026-07-29 and workflows that validate the measure/v1 protocol and data quality.

• agent-cost reads local logs from LLM CLIs (examples: Claude Code and Codex) and normalizes usage events into facts containing model, token kind, timestamp, and count.
• At runtime agent-cost makes no network calls and declares no Python runtime dependencies; installation from PyPI still requires trust in the supply chain.
• agent-cost carries a versioned pricing catalog with a SHA-256 digest and marks unknown models/prices as 'unpriced' or 'lower_bound' instead of inventing values.

Deploying SigNoz with ClickHouse v25 and OTel Gotchas

A developer describes troubleshooting and deployment patterns for self-hosting SigNoz in 2026, focusing on ClickHouse v25 configuration changes, OpenTelemetry Collector incompatibilities, and Docker network isolation. Key recommendations include using config.d/users.d for ClickHouse overrides (instead of replacing the main config), enforcing a deterministic docker-compose boot sequence with one-shot migrator containers (signoz_init_clickhouse and signoz_telemetrystore_migrator), updating exporter names and image tags for the OTel collector (use signozclickhousemetrics and include the 'v' prefix), and relying on internal Docker network isolation rather than per-component passwords to simplify connectivity. The post emphasizes that maintaining self-hosted observability infrastructure requires ongoing engineering effort.

• Author self-hosted SigNoz for Verne Software to control telemetry data.
• ClickHouse v25+ changes require using config.d/ and users.d/ for overrides instead of replacing the main config file.
• SigNoz deployments must run one-shot containers (signoz_init_clickhouse and signoz_telemetrystore_migrator) to install UDFs and create required databases before starting collectors.

Integrating AI Agents with Self-Hosted SigNoz

An engineer describes building ArcNet to instrument and monitor an AI agent fleet using a self-hosted SigNoz instance. The write-up covers installation (SigNoz v0.133.0 via foundryctl), lessons about verifying emitted OpenTelemetry attributes (the Agno instrumentor emitted OpenInference conventions rather than gen_ai.*), turning guardrail results into structured span attributes for alerting, the need to use SigNoz's v5 alerts queries payload, using raw ClickHouse SQL panels as an escape hatch, and the distinction between telemetry (traces in SigNoz/ClickHouse) and replayable session transcripts (stored separately in SQLite). The author notes SigNoz MCP was unreliable in their setup and links code on GitHub.

• Author built ArcNet on self-hosted SigNoz for the Agents of SigNoz hackathon.
• SigNoz v0.133.0 was used and installed with foundryctl (casting.yaml + casting.yaml.lock produced reproducible deployments).
• The Agno instrumentor (openinference-instrumentation-agno) emitted OpenInference semantic conventions rather than OpenTelemetry gen_ai.* attributes, causing initially empty dashboards.

Instrumenting MERN E‑Commerce with SigNoz

A developer case study describing how the author instrumented a MERN-stack e-commerce application called Ram Store with SigNoz and OpenTelemetry during the Agents of SigNoz Hackathon 2026. The project used a self-hosted SigNoz instance (Docker) and the OpenTelemetry Node SDK with automatic instrumentation for Express, HTTP, and MongoDB, exporting via OTLP gRPC. The instrumentation enabled distributed traces, runtime metrics, and structured logs (Winston) correlated inside SigNoz. The post covers setup steps, observed telemetry (traces, metrics, logs), challenges (Docker networking, configuration), lessons learned, and planned improvements like custom dashboards and alerting. Project source code and a demo video are linked.

• Author integrated SigNoz into a MERN e-commerce application named Ram Store as part of the Agents of SigNoz Hackathon 2026.
• SigNoz was self-hosted locally using Docker to provide observability dashboards.
• OpenTelemetry Node SDK with automatic instrumentation was configured for Express, HTTP, and MongoDB, using an OTLP gRPC exporter.