Observed Signal · Jul 26, 2026 · Technical Implementation · Source: DEV Community · Impact: 2/5 · Sentiment: Positive
Application Performance Monitoring (APM) Market: Integrating AI Agents with Self-Hosted SigNoz
An engineer describes building ArcNet to instrument and monitor an AI agent fleet using a self-hosted SigNoz instance. The write-up covers installation (SigNoz v0.133.0 via foundryctl), lessons about verifying emitted OpenTelemetry attributes (the Agno instrumentor emitted OpenInference conventions rather than gen_ai.*), turning guardrail results into structured span attributes for alerting, the need to use SigNoz's v5 alerts queries payload, using raw ClickHouse SQL panels as an escape hatch, and the distinction between telemetry (traces in SigNoz/ClickHouse) and replayable session transcripts (stored separately in SQLite). The author notes SigNoz MCP was unreliable in their setup and links code on GitHub.
Practical, technical guidance for instrumenting and alerting on AI agent fleets with self-hosted observability (SigNoz). Useful for engineering teams running GenAI/agent systems but not industry-shifting platform news.
Key Takeaways & Evidence Grounding
- Author built ArcNet on self-hosted SigNoz for the Agents of SigNoz hackathon.
- SigNoz v0.133.0 was used and installed with foundryctl (casting.yaml + casting.yaml.lock produced reproducible deployments).
- The Agno instrumentor (openinference-instrumentation-agno) emitted OpenInference semantic conventions rather than OpenTelemetry gen_ai.* attributes, causing initially empty dashboards.
- Guardrail verdicts were emitted as structured span attributes (e.g., arcnet.guard.checkpoint, arcnet.guard.action, arcnet.guard.rule, arcnet.guard.risk_score) so they could be aggregated and alerted on; SigNoz alerts API expects the v5 queries payload.
- Traces in SigNoz/ClickHouse are intended for telemetry and not for exact replay transcripts; sessions were recorded to SQLite for replay, and SigNoz MCP server was noted as unreliable in this setup.
Connected Companies & Entities
2 Entities mappedSigNoz
Open-source observability and APM platform for engineering teams.
“I built ArcNet on self-hosted SigNoz for the Agents of SigNoz hackathon, and most of what I learned was about SigNoz internals I could not h...”
ClickHouse
Open-source analytics database with managed cloud and observability.
“SigNoz lets you drop to a raw ClickHouse SQL panel against the traces table....”
Ontology Mapping & Concepts
Track Real-Time Market Signals & Shifts
Set up custom watchlists to receive automated, evidence-grounded executive digests whenever material signals or shifts occur across your tracked landscape.