Files
AgentHub/docs/MONITORING.md
T
nntrivi2001 4aae61297f feat: analytics parity — tool cost, date ranges, budgets, query explorer, replay, snapshots (v2.4.0)
Analytics parity phases 2–7 (docs/plans/analytics-parity/), built on
138e706 and ported onto v2.3.1:
- Cost by AI tool (?tool= on cost/analytics/stats/workflows, by_tool,
  Codex fast/priority and flex tiers, agenthub_cost_usd_total{tool,model})
- Date ranges (from/to) on cost/analytics/workflows; Analytics range select
  with Today / This week (Monday) / This month (1st) / 7d / 30d / 90d / All /
  Custom; workflow filters carried into workflow-stats.js (read worker)
- Budgets and cost alerts as alert rule types (alert_rules rebuilt without
  its rule_type CHECK), GET /api/budgets/status, rule preview, spend pace,
  status-bar budget chip; rules stay editable in place
- Query Explorer (/query, allowlisted sort, CSV/JSON export) and scheduled
  reports (report_definitions/report_runs, Settings → Reports)
- Session Replay tab and immutable snapshots (session_snapshots, /snapshot/:token)
- Remote Codex sync, push ingest with a hashed token, privacy redaction and
  ignored folders (Settings → Data), Cursor sessions (no cost)
- Read-only MCP server, statusline, Grafana/Prometheus files

Also: resizable sidebar, pet run/wave animations, the workspace home keeps a
half-typed prompt, and the pre-commit hook runs only the related tests for a
small commit.
2026-10-06 17:12:28 +07:00

3.4 KiB

Prometheus and Grafana for AgentHub

AgentHub exposes its live counters at GET /api/metrics in the Prometheus text format (see API.md § Prometheus exposition). monitoring/ holds a scrape config, recording rules and three Grafana boards built on those series, and docker-compose.monitoring.yml runs both tools with everything provisioned.

Files

Path What
monitoring/prometheus/prometheus.yml Scrape config: job agenthub, metrics_path: /api/metrics, target host.docker.internal:4820
monitoring/prometheus/agenthub-rules.yml Recording rules agenthub:* (sessions, agents, event and token rates, cost)
monitoring/grafana/dashboards/agenthub-overview.json Overview: up, version, active sessions, working agents, tokens and spend over 24 h, sessions/agents by status, events/min, token burn, RSS, uptime, websocket clients, remote sources
monitoring/grafana/dashboards/agenthub-sessions-agents.json Sessions & agents: counts by status, errored sessions, agents waiting on you, sessions finished per hour
monitoring/grafana/dashboards/agenthub-tokens-cost.json Tokens & cost: tokens per hour by kind, burn rate, cache-read share, spend all-time / 24 h, spend per hour by tool, spend by tool, top models by spend
monitoring/grafana/provisioning/ The Prometheus datasource and the dashboard provider for the compose stack
docker-compose.monitoring.yml Prometheus + Grafana, both on loopback only

The boards choose their datasource through a ds variable, so they also import as they are into an existing Grafana (Dashboards → New → Import).

Metrics used

agenthub_up, agenthub_build_info{version}, agenthub_process_uptime_seconds, agenthub_process_resident_memory_bytes, agenthub_sessions{status}, agenthub_agents{status}, agenthub_events_total, agenthub_websocket_clients, agenthub_remote_sources{enabled}, agenthub_tokens_total{kind}.

The spend panels and the agenthub:cost_usd:* rules read agenthub_cost_usd_total{tool,model} (estimated USD, only ever rising except when pricing is recomputed). A dashboard that does not export it yet leaves those panels empty; each one says so in its description. server/__tests__/monitoring.test.js fails if a board or rule queries any other metric server/routes/metrics.js does not export.

Run it with Docker

# 1. The dashboard on the host must accept the container's Host header
DASHBOARD_ALLOWED_HOSTS=host.docker.internal agenthub start   # or npm start
# 2. Prometheus + Grafana
GF_SECURITY_ADMIN_PASSWORD='choose-one' docker compose -f docker-compose.monitoring.yml up -d
# 3. Grafana: http://127.0.0.1:3000 (admin / the password above), folder "AgentHub"
#    Prometheus: http://127.0.0.1:9090

Without step 1 every scrape gets 403 EBADHOST (the DNS-rebinding guard). When the dashboard sets DASHBOARD_TOKEN, put the token in monitoring/prometheus/dashboard-token and uncomment the authorization block in prometheus.yml. Stop with docker compose -f docker-compose.monitoring.yml down (-v also drops the stored series).

Without Docker

Point an existing Prometheus at the dashboard — copy the agenthub job from prometheus.yml with the target changed to 127.0.0.1:4820 (no DASHBOARD_ALLOWED_HOSTS needed for loopback), add agenthub-rules.yml to its rule_files, and import the three JSON boards into Grafana.