AgentHub/docs/plans/analytics-parity

Analytics parity with the upstream agent monitor (hoangsonww)

Baseline compared: upstream hoangsonww's upstream agent monitor at 523170e (= tag v2.2.6, 2026-10-04) plus its open PRs, against AgentHub main at 151dc8a. Agreed with the user on 2026-10-05: phases 1–6 in full, plus Cursor CLI, a small MCP server, a statusline script and Grafana boards. NOT doing: the deep workflow charts and the web Kanban (dropped on purpose in 4a0f995), SSE (the websocket covers it), extra UI locales.

Each phase ships on its own, with tests, docs (update-project-docs) and the CLAUDE.md marker. Nothing is committed without the user's ask.

Phase 1 — data hygiene ✅

• events.data trimmed at every insert through the SQL function trim_event_data(?) (server/lib/event-payload.js, registered in server/db.js): Edit/MultiEdit/NotebookEdit/Write drop originalFile, oldString/newString (Write also content); Read drops file.content and file.base64; every string in tool_input/tool_response over 8 KB keeps a 6 KB head + 2 KB tail; structuredPatch and everything outside those two fields stay whole; _trimmed names what was dropped. The full text stays in the transcript and its snapshot under <data dir>/transcripts/.
• One-time sweep of stored rows (sweepEventPayloads, user_version 4, after marker 3), 200 rows per write transaction. No VACUUM (user's choice). Measured on the user's DB: 18,518 rows, 595 MB → 86 MB.
• The session list prices every column /api/pricing/cost/:id does (tokensForSessions in server/routes/sessions.js).

Phase 2 — cost by tool ✅

• ?tool= on /api/pricing/cost, /api/analytics, /api/stats, /api/workflows (+ /skill-usage): parseTool/parseScope in server/lib/source-filter.js (id-shaped, else 400), scoped-stats.js now takes a {where, params} predicate over sessions.
• by_tool in the cost response; Analytics gets an AI-tool select (applies to every sub-tab, Workflow included) and a Cost by AI tool card.
• /api/analytics's total_cost now equals /api/pricing/cost (it had ignored the compaction baselines and used UTC start dates).
• Codex service tier read from thread_settings_applied: fast/priority → speed = fast priced with LiteLLM's *_priority rates (synced into the fast columns for non-Claude models), flex → service_tier = flex at half price.
• Prometheus: agenthub_cost_usd_total{tool,model}.
• DEFERRED (with the reason in pricing-sync.js): long-context (>200k/>272k) rates. No current Claude model has one, Codex here runs a 258k window (0 of 17,701 turns over 272k), so a token_usage primary-key rebuild would change no number today.

Phase 3 — budgets, burn rate, forecast ✅ (budgets as alert rules)

• budgets table: name, period (day/week/month), limit USD, scope (all / tool / workspace), thresholds (%), enabled. /api/budgets CRUD + GET /api/budgets/status (spend this period, %, projected end-of-period).
• Evaluated on the alert beat; a crossed threshold fires once per period through the existing alerts feed → webhooks → push.
• Alert rule type cost_threshold (USD per session / per day).
• POST /api/alerts/rules/preview: dry-run a rule over recorded history.
• UI: Settings → Budgets page; burn rate + month-end forecast tiles on Analytics → Cost; budget chip in the status bar when ≥ a threshold.
• As built: a budget is an alert rule of type budget (server/lib/budgets.js; config period / limit_usd / thresholds / tool / lane / tz_offset), so there is no budgets table, no /api/budgets CRUD and no Settings → Budgets page: budgets are created and edited with the other rules in Settings → Notifications → Alert rules (editable in place since v2.3), and GET /api/budgets/status is the one budgets route. cost_threshold, the preview, the Spend pace + Budgets cards and the status-bar chip are as planned.

Phase 4 — analytics UX ✅

• Date range. parseRange (server/lib/source-filter.js): from/to are local YYYY-MM-DD, inclusive, read with tz_offset, either end open; a bad date (or year 9999, or from > to) is 400 INVALID_INPUT. GET /api/pricing/cost keeps only the cost rows of those spend days for total_cost / breakdown / by_tool (the existing spend-day split); daily_costs stays the whole series so the 30-day charts and the spend pace keep their windows. GET /api/analytics counts sessions/agents by started_at and events by created_at (an optional range on the scoped-stats.js functions), and takes tokens and cost from the same cost rows (token_usage has no date); daily_events/daily_sessions keep 365 days. GET /api/workflows (+ /skill-usage): sessions started in the range. The bodies are now computeAnalytics / computeCost (exported). Client: a range select on the Analytics toolbar beside the AI-tool select — 7d / 30d / 90d / All (default) / Custom with two <input type="date"> — in ?range= (+ from/to), client/src/lib/dateRange.ts; it also reaches the Workflow sub-tab.
• Query Explorer. GET /api/query (server/routes/query.js): entity sessions / events / agents, q (literal substring), tool, status, from/to, cwd (folder subtree), sources; sort/dir from a per-entity allowlist (anything else 400 — no request text reaches SQL), ties by id; pages of ≤ 500; format=csv|json exports the whole filter capped at 50,000 rows (X-Total-Count, X-Truncated). CSV: RFC 4180, CRLF, BOM, ' before a cell starting with = + - @ tab CR. Page /query (client/src/pages/QueryExplorer.tsx), sidebar entry Query Explorer (the sidebar has no analytics item of its own; it sits under Sessions). Column headers show the raw field names, which are also the CSV header.
• Scheduled reports. Tables report_definitions / report_runs; server/lib/reports.js + /api/reports (CRUD, POST /:id/run, runs, GET /:id/runs/:runId/:format with the path built from ids + the stored stamp). A run covers the window_days full local days BEFORE the run day, computed with computeAnalytics + computeCost, plus the current budget status; files <data dir>/reports/<id>/<stamp>.{html,json}, HTML fully escaped. Scheduler: unref'd interval in startBackgroundServices (DASHBOARD_REPORTS_MS, default 60 s, 0 = off), each slot claimed by one conditional UPDATE (two dashboards on one DB run it once); a slot missed while down runs once on the next start; create/edit/re-enable claims the slot already past. Weekly reports have a weekday (default Monday) — needed for "weekly", not in the original list. Writes same-origin. Definitions AND runs survive Clear Data (like alert rules and snapshots). UI: Settings → Reports (components/settings/ReportSettings.tsx).
• Not done: OpenAPI entries for the new routes (snapshots/budgets have none either); report runs are never pruned (one file per run per format).
• Upstream's PR branches (#286/#172 feat/query-explorer, #174 feat/scheduled-reports) were not in the reference clone; built from the plan above.
• Tests: analytics-range, query-explorer, reports in server/__tests__/; client dateRange, QueryExplorer, ReportSettings, and a ?range= case in screens.snapshot.test.tsx.

Phase 5 — sessions ✅

• Replay: a Replay tab on /sessions/:id beside Agents / Conversation / Timeline (client/src/components/EventReplay.tsx). It reads the session's events through GET /api/events?session_id= in pages of 500 (the newest 5,000 at most, with a note when there are more), oldest first, on the tab's first visit. A range slider over the event SEQUENCE (not the wall clock — an hour-long pause does not stall it), play/pause at 0.5×/1×/2×/5× (one event per 500 ms at 1×), step back/forward, jump start/end, keys ←/→/Home/End/Space, a mini-map of lifecycle markers (SessionStart, UserPromptSubmit, Compaction, SubagentStop, APIError, Stop) that jump the cursor, a state panel derived up to the cursor (status, tool in flight, prompts, errors, compactions, elapsed, tool-call counts) and the selected event through EventDetail. The pure derivations live in client/src/lib/replay.ts (unit-tested).
• Snapshots: POST /api/snapshots {session_id, redact?: {file_paths?}} freezes {session, agents, events, stats, cost} into an immutable session_snapshots row (a trigger refuses UPDATE) under a 24-byte base64url token; GET /api/snapshots (no payload), GET /api/snapshots/:token, DELETE /api/snapshots/:id; writes behind sameOriginGuard. Redaction runs at capture: path-valued keys → [redacted], the session cwd / home dir in any other string → <cwd> / ~. Snapshots survive Clear Data, like alert rules. UI: a Snapshot action in the session header (confirm with a "redact file paths" box, default on; copies the link), the read-only page /snapshot/:token (client/src/pages/SnapshotView.tsx: SessionOverview fed the frozen stats via its new stats prop, the agents, Replay and Timeline — the Timeline list now client/src/components/EventTimeline.tsx, shared with the analysis page), and Settings → Snapshots (/settings/snapshots, components/settings/SnapshotSettings.tsx: list, open, copy link, delete with confirm, the same on a right-click menu).
• Upstream's PR branches (#265 feat/session-replay, #173 feat/session-snapshots) were not in the reference clone; built from the plan above.

Phase 6 — ingest breadth ✅

• Remote Codex. A remote source has an opt-in sync_codex (+ optional remote_codex_home, default ~/.codex, wsl:~/.codex beside a wsl: Claude home) — two additive columns on remote_sources, a checkbox in the source form, --codex/--codex-home on agenthub remote-sources add. A sync mirrors sessions/ with mtimes kept (scp -r -p, or tar from WSL) plus session_index.jsonl into <data dir>/remote-sources/<id>/codex/, then runs ingestCodexUsage({home, source, activeMs}) — the same journal reader as local Codex (server/lib/codex-usage.js), so sessions arrive as tool = 'codex' rows tagged with the source id, with tokens and cost. A pass only ever writes rows of its own source; the local settle never touches a remote row; the remote activity window is DASHBOARD_REMOTE_ACTIVE_WINDOW_MS. With Codex on, a machine may have only one CLI: one half failing is ok + last_error. Differs from upstream (which always tries both): opt-in, and no separate Codex import pipeline — AgentHub's journal reader is reused.
• Push ingest. POST /api/hooks/ingest-batch — body {source, events: [{hook_type, data, tool?}]}, each event through the SAME function as POST /api/hooks/event (ingestHook in server/routes/hooks.js). Off (503) until Settings → Data → Push ingest creates a token: 32 random bytes, shown once, stored only as its SHA-256 (AGENTHUB_PUSH_TOKEN_SHA256 in the settings file), checked in constant time; rotate/revoke; GET never returns it. Sessions tagged push:<source>; a pushed hook may only extend a session its own source created (EFOREIGN); transcript_paths are dropped (they name the sender's files); no lane/pane binding; 500 events / 1 MB. The UI shows a forwarder script keeping the token in a curl header file, not argv. Differs from upstream: upstream's wire format is a collector's own session/ token/turn records; AgentHub takes plain hook payloads, so a pushed session has events and live status but no token cost (that needs transcripts).
• Live Codex overlay — not built. Upstream's include_transient (codex-process-overlay.js) scans ps for an interactive codex TUI that has not written a rollout yet, i.e. one started but never prompted. In AgentHub a Codex pane started from the dashboard already shows (its tab, and New session in the sidebar from its run), and anything that has done work has a journal the 15 s poll reads (or a SessionStart hook with Codex hooks on). The only row it would add is an idle Codex in an outside terminal with no prompt and no cost — not worth a per-tick process scan.
• Privacy (server/lib/privacy.js, Settings → Data → Privacy, GET/PUT /api/settings/privacy, both off by default, persisted to the settings file, env-pinnable): (a) Redact secrets (AGENTHUB_REDACT_SECRETS=1) — chained in front of the trim in trim_event_data (so a key is redacted before a trim could cut it) plus a new redact_text(?) on every events summary insert; known key shapes (Anthropic, OpenAI, GitHub, GitLab, Slack, AWS, Google, npm, JWT), bearer/basic headers, PEM private keys, password=/*_TOKEN:-style values, and any string under a secret-named JSON key → [redacted], with _redacted: <n> on the payload. New data only; the transcript is untouched; session NAMES (first prompt) are not redacted. (b) Ignored folders (AGENTHUB_IGNORE_CWD, path.delimiter-separated, path-boundary match): hooks dropped whole (ignored: true), and the history import and the Codex reader skip those sessions too.

Phase 7 — extras chosen by the user

• Cursor CLI: session + token ingest from ~/.cursor, Cursor pricing rows.
• MCP server (stdio, read-only, ~15 tools: stats, cost, sessions, session detail, analytics, workflows, budgets, alerts).
• statusline/ script for Claude Code (model, branch, context %, session $, today $ from the dashboard).
• monitoring/ Grafana dashboards + Prometheus rules for /api/metrics.

Phase 7 status — MCP, statusline, monitoring ✅; Cursor ✅ (no cost)

• MCP (mcp/server.js, agenthub mcp, docs/MCP.md): JSON-RPC over stdio written directly (no SDK), 15 GET-only tools (health, info, stats, total / session / lane cost, sessions, session, session stats, analytics, workflows, alerts, lanes, pricing, budget status). AGENTHUB_URL, else the CLI's discovery, now shared as server/lib/dashboard-url.js. A missing route — 404 or the production SPA page served with 200 — reads "Not available on this dashboard", which is what get_budget_status says until Phase 3 lands. NOT auto-registered by the plugin: claude mcp add agenthub -- agenthub mcp.
• Statusline (statusline/statusline.js, agenthub statusline install|uninstall [--force], docs/STATUSLINE.md): model · branch · context bar · session · today (/api/pricing/cost, 1 s timeout, 30 s cache in the OS temp dir, failures cached). Install refuses a foreign statusLine without --force and backs settings.json up first; a plugin install registers the ~/.local/bin/agenthub launcher, not the cache dir.
• Monitoring (monitoring/, docker-compose.monitoring.yml, docs/MONITORING.md): scrape config, agenthub:* recording rules, boards Overview / Sessions & agents / Tokens & cost; the spend panels read Phase 2's agenthub_cost_usd_total{tool,model} and say so.
• Tests: mcp-server, mcp-server-live, statusline, monitoring in server/__tests__/.
• Cursor (server/lib/cursor-usage.js): agent transcripts under ~/.cursor/projects/<slug>/agent-transcripts/ become tool = 'cursor' sessions (cursor_transcripts table, DASHBOARD_CURSOR_POLL_MS, AGENTHUB_CURSOR_DIR). NOT built: tokens, model and cost — the transcripts record none, and state.vscdb (which has them) also holds auth tokens, so it is never read; hence no Cursor pricing rows either.

Ported onto main

Phases 2–7 were built in a separate worktree on 138e706 and ported onto v2.3.1 on 2026-10-06 (phase 1 had shipped on its own), released as v2.4.0.