feat: analytics parity — tool cost, date ranges, budgets, query explorer, replay, snapshots (v2.4.0)
Analytics parity phases 2–7 (docs/plans/analytics-parity/), built on
138e706 and ported onto v2.3.1:
- Cost by AI tool (?tool= on cost/analytics/stats/workflows, by_tool,
Codex fast/priority and flex tiers, agenthub_cost_usd_total{tool,model})
- Date ranges (from/to) on cost/analytics/workflows; Analytics range select
with Today / This week (Monday) / This month (1st) / 7d / 30d / 90d / All /
Custom; workflow filters carried into workflow-stats.js (read worker)
- Budgets and cost alerts as alert rule types (alert_rules rebuilt without
its rule_type CHECK), GET /api/budgets/status, rule preview, spend pace,
status-bar budget chip; rules stay editable in place
- Query Explorer (/query, allowlisted sort, CSV/JSON export) and scheduled
reports (report_definitions/report_runs, Settings → Reports)
- Session Replay tab and immutable snapshots (session_snapshots, /snapshot/:token)
- Remote Codex sync, push ingest with a hashed token, privacy redaction and
ignored folders (Settings → Data), Cursor sessions (no cost)
- Read-only MCP server, statusline, Grafana/Prometheus files
Also: resizable sidebar, pet run/wave animations, the workspace home keeps a
half-typed prompt, and the pre-commit hook runs only the related tests for a
small commit.
This commit is contained in:
+74
-5
@@ -142,6 +142,8 @@ erDiagram
|
||||
text identity_file "Optional private-key path (NULL = SSH default)"
|
||||
text remote_home "Optional remote Claude home (NULL = remote ~/.claude)"
|
||||
integer enabled "1 = eligible for sync, 0 = disabled"
|
||||
integer sync_codex "1 = also pull Codex history (default 0)"
|
||||
text remote_codex_home "Optional remote Codex home (NULL = ~/.codex)"
|
||||
text status "idle | syncing | ok | error"
|
||||
text last_error "Last failure message, or NULL"
|
||||
text last_sync_at "ISO8601 timestamp of last successful sync, or NULL"
|
||||
@@ -210,8 +212,8 @@ CREATE TABLE sessions (
|
||||
| `awaiting_input_since` | TEXT | YES | ISO 8601 stamp set when the session is **Waiting** (Stop, SessionStart with source `startup`/`resume`/`clear`, permission Notification, or watchdog user-interrupt/Esc recovery). NULL otherwise. A SessionStart with source `compact` (auto-compaction fires mid-turn while Claude is working) leaves this column untouched, so a genuinely-active session is not mislabeled Waiting |
|
||||
| `awaiting_reason` | TEXT | YES | Why the row is waiting: `notification`, `stop`, `session_start`, or `interrupted`. Set/cleared in lock-step with `awaiting_input_since` (SessionStart→`session_start`, Stop→`stop`, permission/input Notification→`notification`, watchdog/Esc recovery→`interrupted`). NULL otherwise. Exception: a `compact`-source SessionStart preserves the existing value (neither stamps `session_start` nor clears it) |
|
||||
| `transcript_path` | TEXT | YES | Absolute path to the session's JSONL transcript. Written by `routes/hooks.js` on the first event that carries it (subsequent events no-op via a SQL guard) and read by the periodic compaction sweep — so the sweep touches only active session rows instead of scanning the entire `events` table for `json_extract(data,'$.transcript_path')`. Backfilled once from `events` by the `db.js` migration |
|
||||
| `source` | TEXT | NO | Data source this session was captured from. `'local'` for this machine's own Claude Code history (the default); otherwise the `remote_sources.id` of the remote SSH machine it was pulled from. Powers the `sources` query filter on `/api/sessions`, `/api/events`, `/api/agents`, `/api/stats`, and `/api/analytics`, and the `sources` facet on `/api/sessions/facets`. Indexed by `idx_sessions_source` |
|
||||
| `tool` | TEXT | NO | The AI coding CLI that produced the session. `'claude'` for every hook-created and imported row (the default); `'codex'` for a session ingested from Codex's rollout journals by `server/lib/codex-usage.js` (see `codex_rollouts`) or created by a Codex hook (`tool: "codex"`, `server/lib/codex-hooks.js`) — a Codex session keeps the status its hooks give it while they are current (its newest event at or after its newest journal write minus two minutes); once they stop (removed, untrusted) the journal decides again. A codex row has no `transcript_path`, and, going by the journal, its `status` is `active` only while one of its journals was written in the last two minutes. `'antigravity'` for an agy CLI conversation, ingested by `server/lib/antigravity-usage.js` (see `antigravity_conversations`) or created by an Antigravity hook (`server/lib/antigravity-hooks.js`); no `transcript_path`, and `active` until it has been quiet — no write to its conversation file (or a non-empty `-wal`) or transcript, no hook event — for ten minutes when it has hook events, two minutes when it has none; it then settles to `completed`, its agents too. The startup cleanup of stale `active` sessions and a workspace purge touch only the matching tool's rows (a purge takes the sessions of the workspace's own `tool`). The Workspace's attach/resume list offers `'claude'` rows only |
|
||||
| `source` | TEXT | NO | Data source this session was captured from. `'local'` for this machine's own Claude Code history (the default); otherwise the `remote_sources.id` of the remote SSH machine it was pulled from, or `push:<name>` for a session another machine pushed to `POST /api/hooks/ingest-batch` (a pushed hook may only extend a session its own source created). Powers the `sources` query filter on `/api/sessions`, `/api/events`, `/api/agents`, `/api/stats`, and `/api/analytics`, and the `sources` facet on `/api/sessions/facets`. Indexed by `idx_sessions_source` |
|
||||
| `tool` | TEXT | NO | The AI coding CLI that produced the session. `'claude'` for every hook-created and imported row (the default); `'codex'` for a session ingested from Codex's rollout journals by `server/lib/codex-usage.js` (see `codex_rollouts`) or created by a Codex hook (`tool: "codex"`, `server/lib/codex-hooks.js`) — a Codex session keeps the status its hooks give it while they are current (its newest event at or after its newest journal write minus two minutes); once they stop (removed, untrusted) the journal decides again. A codex row has no `transcript_path`, and, going by the journal, its `status` is `active` only while one of its journals was written in the last two minutes. `'antigravity'` for an agy CLI conversation, ingested by `server/lib/antigravity-usage.js` (see `antigravity_conversations`) or created by an Antigravity hook (`server/lib/antigravity-hooks.js`); no `transcript_path`, and `active` until it has been quiet — no write to its conversation file (or a non-empty `-wal`) or transcript, no hook event — for ten minutes when it has hook events, two minutes when it has none; it then settles to `completed`, its agents too. `'cursor'` for a Cursor agent transcript, ingested by `server/lib/cursor-usage.js` (see `cursor_transcripts`): no `transcript_path`, no `model` and no `token_usage` (the transcripts record none), `active` while its transcript was written in the last two minutes, else `completed`. The startup cleanup of stale `active` sessions and a workspace purge touch only the matching tool's rows (a purge takes the sessions of the workspace's own `tool`). The Workspace's attach/resume list offers `'claude'` rows only |
|
||||
| `lane_id` | INTEGER | YES | The workspace that OWNS the session — the only sessions a workspace lists: the lane whose dashboard pane held it, stamped by `bindRunSession` (`server/routes/hooks.js`) from the hook's `run_id` — first writer wins, a nested `claude -p` excluded — or, for a Codex/Antigravity session, by `claimToolSessions` (`server/lib/dashboard-runs.js`) from the same-tool run that was live in its folder when it started. NULL for a session no workspace started (a terminal, an import, a tool with no hooks). No FK: removing a workspace leaves its sessions. Backfilled once from `dashboard_runs`. Indexed by `idx_sessions_lane` |
|
||||
| `archived_at` | TEXT | YES | When the user archived the session out of its workspace's lists (`PATCH /api/sessions/:id {archived}`). Hides only — nothing is deleted |
|
||||
|
||||
@@ -474,7 +476,10 @@ CREATE TABLE remote_sources (
|
||||
last_sync_at TEXT,
|
||||
last_sync_counts TEXT,
|
||||
created_at TEXT DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now')),
|
||||
updated_at TEXT DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now'))
|
||||
updated_at TEXT DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now')),
|
||||
-- added by migration (additive):
|
||||
sync_codex INTEGER NOT NULL DEFAULT 0,
|
||||
remote_codex_home TEXT
|
||||
);
|
||||
```
|
||||
|
||||
@@ -489,10 +494,12 @@ CREATE TABLE remote_sources (
|
||||
| `identity_file` | TEXT | YES | Optional private-key path passed to ssh (`-i`); NULL to omit |
|
||||
| `remote_home` | TEXT | YES | Optional remote Claude home to read transcripts from; NULL defaults to remote `~/.claude` |
|
||||
| `enabled` | INTEGER | NO | `1` = eligible for scheduled/manual syncs, `0` = disabled (default `1`) |
|
||||
| `sync_codex` | INTEGER | NO | `1` = a sync also mirrors the machine's Codex `sessions/` and reads it through `server/lib/codex-usage.js` into `tool = 'codex'` sessions tagged with this source (default `0`) |
|
||||
| `remote_codex_home` | TEXT | YES | Optional remote CODEX_HOME; NULL = `~/.codex` (`wsl:~/.codex` when `remote_home` is a `wsl:` path) |
|
||||
| `status` | TEXT | NO | Last sync status: `idle`, `syncing`, `ok`, or `error` (CHECK-constrained) |
|
||||
| `last_error` | TEXT | YES | Error message from the last failed sync/test, or NULL |
|
||||
| `last_sync_at` | TEXT | YES | ISO 8601 timestamp of the last successful sync, or NULL |
|
||||
| `last_sync_counts` | TEXT | YES | JSON blob of the last sync's counters (imported/skipped/backfilled/errors/sessions_seen/sessions_tagged), or NULL |
|
||||
| `last_sync_counts` | TEXT | YES | JSON blob of the last sync's counters (imported/skipped/backfilled/errors/sessions_seen/sessions_tagged, plus codex_files/codex_sessions with Codex on), or NULL |
|
||||
| `created_at` | TEXT | YES | ISO 8601 creation timestamp |
|
||||
| `updated_at` | TEXT | YES | ISO 8601 timestamp of the last edit |
|
||||
|
||||
@@ -508,7 +515,7 @@ The same buckets as `token_usage` (session, model, speed, inference_geo, service
|
||||
|
||||
### codex_rollouts
|
||||
|
||||
How far `server/lib/codex-usage.js` has read each Codex rollout journal (`$CODEX_HOME/sessions/YYYY/MM/DD/rollout-<ts>-<thread>.jsonl`). A file whose `size` and `mtime_ms` are unchanged is not read again; a grown one is read from `offset` to its last newline, at most 16 MB per pass (`state.more` marks a file with more to read); a shrunken one from 0. There is one row per `thread_id`: the same journal met under a new path (CODEX_HOME moved, or reached through a symlink) replaces its old row instead of being counted twice. Because it lives in the DB, a Codex session the user deleted stays gone until Codex writes to that journal again, while a fresh database re-imports every journal. `POST /api/settings/clear-data` empties this table (and `antigravity_conversations`) with the sessions, so a cleared session is imported again whole instead of never, or half on its next write.
|
||||
How far `server/lib/codex-usage.js` has read each Codex rollout journal (`$CODEX_HOME/sessions/YYYY/MM/DD/rollout-<ts>-<thread>.jsonl`). A file whose `size` and `mtime_ms` are unchanged is not read again; a grown one is read from `offset` to its last newline, at most 16 MB per pass (`state.more` marks a file with more to read); a shrunken one from 0. There is one row per `thread_id`: the same journal met under a new path (CODEX_HOME moved, or reached through a symlink) replaces its old row instead of being counted twice. Because it lives in the DB, a Codex session the user deleted stays gone until Codex writes to that journal again, while a fresh database re-imports every journal. `POST /api/settings/clear-data` empties this table (and `antigravity_conversations`, `cursor_transcripts`) with the sessions, so a cleared session is imported again whole instead of never, or half on its next write.
|
||||
|
||||
```sql
|
||||
CREATE TABLE codex_rollouts (
|
||||
@@ -540,6 +547,68 @@ CREATE TABLE antigravity_conversations (
|
||||
);
|
||||
```
|
||||
|
||||
### cursor_transcripts
|
||||
|
||||
Which Cursor agent transcripts `server/lib/cursor-usage.js` has read (`~/.cursor/projects/<project slug>/agent-transcripts/<id>/<id>.jsonl`), at what mtime. An unchanged file is not read again (only its `active` session settled once quiet); a changed one rewrites its session row. As with `codex_rollouts`, a session the user deleted stays gone until Cursor writes to that transcript again. Additive: `CREATE TABLE IF NOT EXISTS`.
|
||||
|
||||
```sql
|
||||
CREATE TABLE cursor_transcripts (
|
||||
path TEXT PRIMARY KEY, -- absolute transcript path
|
||||
session_id TEXT NOT NULL, -- the file name's id
|
||||
mtime_ms INTEGER NOT NULL DEFAULT 0,
|
||||
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now'))
|
||||
);
|
||||
```
|
||||
|
||||
### session_snapshots
|
||||
|
||||
Frozen copies of a session (`server/routes/snapshots.js`, `POST /api/snapshots`). `payload` is one JSON document — `{version, captured_at, redacted: {file_paths}, session, agents, events, stats, cost}` — read back only by `token`. A trigger refuses every `UPDATE`; `session_id` has no foreign key, so a snapshot outlives its session and survives Clear Data (deleted only by `DELETE /api/snapshots/:id`). Additive: `CREATE TABLE IF NOT EXISTS`.
|
||||
|
||||
```sql
|
||||
CREATE TABLE session_snapshots (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
token TEXT NOT NULL UNIQUE, -- crypto.randomBytes(24) base64url
|
||||
session_id TEXT NOT NULL,
|
||||
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
|
||||
payload TEXT NOT NULL
|
||||
);
|
||||
CREATE TRIGGER session_snapshots_immutable BEFORE UPDATE ON session_snapshots
|
||||
BEGIN SELECT RAISE(ABORT, 'session snapshots are immutable'); END;
|
||||
```
|
||||
|
||||
### report_definitions / report_runs
|
||||
|
||||
Scheduled reports (`server/lib/reports.js`, `/api/reports`). A definition names the window (`window_days` full local days before the run day), the schedule (`frequency` daily | weekly, `weekday` 0 = Sunday, local `hour`, at `tz_offset` = the browser's `getTimezoneOffset()`), the `formats` (JSON array of `html` / `json`) and `enabled`. `last_slot` is the scheduled instant last claimed — the scheduler claims a slot with one conditional `UPDATE … WHERE last_slot IS NULL OR last_slot < ?`, so of two dashboards on one database only one runs it. Each run is a `report_runs` row; its files are `<data dir>/reports/<report_id>/<stamp>.<format>` (`files` lists the formats written). Runs cascade with their definition. Both tables survive Clear Data. Additive: `CREATE TABLE IF NOT EXISTS`.
|
||||
|
||||
```sql
|
||||
CREATE TABLE report_definitions (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
name TEXT NOT NULL,
|
||||
window_days INTEGER NOT NULL DEFAULT 7,
|
||||
frequency TEXT NOT NULL DEFAULT 'weekly' CHECK(frequency IN ('daily','weekly')),
|
||||
hour INTEGER NOT NULL DEFAULT 8,
|
||||
weekday INTEGER NOT NULL DEFAULT 1,
|
||||
tz_offset INTEGER NOT NULL DEFAULT 0,
|
||||
formats TEXT NOT NULL DEFAULT '["html"]',
|
||||
enabled INTEGER NOT NULL DEFAULT 1,
|
||||
last_slot TEXT, -- ISO instant of the last claimed slot
|
||||
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
|
||||
);
|
||||
CREATE TABLE report_runs (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
report_id INTEGER NOT NULL REFERENCES report_definitions(id) ON DELETE CASCADE,
|
||||
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
|
||||
trigger TEXT NOT NULL, -- schedule | manual
|
||||
status TEXT NOT NULL, -- ok | error
|
||||
window_from TEXT, -- local YYYY-MM-DD, inclusive
|
||||
window_to TEXT,
|
||||
stamp TEXT, -- file name stem, made by the server
|
||||
files TEXT NOT NULL DEFAULT '[]',
|
||||
error TEXT
|
||||
);
|
||||
CREATE INDEX idx_report_runs_report ON report_runs(report_id, id);
|
||||
```
|
||||
|
||||
### lane_sources (and `lanes.instructions`)
|
||||
|
||||
A workspace's own context, as on its home page (`server/lib/lane-sources.js`). `lanes.instructions` (TEXT, nullable, ≤ 16 000 characters) is appended to every new pane's system prompt; `lane_sources` holds its extra **folders** (the user's directories, never deleted by AgentHub) and uploaded context **files** (AgentHub's own copies under `<data dir>/workspace-context/<lane id>/`, deleted with the row and with the workspace). Both are read at spawn from these rows only (`launchExtras`), never from a request. Both are additive migrations; the rows cascade when the lane row is deleted.
|
||||
|
||||
Reference in New Issue
Block a user