Smartgift-AI/Claude-Code-Monitor
3
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

feat: Claude Code Monitor — lanes, pipelines and a merged workspace

Browse Source 
Internal SmartGift build of a Claude Code monitoring dashboard.

Lanes: a durable unit of parallel agent work, one per working directory,
tracked across session restarts. Managed lanes are git worktrees the
dashboard provisions and can reset or remove behind a three-check destroy
guard and a counted preflight; adopted lanes are directories you already
own and are never destroyable.

Pipelines: a lane moves through pipeline stages. A stage the agent declares
with evidence renders green; a stage inferred from the tool-event stream
renders dashed amber and never counts as done. Detection is forward-only
within a 30-minute window, and never writes the declared stage.

Workspace: one page at /run with a lane grid, the selected lane's pipeline,
and a full Claude console behind a disclosure.
This commit is contained in:
nntrivi2001
2026-07-29 17:07:45 +07:00
commit 57dc91585d
783 changed files with 221743 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/API.md
+1626
View File
File diff suppressed because it is too large Load Diff
docs/CLI.md
+270
View File
@@ -0,0 +1,270 @@
# `ccam` CLI Reference
The complete guide to `ccam`, the Claude Code Agent Monitor command-line interface — the full dashboard feature surface, in your terminal.
---
## Table of Contents
- [Overview](#overview)
- [Installation & Linking](#installation--linking)
- [Server Discovery](#server-discovery)
- [Commands](#commands)
- [Server Lifecycle](#server-lifecycle)
- [Interactive REPL](#interactive-repl)
- [Offline Mode](#offline-mode)
- [Monitoring](#monitoring)
- [Data Browsing](#data-browsing)
- [Insights](#insights)
- [Alerts & Webhooks](#alerts--webhooks)
- [Pricing](#pricing)
- [Import](#import)
- [Remote Sources](#remote-sources)
- [Administration](#administration)
- [Safety Model](#safety-model)
- [Output & Scripting](#output--scripting)
- [Troubleshooting](#troubleshooting)
---
## Overview
`ccam` (`bin/ccam.js`) is a **dependency-free** Node.js CLI over the local dashboard API. Everything the web app can do — monitoring, browsing, analytics, alerting, pricing, imports, administration — is available as a terminal command. It ships with the repository, requires no additional install step beyond the normal project setup, and talks only to your local dashboard server.
```
ccam <command> [options]
```
```mermaid
flowchart LR
U["Terminal\nccam <command>"] --> CLI["bin/ccam.js\n(zero dependencies)"]
CLI -->|"env override"| ENV["CLAUDE_DASHBOARD_PORT /\nDASHBOARD_PORT"]
CLI -->|"else discovery"| REG["~/.claude/.agent-dashboard.json\n(PID-liveness-checked)"]
CLI -->|"else fallback"| DEF["http://127.0.0.1:4820"]
ENV --> API["Dashboard REST API"]
REG --> API
DEF --> API
API --> OUT["Box-drawn tables / status icons / bar charts /\nplain text when piped"]
```
## Installation & Linking
`npm run setup` ends with a fail-soft `npm link` (the `link-cli` script), so after a normal local setup `ccam` is on your PATH from any directory:
```bash
git clone https://git.smartgift.io.vn/Smartgift-AI/Claude-Code-Monitor.git
cd Claude-Code-Agent-Monitor
npm run setup # installs deps AND links ccam globally
ccam help
```
If linking needed elevated permissions in your environment, setup still succeeds and prints a hint — run `npm link` once from the repo root yourself, or invoke the CLI directly with `node bin/ccam.js <command>`.
## Server Discovery
The CLI finds your running dashboard the same way the Claude Code hook handler does:
| Priority | Source | Notes |
| -------- | ------ | ----- |
| 1 | `CLAUDE_DASHBOARD_PORT` / `DASHBOARD_PORT` env vars | Explicit override wins |
| 2 | `~/.claude/.agent-dashboard.json` | Written by every running dashboard (`{port, pid, startedAt}` entries); stale entries are skipped via a PID liveness check |
| 3 | `http://127.0.0.1:4820` | Default port fallback |
If no server answers, every API-backed command exits `1` with the `○ Dashboard server is NOT running` indicator and the ways to start one (see [Server Lifecycle](#server-lifecycle)).
## Commands
### Server Lifecycle
The CLI talks to the local dashboard server — **API-backed commands require it to be running**. When it isn't, every such command prints a consistent indicator and exits `1`:
```
○ Dashboard server is NOT running (tried http://127.0.0.1:4820)
This command needs the server. Start it with one of:
ccam start # production server in the background
npm run dev # dev mode (hot reload), foreground
npm start # production mode, foreground
```
| Command | Description |
| ------- | ----------- |
| `ccam status` | At-a-glance up/down indicator (`●` running / `○` not running); exits `1` when down |
| `ccam start [--port N]` | Start the production server **in the background** (detached; survives closing the terminal), wait up to 30 s for `/api/health`, print the URL + PID and the `kill <pid>` stop command. Logs append to `data/ccam-server.log`. No-ops with a pointer when a server is already up. Requires a built client (`npm run build` once) |
| `ccam repl` (aliases `shell`, `i`) | Open the **interactive shell** — see [Interactive REPL](#interactive-repl) |
### Interactive REPL
`ccam repl` (also `ccam shell` / `ccam i`) opens a persistent prompt where you type commands **without the `ccam` prefix** — ideal for a monitoring session where you run `sessions`, drill into a `session <id>`, check `kanban`, then `cost`, without re-typing `ccam` each time. On entry it prints a **CCAM word-mark welcome banner** with the version and live server status.
```
_____ _____ _____ _____
/\ \ /\ \ /\ \ /\ \
/::\ \ /::\ \ /::\ \ /::\____\
… (CCAM word-mark) …
Claude Code Agent Monitor · interactive shell · v1.3.0 ● 127.0.0.1:4820
Type commands without the 'ccam' prefix — e.g. sessions --limit 5
help all commands · help <cmd> details · Tab completes · ↑/↓ history · exit to quit
● ccam 127.0.0.1:4820 › sessions --limit 3
… table …
○ ccam offline › stats # prompt dot turns red when the server is down
```
- **Live status prompt** — a green `●` + resolved host when the server is up, a red `○` + `offline` when it isn't (probed with a short, cached health check).
- **Tab completion** for commands, subcommands (`alerts ack`, `pricing set`, …), and flags (`--limit`, `--status`, …).
- **Arrow-key history**, persisted across sessions to `data/.ccam_repl_history`.
- **Full command surface** — every command in this reference works inside the shell exactly as on the one-shot CLI (they are dispatched as child `ccam` processes).
- **Shell built-ins:**
| Built-in | Description |
| -------- | ----------- |
| `help` / `?` | Shell built-ins **plus the full grouped command catalog** |
| `help <command>` | Details (invocation + description) for one command |
| `commands` | Compact list of every command, grouped by category |
| `watch [seconds] <command …>` | Re-run a command on a timer (default 2 s), clearing the screen each tick, until `Ctrl+C` — a terminal live view (e.g. `watch 5 kanban`) |
| `history` | Recent command history |
| `banner` | Reprint the welcome banner |
| `clear` / `cls` | Clear the screen |
| `exit` / `quit` / `q` | Leave the shell (also `Ctrl+D`) |
- **Robust isolation** — each entered line runs as a short-lived child `ccam` process, so a non-zero exit, an offline refusal, or a blocking `tail` / `watch` (both stop on `Ctrl+C`) can **never** take the shell down with it. Offline reads and server-only refusals behave exactly as they do on the one-shot CLI.
- Works with piped input too (`printf 'stats\nexit\n' | ccam repl`) for scripting, running each line in order and exiting at EOF.
### Offline Mode
When the server is down, **read-only commands automatically fall back to reading `data/dashboard.db` directly** (SQLite; a safe second reader). Every offline run starts with a banner:
```
⚠ Offline mode — server not running; reading data/dashboard.db directly.
Data is as of the last capture — live capture and full features need the server: ccam start
```
| Works offline | Server required (with the printed reason) |
| ------------- | ----------------------------------------- |
| `sessions`, `session <id>`*, `agents`, `events`, `kanban`, `stats`, `pricing` (list), `alerts` (list), `rules`, `export`, `doctor` | `tail` (live capture), `analytics` / `workflows` / `runs` / `cost` (server-side aggregation & pricing math), `alerts ack`, `webhooks` (all), `pricing set/delete/reset`, `import`, `remote-sources` (all — SSH pull needs the server), `cleanup`, `clear-data`, `reinstall-hooks`, `update-check` (server-side git fetch), `info`, `health` |
\* `session <id>` shows everything except the cost line, which requires the server's pricing engine. Offline export payloads carry `"exported_offline": true`. Offline data is as of the last capture — with no server running, no hooks are being ingested either.
**Status correctness offline:** while the server is down its dead-session liveness reap isn't running, so the DB can hold `active`/`waiting` rows for sessions that have since exited. Offline output therefore runs the **same process-liveness probe** the server's watchdog uses and corrects the *displayed* status of any active session whose cwd has no running `claude` process (footnote: `※ N session(s) displayed as completed by the process-liveness probe`) — the database itself is never modified. Where the probe can't answer (Windows, containers), a `※ Statuses are as stored…` caveat is printed instead whenever active rows are shown.
### Monitoring
| Command | Description |
| ------- | ----------- |
| `ccam health` | One-line reachability check with the resolved URL and server timestamp |
| `ccam stats` | Totals (sessions, agents, events), today's event count, WS connections, and the sessions-by-status distribution |
| `ccam kanban` | The Kanban board as text: sessions grouped into Active / Waiting / Completed / Error / Abandoned and agents into Working / Waiting / Completed / Error, with current tools |
| `ccam tail [--session <id>]` | Live event feed — polls `/api/events` every 2 s and prints only new rows (the Activity Feed without a WebSocket client). `Ctrl+C` stops |
### Data Browsing
| Command | Description |
| ------- | ----------- |
| `ccam sessions [--status s] [--q text] [--limit n]` | Server-filtered session table: short ID, status, name, agent count, duration, model, relative last-update |
| `ccam session <id>` | Deep dive: metadata card, per-session cost, a parent→child **agent tree** (`├─`/`└─`) with live tools, and the most recent events |
| `ccam agents [--status s] [--session id] [--limit n]` | Agent table with type, current tool, and duration |
| `ccam events [--session id] [--limit n]` | Newest-first event log with type, tool, and summary |
### Insights
| Command | Description |
| ------- | ----------- |
| `ccam analytics` | Token totals (input / output / cache read / cache write), top tools by call count, agent-type distribution, average events per session |
| `ccam workflows [--session id]` | Workflow-intelligence stats (sessions analyzed, subagents, success rate, depth, compactions) and the top detected patterns; `--session` drills into one session |
| `ccam runs [--session id]` | Dynamic Workflow-tool runs: status, agent count, tokens, tool calls, duration |
| `ccam cost [--session <id>]` | Total estimated cost with a per-model bar-chart breakdown; `--session` scopes it to one session (mirrors `/api/pricing/cost/:sessionId`). Any billed **server-tool surcharges** (web search $/1k, code-execution container-time) are shown on a surcharges line. Models with usage but **no matching pricing rule** (priced at $0 and excluded from the total) are listed in a warning with their token volume and the `ccam pricing set` invocation that fixes it |
### Alerts & Webhooks
| Command | Description |
| ------- | ----------- |
| `ccam alerts [--unacked] [--limit n]` | Fired-alert feed with state, trigger time, rule, and message |
| `ccam alerts ack <id>` | Acknowledge one alert |
| `ccam alerts ack-all` | Acknowledge every unacknowledged alert |
| `ccam rules` | Alert rules with enabled state, type, and cooldown |
| `ccam webhooks` | Webhook targets (URLs masked server-side, secrets never returned) |
| `ccam webhooks test <id>` | Fire a synthetic test alert at a target and report the delivery result; exits non-zero on failure |
### Pricing
| Command | Description |
| ------- | ----------- |
| `ccam pricing` | All model pricing rules with per-mtok rates, including **Fast In/Out** and **Intro In/Out** columns for fast-mode premiums and time-limited promo pricing |
| `ccam pricing set <pattern> --input N --output N [--cache-read N] [--cache-write N] [--cache-write-1h N] [--name label]` | Create or update a rule (SQL `LIKE` pattern, e.g. `claude-opus-4-6%`) |
| `ccam pricing set <pattern> … [--fast-input N] [--fast-output N]` | Also set **fast-mode** premium rates on the rule |
| `ccam pricing set <pattern> … [--intro-input N] [--intro-output N] [--intro-cache-read N] [--intro-cache-write N] [--intro-cache-write-1h N] --intro-until YYYY-MM-DD` | Set a **time-limited introductory (promo) rate block**. The intro fields are only sent when an `--intro-*` flag is present, so a plain rate edit never clobbers an existing promo; a bare `--intro-until` (no date) clears it |
| `ccam pricing delete <pattern>` | Delete a rule |
| `ccam pricing reset` | Restore the default rate table |
### Import
| Command | Description |
| ------- | ----------- |
| `ccam import rescan` | Re-scan the default `~/.claude/projects` tree (idempotent; prints imported / backfilled / skipped / errors) |
| `ccam import path <dir>` | Recursively import every `.jsonl` under an absolute directory (`~` is expanded server-side) |
| `ccam import-data <file.json>` | Restore a full dashboard export produced by `ccam export` (or **Settings → Export data**). Idempotent and non-destructive — sessions already present are skipped whole, so it safely **consolidates several machines** into one dashboard. The file path is resolved to absolute and read server-side |
### Remote Sources
Manage the remote (SSH) machines this dashboard pulls Claude Code history from — the terminal equivalent of **Settings → Remote Data Sources**. Authentication defers entirely to your SSH stack (`~/.ssh/config`, ssh-agent, keys, known_hosts); **no secrets are passed or stored**. `remotes` is an alias for `remote-sources`.
| Command | Description |
| ------- | ----------- |
| `ccam remote-sources` (alias `remotes`) | List configured sources with id, auto-sync on/off, status, label, host, **session count**, and last-sync time, followed by a totals line (sources / auto-syncing / sessions collected) |
| `ccam remote-sources add --label <name> --host <user@host> [--port N] [--identity <path>] [--remote-home <path>] [--disabled]` | Add a source. `--host` is an ssh destination (`user@host`) or a `~/.ssh/config` alias; `--disabled` skips it in the background poller |
| `ccam remote-sources test <id>` | Probe SSH connectivity and check the remote `~/.claude/projects` exists; exits non-zero on failure |
| `ccam remote-sources sync [id]` | Pull history now — one source by id, or **all** sources when the id is omitted. Prints imported / tagged counts |
| `ccam remote-sources rm <id> [--purge]` | Remove a source (its imported sessions are detached back to `local` by default; `--purge` also **deletes** them) |
### Administration
| Command | Description |
| ------- | ----------- |
| `ccam doctor` | Diagnosis: API reachability, hook installation status + path, database path/size/row counts, server uptime and Node version, WS connections |
| `ccam info` | The raw `/api/settings/info` JSON (pipe it to `jq`) |
| `ccam export [file.json]` | Full JSON data export (sessions, agents, events, tokens, workflows, dashboard runs, alert rules, pricing) — defaults to a dated filename. Re-importable via `ccam import-data` |
| `ccam cleanup --hours N --days M` | Abandon active sessions idle for `N` hours and/or purge completed sessions older than `M` days |
| `ccam reinstall-hooks` | Rewrite the Claude Code hook entries in `~/.claude/settings.json` |
| `ccam update-check` | Ask the server whether the dashboard checkout is behind the canonical remote (branch- and fork-aware). Prints the behind-by count, a situation note for fork/feature-branch checkouts, and the **copy-paste update command** — the dashboard never restarts itself. Also refreshes the update banner in any open dashboard tab (same `update_status` broadcast) |
| `ccam clear-data --yes` | Delete **all** data (schema preserved). Refuses to run without `--yes` |
| `ccam open` | Open the dashboard in your default browser (`open` / `xdg-open` / `start`) |
| `ccam version` | Print the ccam version (also `--version` / `-v`) |
| `ccam help` | Full command reference (also shown with no arguments) |
## Safety Model
- **Read commands are always safe** — they only issue `GET`s.
- **Mutating commands** (`alerts ack`, `pricing set/delete/reset`, `import`, `cleanup`, `reinstall-hooks`) map 1:1 to explicit dashboard actions and run immediately, exactly like clicking the equivalent button.
- **The one destructive command, `clear-data`, refuses to run without `--yes`** and prints exactly what it would delete. There is no bulk-destructive behavior anywhere else.
## Output & Scripting
The CLI renders a full terminal UI while staying 100% script-friendly:
- **Box-drawn tables** with bold headers, right-aligned numeric columns, and terminal-width fitting — over-wide columns are clipped with an ellipsis so the frame never wraps mid-row.
- **Status icons + colors** everywhere a status appears: `● active` (green), `◐ working` (green), `○ waiting` (yellow), `✔ completed` (dim), `✖ error` (red), `◦ abandoned` (dim).
- **Inline bar charts** for the sessions-by-status distribution (`stats`), top tools and agent types (`analytics`), and the per-model cost breakdown (`cost`).
- **Real tree rendering** (`├─`/`└─` with continuation rails) for the agent hierarchy in `session <id>`, and status lanes with branch rows in `kanban`.
- Session tables include a relative **Updated** column (`4m ago`) so freshness is visible at a glance; event types are color-coded consistently across `events`, `tail`, and `session <id>`.
- `ccam start` animates a spinner on a TTY (dot-trail when piped).
Color rules (informal CLI conventions):
| Condition | Effect |
| --------- | ------ |
| stdout is a TTY | Colors **on** |
| Output piped / redirected | Colors **off** automatically — `ccam sessions \| grep error` and `ccam info \| jq .db.counts` see plain text |
| `NO_COLOR=1` env or `--no-color` anywhere on the command line | Colors **off** |
| `FORCE_COLOR=1` or `CCAM_COLOR=1` | Colors **on** even when piped (useful under `watch`/CI) |
- `ccam version` (also `--version` / `-v`) prints the package version.
- Exit codes: `0` success, `1` for unreachable server, API errors, usage errors, unknown commands, or a failed `webhooks test` — safe to use in scripts and CI.
## Troubleshooting
| Symptom | Fix |
| ------- | --- |
| `○ Dashboard server is NOT running` | Start it: `ccam start` (background), `npm run dev`, or `npm start`. If it runs on a custom port, set `DASHBOARD_PORT` or rely on the discovery file |
| `ccam: command not found` | Run `npm link` from the repo root (setup's fail-soft link may have skipped on permissions), or use `node bin/ccam.js …` |
| Wrong server answers (multiple dashboards) | Set `CLAUDE_DASHBOARD_PORT` explicitly — env overrides always beat discovery |
| `tail` shows nothing | Events only flow while hooks are installed and a Claude Code session is active — check `ccam doctor` |
docs/DATABASE.md
+864
View File
@@ -0,0 +1,864 @@
# Database Schema Reference
Comprehensive database schema documentation for Agent Dashboard SQLite database.
---
## Table of Contents
- [Overview](#overview)
- [Schema Diagram](#schema-diagram)
- [Table Definitions](#table-definitions)
- [Indexes](#indexes)
- [Migrations](#migrations)
- [Query Patterns](#query-patterns)
- [Performance Optimization](#performance-optimization)
- [Data Integrity](#data-integrity)
- [Backup Strategies](#backup-strategies)
---
## Overview
Agent Dashboard uses **SQLite 3** as its primary data store with the following characteristics:
- **File-based** - Single database file, portable across systems
- **Embedded** - No separate server process required
- **ACID compliant** - Transactions ensure data integrity
- **WAL mode** - Write-Ahead Logging for better concurrency
- **Prepared statements** - Prevent SQL injection, optimize performance
```mermaid
graph TB
subgraph "Database File"
DB[(dashboard.db)]
end
subgraph "Tables"
Sessions[sessions]
Agents[agents]
Tools[tool_executions]
Notifs[notifications]
Pricing[pricing_rules]
Remote[remote_sources]
end
subgraph "Indexes"
Idx1[session_id, status, updated_at]
Idx2[agent_id, session_id, status]
Idx3[agent_id, created_at]
end
DB --> Sessions
DB --> Agents
DB --> Tools
DB --> Notifs
DB --> Pricing
DB --> Remote
Sessions --> Idx1
Agents --> Idx2
Tools --> Idx3
style DB fill:#003B57,color:#fff
```
**Database Location:**
- **Canonical (default):** `~/.claude/agent-dashboard/dashboard.db` — shared by `npm start`, `npm run dev`, Docker (bind mount), and the desktop app when it uses the same data dir
- **Override:** set `DASHBOARD_DATA_DIR` (directory) or `DASHBOARD_DB_PATH` (file path) for tests or custom deployments
- **Legacy:** repo-local `./data/dashboard.db` is migrated into the canonical location on first launch (see `server/db.js`)
---
## Schema Diagram
### Entity-Relationship Diagram
```mermaid
erDiagram
sessions ||--o{ agents : "has many"
agents ||--o{ tool_executions : "has many"
sessions ||--o{ notifications : "has many"
remote_sources ||--o{ sessions : "tags (source)"
sessions {
integer id PK "Primary key"
text session_id UK "Unique session identifier"
text model "Raw model slug (e.g., claude-sonnet-4-5-20250514); UI displays via formatModelName()"
text status "active | completed"
real total_cost "Aggregated cost from all agents"
text source "'local' or a remote_sources.id"
text created_at "ISO8601 timestamp"
text updated_at "ISO8601 timestamp (bumped on every hook)"
}
agents {
integer id PK "Primary key"
text agent_id UK "Unique agent identifier"
text session_id FK "Foreign key to sessions"
text agent_type "explore, task, general-purpose, etc."
text status "running | completed | failed"
text current_tool "Currently executing tool (or NULL)"
integer input_tokens "Cumulative input tokens"
integer output_tokens "Cumulative output tokens"
real cost "Calculated cost for this agent"
text created_at "ISO8601 timestamp"
text updated_at "ISO8601 timestamp"
}
tool_executions {
integer id PK "Primary key"
text agent_id FK "Foreign key to agents"
text tool_name "bash, view, edit, grep, etc."
integer duration_ms "Execution time in milliseconds"
integer success "1 = success, 0 = failure"
text error_message "NULL if success, error details if failed"
text created_at "ISO8601 timestamp"
}
notifications {
integer id PK "Primary key"
text session_id FK "Foreign key to sessions"
text notification_type "backgroundTaskComplete, etc."
text message "Notification message"
text created_at "ISO8601 timestamp"
}
pricing_rules {
integer id PK "Primary key"
text pattern UK "Model pattern (e.g., claude-sonnet-4)"
real input_cost_per_1m "Input cost per 1M tokens (USD)"
real output_cost_per_1m "Output cost per 1M tokens (USD)"
text created_at "ISO8601 timestamp"
}
remote_sources {
text id PK "Remote-source id (also used as sessions.source)"
text label "Human-readable name"
text host "SSH destination user@host or ~/.ssh/config alias"
integer ssh_port "Optional SSH port (NULL = SSH default)"
text identity_file "Optional private-key path (NULL = SSH default)"
text remote_home "Optional remote Claude home (NULL = remote ~/.claude)"
integer enabled "1 = eligible for sync, 0 = disabled"
text status "idle | syncing | ok | error"
text last_error "Last failure message, or NULL"
text last_sync_at "ISO8601 timestamp of last successful sync, or NULL"
text last_sync_counts "JSON blob of last sync counters, or NULL"
text created_at "ISO8601 timestamp"
text updated_at "ISO8601 timestamp"
}
```
### Relationship Cardinality
```mermaid
graph LR
Session[Session<br/>1] -->|1:N| Agents[Agents<br/>N]
Session -->|1:N| Notifications[Notifications<br/>N]
Agents -->|1:N| Tools[Tool Executions<br/>N]
style Session fill:#3B82F6
style Agents fill:#10B981
style Tools fill:#F59E0B
```
---
## Table Definitions
### sessions
Tracks Claude Code sessions (one per CLI invocation or background task). Schema mirrors `server/db.js`.
> **Cursor (informational):** Rows imported from `~/.claude` JSONL transcripts may also represent **Cursor** agent sessions — Cursor happens to use the same on-disk layout as Claude Code. The schema does not record which app created a session.
```sql
CREATE TABLE sessions (
id TEXT PRIMARY KEY, -- UUID from Claude Code
name TEXT,
status TEXT NOT NULL DEFAULT 'active'
CHECK (status IN ('active','completed','error','abandoned')),
cwd TEXT,
model TEXT,
started_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now')),
ended_at TEXT,
metadata TEXT,
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now')),
awaiting_input_since TEXT, -- NULL unless Waiting
awaiting_reason TEXT, -- notification|stop|session_start|interrupted, or NULL
transcript_path TEXT, -- absolute path to JSONL transcript
source TEXT NOT NULL DEFAULT 'local' -- data source: 'local' or a remote_sources.id
);
```
**Columns:**
| Column | Type | Nullable | Description |
|--------|------|----------|-------------|
| `id` | TEXT | NO | Session UUID (assigned by Claude Code) |
| `name` | TEXT | YES | Human-readable label. Synced from the transcript title by `routes/hooks.js` (and the 15 s watchdog) on every event: the `custom-title` line (`/rename`, `claude -n`, picker `Ctrl+R`) always wins, otherwise the auto-generated `ai-title` fills a placeholder/auto name, otherwise the session's first user prompt (60-char label) fills it. Falls back to `Session <id8>` |
| `status` | TEXT | NO | `active`, `completed`, `error`, or `abandoned` (CHECK-constrained). Besides the `SessionEnd` hook, the 15 s watchdog's **liveness reap** also lands `active` → `completed` when no running `claude` process has the session's `cwd` (a `SessionEnd` lost while the dashboard was down); gated by `DASHBOARD_LIVENESS_IDLE_SECONDS`, disabled via `DASHBOARD_LIVENESS_PROBE=0`. Sessions with a non-`local` `source` (Remote Data Sources) are exempt from the reap and both stale sweeps — their status is reconciled from the SSH mirror by `remote-sync.js` instead |
| `cwd` | TEXT | YES | Working directory the CLI was launched from |
| `model` | TEXT | YES | Claude model ID (e.g. `claude-opus-4-7`) |
| `started_at` | TEXT | NO | ISO 8601 timestamp |
| `ended_at` | TEXT | YES | ISO 8601 timestamp on terminal transition |
| `metadata` | TEXT | YES | JSON blob for extras (turn duration totals, thinking blocks, …) |
| `updated_at` | TEXT | NO | Bumped on every event for staleness detection |
| `awaiting_input_since` | TEXT | YES | ISO 8601 stamp set when the session is **Waiting** (Stop, SessionStart with source `startup`/`resume`/`clear`, permission Notification, or watchdog user-interrupt/Esc recovery). NULL otherwise. A SessionStart with source `compact` (auto-compaction fires mid-turn while Claude is working) leaves this column untouched, so a genuinely-active session is not mislabeled Waiting |
| `awaiting_reason` | TEXT | YES | Why the row is waiting: `notification`, `stop`, `session_start`, or `interrupted`. Set/cleared in lock-step with `awaiting_input_since` (SessionStart→`session_start`, Stop→`stop`, permission/input Notification→`notification`, watchdog/Esc recovery→`interrupted`). NULL otherwise. Exception: a `compact`-source SessionStart preserves the existing value (neither stamps `session_start` nor clears it) |
| `transcript_path` | TEXT | YES | Absolute path to the session's JSONL transcript. Written by `routes/hooks.js` on the first event that carries it (subsequent events no-op via a SQL guard) and read by the periodic compaction sweep — so the sweep touches only active session rows instead of scanning the entire `events` table for `json_extract(data,'$.transcript_path')`. Backfilled once from `events` by the `db.js` migration |
| `source` | TEXT | NO | Data source this session was captured from. `'local'` for this machine's own Claude Code history (the default); otherwise the `remote_sources.id` of the remote SSH machine it was pulled from. Powers the `sources` query filter on `/api/sessions`, `/api/events`, `/api/agents`, `/api/stats`, and `/api/analytics`, and the `sources` facet on `/api/sessions/facets`. Indexed by `idx_sessions_source` |
**Constraints:**
- `status` must be one of the four enum values
- `awaiting_input_since` is ignored on non-`active` sessions for UI bucketing
**Lifecycle:**
```mermaid
stateDiagram-v2
[*] --> waiting: SessionStart startup/resume/clear (status=active + awaiting_input_since)
active --> active: SessionStart compact (mid-turn — state preserved)
waiting --> active: UserPromptSubmit / PreToolUse / PostToolUse
active --> waiting: Stop (non-error) / Permission Notification
active --> waiting: Esc cancel (watchdog marker or idle timeout)
active --> error: Stop (stop_reason=error)
waiting --> completed: SessionEnd
active --> completed: SessionEnd
waiting --> abandoned: Stale > DASHBOARD_STALE_MINUTES
active --> abandoned: Stale > DASHBOARD_STALE_MINUTES
completed --> active: Resumed
error --> active: Resumed
abandoned --> active: Resumed
completed --> [*]
error --> [*]
abandoned --> [*]
```
---
### agents
Tracks main agents and subagents within a session. Main agents have id `${session_id}-main`; subagents get a fresh UUID.
```sql
CREATE TABLE agents (
id TEXT PRIMARY KEY,
session_id TEXT NOT NULL,
name TEXT NOT NULL,
type TEXT NOT NULL DEFAULT 'main' CHECK (type IN ('main','subagent')),
subagent_type TEXT,
status TEXT NOT NULL DEFAULT 'idle'
CHECK (status IN ('idle','connected','working','completed','error')),
task TEXT,
current_tool TEXT,
started_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now')),
ended_at TEXT,
parent_agent_id TEXT,
metadata TEXT,
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now')),
awaiting_input_since TEXT, -- main-agent waiting flag
awaiting_reason TEXT, -- notification|stop|session_start|interrupted, or NULL
FOREIGN KEY (session_id) REFERENCES sessions(id) ON DELETE CASCADE,
FOREIGN KEY (parent_agent_id) REFERENCES agents(id) ON DELETE SET NULL
);
```
**Columns:**
| Column | Type | Nullable | Description |
|--------|------|----------|-------------|
| `id` | TEXT | NO | UUID (subagents) or `${session_id}-main` (main agent) |
| `session_id` | TEXT | NO | FK to `sessions.id`, cascades on delete |
| `name` | TEXT | NO | Display label (e.g. `Main Agent - {session name}` or subagent description) |
| `type` | TEXT | NO | `main` or `subagent` |
| `subagent_type` | TEXT | YES | `Explore`, `general-purpose`, `code-review`, `compaction`, … |
| `status` | TEXT | NO | `idle`, `connected`, `working`, `completed`, `error` (CHECK-constrained). The dashboard's **Waiting** badge is the UI overlay produced by `awaiting_input_since`; it is not a persisted status |
| `task` | TEXT | YES | Subagent prompt / brief |
| `current_tool` | TEXT | YES | Tool currently running (cleared on `PostToolUse`) |
| `parent_agent_id` | TEXT | YES | FK to the spawning agent for nested subagent trees (`ON DELETE SET NULL`). Set to the main agent at insert, then repointed to the true spawner by `reconcileSubagentParents` from each subagent transcript's Task tool result (`toolUseResult.agentId`), so subagents-of-subagents nest correctly instead of flattening under main |
| `metadata` | TEXT | YES | JSON blob for extras. For subagents it carries `model` (the subagent's own model, issue #185) and `tokens` — an array of per-agent token buckets parsed from the subagent's transcript. The agent-list endpoints price `tokens` at the current rates to attach a per-agent `cost` (so a subagent card shows its OWN cost, not the session total). Empty `[]` means the subagent did no billable work; absent means its transcript wasn't available to parse |
| `awaiting_input_since` | TEXT | YES | Mirrors the parent session's flag for the main agent. NULL on subagents |
| `awaiting_reason` | TEXT | YES | Why the row is waiting: `notification`, `stop`, `session_start`, or `interrupted`. Set/cleared in lock-step with `awaiting_input_since`; explains why the main agent is waiting. NULL on subagents |
**Lifecycle:**
```mermaid
stateDiagram-v2
[*] --> Running: Agent created (SessionStart/PreToolUse)
Running --> Running: PreToolUse (set current_tool)
Running --> Running: PostToolUse (increment tokens, cost)
Running --> Completed: Stop/SubagentStop hook
Running --> Failed: Error during processing
Completed --> [*]
Failed --> [*]
```
**current_tool Behavior:**
- Set to tool name on `PreToolUse` hook (e.g., `"bash"`, `"view"`)
- Cleared to `NULL` on `PostToolUse` hook
- Used to show real-time tool execution in UI
---
### tool_executions
Records each tool call made by agents.
```sql
CREATE TABLE tool_executions (
id INTEGER PRIMARY KEY AUTOINCREMENT,
agent_id TEXT NOT NULL,
tool_name TEXT NOT NULL,
duration_ms INTEGER,
success INTEGER DEFAULT 1,
error_message TEXT,
created_at TEXT DEFAULT (datetime('now')),
FOREIGN KEY (agent_id) REFERENCES agents(agent_id)
);
```
**Columns:**
| Column | Type | Nullable | Description |
|--------|------|----------|-------------|
| `id` | INTEGER | NO | Auto-increment primary key |
| `agent_id` | TEXT | NO | Foreign key to `agents.agent_id` |
| `tool_name` | TEXT | NO | Tool name (`bash`, `view`, `edit`, `grep`, etc.) |
| `duration_ms` | INTEGER | YES | Execution time in milliseconds |
| `success` | INTEGER | NO | 1 = success, 0 = failure |
| `error_message` | TEXT | YES | NULL if success, error details if failed |
| `created_at` | TEXT | NO | ISO8601 timestamp of execution |
**Common Tool Names:**
- `bash` - Shell command execution
- `view` - File/directory viewing
- `edit` - File editing
- `grep` - Code search
- `glob` - File pattern matching
- `task` - Sub-agent invocation
- `sql` - SQLite query execution
**Duration Distribution:**
```mermaid
graph TB
Tools[Tool Executions] --> Fast[Fast<br/>&lt; 100ms<br/>view, grep]
Tools --> Medium[Medium<br/>100ms - 1s<br/>edit, bash]
Tools --> Slow[Slow<br/>&gt; 1s<br/>task, build commands]
style Fast fill:#10B981
style Medium fill:#F59E0B
style Slow fill:#EF4444
```
---
### notifications
Stores system notifications from Claude Code.
```sql
CREATE TABLE notifications (
id INTEGER PRIMARY KEY AUTOINCREMENT,
session_id TEXT NOT NULL,
notification_type TEXT NOT NULL,
message TEXT,
created_at TEXT DEFAULT (datetime('now')),
FOREIGN KEY (session_id) REFERENCES sessions(session_id)
);
```
**Columns:**
| Column | Type | Nullable | Description |
|--------|------|----------|-------------|
| `id` | INTEGER | NO | Auto-increment primary key |
| `session_id` | TEXT | NO | Foreign key to `sessions.session_id` |
| `notification_type` | TEXT | NO | Type of notification |
| `message` | TEXT | YES | Notification message content |
| `created_at` | TEXT | NO | ISO8601 timestamp |
**Common Notification Types:**
- `backgroundTaskComplete` - Background agent finished
- `errorOccurred` - Error during execution
- `systemMessage` - General system message
---
### model_pricing
Per-model pricing rules for cost calculation, keyed by `model_pattern` (a SQL-style glob; `%` matches any characters). Rates are per **million** tokens (USD).
```sql
CREATE TABLE model_pricing (
model_pattern TEXT PRIMARY KEY,
display_name TEXT NOT NULL,
input_per_mtok REAL NOT NULL DEFAULT 0,
output_per_mtok REAL NOT NULL DEFAULT 0,
cache_read_per_mtok REAL NOT NULL DEFAULT 0,
cache_write_per_mtok REAL NOT NULL DEFAULT 0,
cache_write_1h_per_mtok REAL NOT NULL DEFAULT 0, -- 1h-ephemeral cache-write tier
fast_input_per_mtok REAL NOT NULL DEFAULT 0, -- fast-mode premium rates
fast_output_per_mtok REAL NOT NULL DEFAULT 0,
-- Time-limited introductory (promo) rates. When intro_until is set, usage on
-- or before that date (YYYY-MM-DD) is priced at the intro_* rates and usage
-- after it at the standard rates. All 0 / NULL = no promo.
intro_input_per_mtok REAL NOT NULL DEFAULT 0,
intro_output_per_mtok REAL NOT NULL DEFAULT 0,
intro_cache_read_per_mtok REAL NOT NULL DEFAULT 0,
intro_cache_write_per_mtok REAL NOT NULL DEFAULT 0,
intro_cache_write_1h_per_mtok REAL NOT NULL DEFAULT 0,
intro_until TEXT, -- promo cutoff YYYY-MM-DD, or NULL
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now'))
);
```
**Columns (highlights):**
| Column | Type | Nullable | Description |
|--------|------|----------|-------------|
| `model_pattern` | TEXT | NO | Primary key. SQL-style glob (e.g. `claude-opus-4-7%`, `claude-%-haiku`). Rules are matched longest-pattern-first |
| `display_name` | TEXT | NO | Human-readable model name shown in Settings |
| `input_per_mtok` / `output_per_mtok` | REAL | NO | Standard input / output rate per 1M tokens |
| `cache_read_per_mtok` / `cache_write_per_mtok` / `cache_write_1h_per_mtok` | REAL | NO | Cache read + 5m/1h cache-write rates |
| `fast_input_per_mtok` / `fast_output_per_mtok` | REAL | NO | Fast-mode premium rates (0 = no premium) |
| `intro_*_per_mtok` | REAL | NO | Introductory (promo) rates, mirroring the standard fields |
| `intro_until` | TEXT | YES | Promo cutoff `YYYY-MM-DD`. Usage on/before it uses the intro rates; NULL = no promo. Editable per-rule in Settings |
| `updated_at` | TEXT | NO | ISO8601 timestamp of the last edit |
Standard rates and intro rates are edited independently: the pricing update path writes intro columns only when the caller sends intro fields, so a standard-rate edit never disturbs a promo (and vice versa). Clearing `intro_until` also zeroes the intro rates.
**Example default rule (Claude Sonnet 5, with its launch promo):**
| Pattern | Input | Output | Intro Input | Intro Output | Intro Until |
|---------|-------|--------|-------------|--------------|-------------|
| `claude-sonnet-5%` | $3.00 | $15.00 | $2.00 | $10.00 | `2026-08-31` |
---
### remote_sources
Config for remote SSH machines the dashboard pulls Claude Code history from, so a single dashboard can consolidate sessions from several machines. **No secrets are stored** — SSH authentication defers entirely to the host's SSH stack (ssh-agent, `~/.ssh/config`, key files). Each row's `id` is used as the `source` value on every session imported from that machine (see `sessions.source`).
```sql
CREATE TABLE remote_sources (
id TEXT PRIMARY KEY,
label TEXT NOT NULL,
host TEXT NOT NULL,
ssh_port INTEGER,
identity_file TEXT,
remote_home TEXT,
enabled INTEGER NOT NULL DEFAULT 1,
status TEXT NOT NULL DEFAULT 'idle'
CHECK (status IN ('idle','syncing','ok','error')),
last_error TEXT,
last_sync_at TEXT,
last_sync_counts TEXT,
created_at TEXT DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now')),
updated_at TEXT DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now'))
);
```
**Columns:**
| Column | Type | Nullable | Description |
|--------|------|----------|-------------|
| `id` | TEXT | NO | Primary key. Also used as `sessions.source` for sessions pulled from this machine |
| `label` | TEXT | NO | Human-readable name shown in the UI |
| `host` | TEXT | NO | SSH destination (`user@host`) or a `~/.ssh/config` alias |
| `ssh_port` | INTEGER | YES | Optional SSH port; NULL defers to the SSH default / `~/.ssh/config` |
| `identity_file` | TEXT | YES | Optional private-key path passed to ssh (`-i`); NULL to omit |
| `remote_home` | TEXT | YES | Optional remote Claude home to read transcripts from; NULL defaults to remote `~/.claude` |
| `enabled` | INTEGER | NO | `1` = eligible for scheduled/manual syncs, `0` = disabled (default `1`) |
| `status` | TEXT | NO | Last sync status: `idle`, `syncing`, `ok`, or `error` (CHECK-constrained) |
| `last_error` | TEXT | YES | Error message from the last failed sync/test, or NULL |
| `last_sync_at` | TEXT | YES | ISO 8601 timestamp of the last successful sync, or NULL |
| `last_sync_counts` | TEXT | YES | JSON blob of the last sync's counters (imported/skipped/backfilled/errors/sessions_seen/sessions_tagged), or NULL |
| `created_at` | TEXT | YES | ISO 8601 creation timestamp |
| `updated_at` | TEXT | YES | ISO 8601 timestamp of the last edit |
Managed through the `/api/remote-sources/*` routes; sync/status changes are broadcast over the WebSocket as `remote_source.status` and, on success, `remote_data.updated` plus per-session `session_created` / `session_updated`. See [docs/API.md → Remote Data Sources](./API.md#remote-data-sources).
---
## Indexes
### sessions Indexes
```sql
CREATE INDEX idx_sessions_session_id ON sessions(session_id);
CREATE INDEX idx_sessions_status ON sessions(status);
CREATE INDEX idx_sessions_updated_at ON sessions(updated_at DESC);
CREATE INDEX idx_sessions_source ON sessions(source); -- powers the `sources` query filter
-- Partial index covering only the rows the periodic compaction sweep reads:
-- active sessions with a known transcript_path. Writes to other sessions skip
-- the index entirely, so the maintenance cost stays bounded by the small set
-- of live sessions.
CREATE INDEX idx_sessions_active_tp
ON sessions(status, transcript_path)
WHERE status='active' AND transcript_path IS NOT NULL;
```
**Query Patterns:**
- `SELECT * FROM sessions WHERE session_id = ?` - Primary key lookup
- `SELECT * FROM sessions WHERE status = 'active'` - Filter by status
- `SELECT * FROM sessions WHERE source IN ('local', ?)` - Filter by data source (covered by `idx_sessions_source`)
- `SELECT * FROM sessions ORDER BY updated_at DESC LIMIT 50` - Recent sessions
- `SELECT id, transcript_path FROM sessions WHERE status='active' AND transcript_path IS NOT NULL ORDER BY updated_at DESC` — periodic compaction sweep (covered by the partial index above)
### agents Indexes
```sql
CREATE INDEX idx_agents_agent_id ON agents(agent_id);
CREATE INDEX idx_agents_session_id ON agents(session_id);
CREATE INDEX idx_agents_status ON agents(status);
```
**Query Patterns:**
- `SELECT * FROM agents WHERE agent_id = ?` - Primary key lookup
- `SELECT * FROM agents WHERE session_id = ?` - All agents for session
- `SELECT * FROM agents WHERE status = 'running'` - Active agents
### events Indexes
```sql
-- Keeps the per-tool-event dedup used by subagent import an index seek instead
-- of a full events scan. importSubagentFromJsonl checks
-- `... WHERE agent_id = ? AND event_type = ? AND data LIKE '%"tool_use_id":"X"%'`
-- before inserting; on a subagent-heavy re-import this drops a large sweep from
-- tens of seconds to sub-second.
CREATE INDEX idx_events_agent_type ON events(agent_id, event_type);
```
### tool_executions Indexes
```sql
CREATE INDEX idx_tools_agent_id ON tool_executions(agent_id);
CREATE INDEX idx_tools_created_at ON tool_executions(created_at DESC);
```
**Query Patterns:**
- `SELECT * FROM tool_executions WHERE agent_id = ?` - All tools for agent
- `SELECT * FROM tool_executions ORDER BY created_at DESC LIMIT 100` - Recent tools
### notifications Indexes
```sql
CREATE INDEX idx_notifications_session_id ON notifications(session_id);
```
**Query Patterns:**
- `SELECT * FROM notifications WHERE session_id = ?` - All notifications for session
---
## Migrations
### Schema Versioning
```mermaid
graph TB
V1[Version 1<br/>Initial schema] --> V2[Version 2<br/>Add updated_at]
V2 --> V3[Version 3<br/>Add pricing_rules]
V3 --> VN[Version N<br/>Future migrations]
style V1 fill:#3B82F6
style V2 fill:#10B981
style V3 fill:#F59E0B
```
### Migration Strategy
```javascript
// db.js - Schema versioning
const SCHEMA_VERSION = 3;
function runMigrations() {
const currentVersion = db.pragma('user_version', { simple: true });
if (currentVersion < 1) {
// Initial schema
db.exec(`
CREATE TABLE sessions (...);
CREATE TABLE agents (...);
-- etc.
`);
db.pragma('user_version = 1');
}
if (currentVersion < 2) {
// Add updated_at column
db.exec(`ALTER TABLE sessions ADD COLUMN updated_at TEXT DEFAULT (datetime('now'))`);
db.pragma('user_version = 2');
}
if (currentVersion < 3) {
// Add pricing_rules table
db.exec(`CREATE TABLE pricing_rules (...)`);
db.pragma('user_version = 3');
}
}
```
### Migration Workflow
```mermaid
sequenceDiagram
participant App
participant DB
participant Migrations
App->>DB: Open connection
DB->>Migrations: Check PRAGMA user_version
Migrations->>Migrations: Compare with SCHEMA_VERSION
alt Version mismatch
Migrations->>DB: Run migration scripts
DB->>Migrations: Success
Migrations->>DB: Update user_version
else Version match
Migrations->>App: Ready
end
App->>DB: Application queries
```
---
## Query Patterns
### Common Queries
#### List Recent Sessions
```sql
SELECT
s.*,
COUNT(DISTINCT a.id) as agent_count,
COUNT(DISTINCT t.id) as tool_count
FROM sessions s
LEFT JOIN agents a ON s.session_id = a.session_id
LEFT JOIN tool_executions t ON a.agent_id = t.agent_id
GROUP BY s.id
ORDER BY s.updated_at DESC
LIMIT 50;
```
**Performance:** ~5-10ms (with indexes)
#### Get Session with Agents
```sql
SELECT * FROM sessions WHERE session_id = 'sess_abc123';
SELECT * FROM agents WHERE session_id = 'sess_abc123';
```
**Performance:** ~1-2ms per query
#### Get Agent Tools
```sql
SELECT * FROM tool_executions
WHERE agent_id = 'agent_xyz789'
ORDER BY created_at DESC;
```
**Performance:** ~2-5ms
#### Calculate Total Cost
```sql
SELECT
SUM(cost) as total_cost
FROM agents
WHERE session_id = 'sess_abc123';
```
**Performance:** ~1-2ms
### Query Optimization
```mermaid
graph TB
Query[SQL Query] --> Explain[EXPLAIN QUERY PLAN]
Explain --> Scan{Full Table<br/>Scan?}
Scan -->|Yes| AddIndex[Add Index]
Scan -->|No| Check{Query Time<br/>>10ms?}
AddIndex --> Retest[Re-test Query]
Retest --> Check
Check -->|Yes| Optimize[Optimize Query<br/>Rewrite, Denormalize]
Check -->|No| Done[Acceptable Performance]
style AddIndex fill:#F59E0B
style Optimize fill:#EF4444
style Done fill:#10B981
```
---
## Performance Optimization
### SQLite Pragmas
```javascript
// db.js - Performance tuning
db.pragma('journal_mode = WAL'); // Write-Ahead Logging
db.pragma('synchronous = NORMAL'); // Faster writes (safe with WAL)
db.pragma('cache_size = -64000'); // 64MB cache
db.pragma('temp_store = MEMORY'); // Temp tables in memory
db.pragma('mmap_size = 30000000000'); // Memory-mapped I/O (30GB)
db.pragma('page_size = 4096'); // Optimal page size
```
### Prepared Statements
```javascript
// db.js - Prepared statements prevent SQL injection + optimize performance
const stmts = {
findSession: db.prepare('SELECT * FROM sessions WHERE session_id = ?'),
createSession: db.prepare('INSERT INTO sessions (session_id, model) VALUES (?, ?)'),
updateSession: db.prepare('UPDATE sessions SET status = ?, total_cost = ? WHERE session_id = ?'),
touchSession: db.prepare("UPDATE sessions SET updated_at = datetime('now') WHERE session_id = ?")
};
// Usage
const session = stmts.findSession.get('sess_abc123');
stmts.touchSession.run('sess_abc123');
```
### Transaction Batching
```javascript
// Batch multiple writes in a transaction
const insertMany = db.transaction((tools) => {
for (const tool of tools) {
stmts.createToolExecution.run(tool.agent_id, tool.tool_name, tool.duration_ms);
}
});
insertMany([
{ agent_id: 'agent_1', tool_name: 'bash', duration_ms: 100 },
{ agent_id: 'agent_1', tool_name: 'view', duration_ms: 50 },
// ... more tools
]);
```
### Performance Benchmarks
| Operation | Without Optimization | With Optimization | Improvement |
|-----------|---------------------|-------------------|-------------|
| Session list (50) | 25ms | 5ms | 5x faster |
| Hook processing | 15ms | 2ms | 7.5x faster |
| Batch insert (100 tools) | 500ms | 50ms | 10x faster |
---
## Data Integrity
### Foreign Key Constraints
```sql
-- Enabled by default in db.js
PRAGMA foreign_keys = ON;
```
**Constraint Enforcement:**
```mermaid
graph TB
Insert[INSERT agent] --> Check{session_id exists?}
Check -->|Yes| Allow[Insert Allowed]
Check -->|No| Reject[FOREIGN KEY constraint failed]
Delete[DELETE session] --> Cascade{Cascade enabled?}
Cascade -->|Yes| DeleteChildren[Delete agents and tools]
Cascade -->|No| BlockDelete[Cannot delete FK exists]
style Allow fill:#10B981
style Reject fill:#EF4444
style DeleteChildren fill:#F59E0B
```
### Data Validation
```javascript
// Validate before insert
function validateSession(session) {
if (!session.session_id) throw new Error('session_id required');
if (session.total_cost < 0) throw new Error('total_cost must be >= 0');
if (!['active', 'completed'].includes(session.status)) {
throw new Error('Invalid status');
}
}
```
---
## Backup Strategies
### Online Backup (Recommended)
```sql
-- Using VACUUM INTO (SQLite 3.27+)
VACUUM INTO '/backups/dashboard_20240318.db';
```
### Offline Backup
```bash
#!/bin/bash
# Stop application
systemctl stop agent-dashboard
# Copy database file
cp /var/lib/agent-dashboard/dashboard.db /backups/dashboard_$(date +%Y%m%d).db
# Start application
systemctl start agent-dashboard
```
### Backup Schedule
```mermaid
graph TB
subgraph "Automated Backups"
Daily[Daily Backup<br/>2 AM UTC]
Weekly[Weekly Backup<br/>Sunday 2 AM]
Monthly[Monthly Backup<br/>1st of month]
end
subgraph "Retention"
Daily --> R7[Keep 7 days]
Weekly --> R4[Keep 4 weeks]
Monthly --> R12[Keep 12 months]
end
subgraph "Storage"
R7 --> Local[Local Disk]
R4 --> S3[AWS S3]
R12 --> Glacier[AWS Glacier]
end
style Daily fill:#3B82F6
style S3 fill:#FF9900
```
---
## Summary
The database schema provides:
- ✅ **Normalized design** - Minimal redundancy, clear relationships
- ✅ **Performance optimized** - Indexes, prepared statements, WAL mode
- ✅ **Data integrity** - Foreign keys, constraints, transactions
- ✅ **Migration support** - Schema versioning with PRAGMA user_version
- ✅ **Comprehensive indexing** - Fast queries for common access patterns
- ✅ **Backup strategies** - Online + offline backup options
For API usage, see [docs/API.md](./API.md).
docs/DEPLOYMENT.md
+920
View File
@@ -0,0 +1,920 @@
# Deployment Guide
Enterprise deployment strategies for Agent Dashboard across development, staging, and production environments.
---
## Table of Contents
- [Overview](#overview)
- [Deployment Architecture](#deployment-architecture)
- [Local Development](#local-development)
- [Production Deployment](#production-deployment)
- [Docker Deployment](#docker-deployment)
- [Cloud Deployment](#cloud-deployment)
- [Process Management](#process-management)
- [Monitoring & Logging](#monitoring--logging)
- [Backup & Recovery](#backup--recovery)
- [Security Hardening](#security-hardening)
- [Performance Tuning](#performance-tuning)
- [Troubleshooting](#troubleshooting)
---
## Overview
Agent Dashboard supports multiple deployment modes:
- **Local Development** - Hot reload for rapid iteration
- **Docker** - Containerized deployment with Docker/Podman
- **PM2** - Process management for production
- **Systemd** - System service on Linux
- **Cloud** - Deploy to AWS, Azure, GCP, or other cloud providers
```mermaid
graph TB
subgraph "Deployment Modes"
Dev[Local Development<br/>npm run dev]
Docker[Docker Container<br/>docker compose up]
PM2[PM2 Process Manager<br/>pm2 start]
Systemd[Systemd Service<br/>systemctl start]
Cloud[Cloud Platform<br/>Kubernetes, ECS, etc.]
end
subgraph "Environment"
DevEnv[Development<br/>Hot reload, verbose logs]
StagingEnv[Staging<br/>Production build, test data]
ProdEnv[Production<br/>Optimized, monitoring]
end
Dev --> DevEnv
Docker --> DevEnv
Docker --> StagingEnv
PM2 --> ProdEnv
Systemd --> ProdEnv
Cloud --> ProdEnv
style Dev fill:#3B82F6
style PM2 fill:#10B981
style Cloud fill:#F59E0B
```
---
## Deployment Architecture
### Single-Server Architecture
```mermaid
graph TB
subgraph "Server Host"
subgraph "Node.js Process"
Express[Express Server<br/>:4820]
Static[Static File Serving<br/>client/dist/]
API[REST API]
WS[WebSocket Server]
DB[(SQLite DB<br/>data/dashboard.db)]
end
end
subgraph "Clients"
Browser[Web Browsers]
MCP[MCP Clients]
end
subgraph "Claude Code"
Hooks[Hook Events]
end
Browser -->|HTTP/WS| Express
MCP -->|HTTP| API
Hooks -->|HTTP POST| Express
Express --> Static
Express --> API
Express --> WS
API --> DB
Express --> DB
style Express fill:#000000,color:#fff
style DB fill:#003B57,color:#fff
```
### High-Availability Architecture
```mermaid
graph TB
subgraph "Load Balancer"
LB[Nginx/HAProxy]
end
subgraph "Application Servers"
App1[Node.js Server 1<br/>:4820]
App2[Node.js Server 2<br/>:4820]
App3[Node.js Server 3<br/>:4820]
end
subgraph "Data Layer"
Redis[Redis<br/>WebSocket pub/sub]
DB[(PostgreSQL<br/>Shared database)]
end
LB --> App1
LB --> App2
LB --> App3
App1 --> Redis
App2 --> Redis
App3 --> Redis
App1 --> DB
App2 --> DB
App3 --> DB
style LB fill:#10B981
style Redis fill:#DC2626
style DB fill:#2563EB
```
---
## Local Development
### Prerequisites
- Node.js >= 20.0.0
- npm >= 9.0.0
### Setup
```bash
# Clone repository
git clone https://github.com/your-org/agent-dashboard.git
cd agent-dashboard
# Install dependencies
npm run setup
# Start development servers
npm run dev
```
### Development Architecture
```mermaid
sequenceDiagram
participant Dev as Developer
participant Server as Dev Server<br/>:4820 (watch mode)
participant Client as Vite Dev<br/>:5173 (HMR)
participant Browser
Dev->>Server: Edit server/*.js
Server->>Server: Auto-reload
Server-->>Dev: Ready
Dev->>Client: Edit client/src/*
Client->>Client: HMR rebuild
Client->>Browser: Hot update
Browser->>Browser: Re-render
Note over Browser: State preserved!
```
### Running Components Separately
```bash
# Terminal 1: Server only
npm run dev:server
# Terminal 2: Client only
npm run dev:client
# Terminal 3: MCP server (optional)
npm run mcp:dev
```
---
## Production Deployment
### Build Process
```mermaid
graph TB
Source[Source Code] --> Install[Install Dependencies<br/>npm ci --production]
Install --> BuildClient[Build Client<br/>npm run build]
BuildClient --> Bundle[Bundled Assets<br/>client/dist/]
Bundle --> Deploy[Deploy to Server]
Deploy --> Server[Start Server<br/>node server/index.js]
style BuildClient fill:#646CFF
style Server fill:#10B981
```
### Production Checklist
```bash
# 1. Install dependencies (production only)
npm ci --production
cd client && npm ci --production && cd ..
# 2. Build client
npm run build
# 3. Set environment variables
export NODE_ENV=production
export PORT=4820
export DASHBOARD_DB_PATH=/var/lib/agent-dashboard/dashboard.db
# 4. Create data directory
mkdir -p /var/lib/agent-dashboard
# 5. Start server
node server/index.js
```
### Environment Variables
```bash
# Server
DASHBOARD_PORT=4820 # Server port
NODE_ENV=production # Environment mode
# Network exposure (SECURITY — GHSA-gr74-4xfh-6jw9)
# The server binds 127.0.0.1 by default and is NOT network-reachable. It reads
# transcripts, exports all data, and can spawn `claude`, so only widen the bind
# deliberately — and require a token when you do.
DASHBOARD_HOST=127.0.0.1 # set 0.0.0.0 ONLY if you must expose it
DASHBOARD_TOKEN= # required on /api/* + WS when set; use with a non-loopback host
DASHBOARD_ALLOWED_HOSTS= # extra Host names (comma-sep) for a LAN bind
# Database
DASHBOARD_DB_PATH=/var/lib/agent-dashboard/dashboard.db
# Logging
LOG_LEVEL=info # debug | info | warn | error
```
> **Reverse-proxy / Docker exposure:** the app binds loopback, so publish it to a
> network only through a proxy you control that adds TLS + auth, or set
> `DASHBOARD_HOST=0.0.0.0` **with** `DASHBOARD_TOKEN`. A `-p 4820:4820` Docker
> mapping assumes a trusted host network — do not expose it publicly without a
> token and a proxy.
---
## Docker Deployment
### Docker Compose (Recommended)
```yaml
# docker-compose.yml
version: '3.8'
services:
agent-dashboard:
build: .
ports:
- "4820:4820"
volumes:
- ./data:/app/data
environment:
- NODE_ENV=production
- PORT=4820
restart: unless-stopped
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:4820/api/sessions"]
interval: 30s
timeout: 10s
retries: 3
start_period: 40s
```
### Build & Run
```bash
# Build image
docker compose build
# Start container
docker compose up -d
# View logs
docker compose logs -f
# Stop container
docker compose down
```
### Multi-Stage Dockerfile
```dockerfile
# Build stage
FROM node:22-alpine AS builder
WORKDIR /app
# Install dependencies
COPY package*.json ./
COPY client/package*.json ./client/
RUN npm ci && cd client && npm ci
# Build client
COPY client ./client
RUN cd client && npm run build
# Production stage
FROM node:22-alpine
WORKDIR /app
# Copy built artifacts
COPY --from=builder /app/client/dist ./client/dist
COPY --from=builder /app/node_modules ./node_modules
COPY server ./server
COPY package.json ./
# Create data directory
RUN mkdir -p /app/data
EXPOSE 4820
HEALTHCHECK --interval=30s --timeout=10s --start-period=40s --retries=3 \
CMD node -e "require('http').get('http://localhost:4820/api/sessions', (res) => process.exit(res.statusCode === 200 ? 0 : 1))"
CMD ["node", "server/index.js"]
```
### Container Architecture
```mermaid
graph TB
subgraph "Docker Host"
subgraph "Container"
App[Node.js App<br/>:4820]
Volume[Volume Mount<br/>/app/data]
end
end
Host[Host Filesystem<br/>./data] -->|Bind Mount| Volume
App --> Volume
Client[External Clients] -->|Port 4820| App
style App fill:#2496ED,color:#fff
style Volume fill:#FFA500
```
---
## Cloud Deployment
### AWS (Elastic Beanstalk)
```bash
# Install EB CLI
pip install awsebcli
# Initialize
eb init -p node.js agent-dashboard
# Create environment
eb create production
# Deploy
eb deploy
# Open in browser
eb open
```
### Kubernetes
```yaml
# k8s/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: agent-dashboard
spec:
replicas: 3
selector:
matchLabels:
app: agent-dashboard
template:
metadata:
labels:
app: agent-dashboard
spec:
containers:
- name: agent-dashboard
image: agent-dashboard:latest
ports:
- containerPort: 4820
env:
- name: NODE_ENV
value: "production"
volumeMounts:
- name: data
mountPath: /app/data
volumes:
- name: data
persistentVolumeClaim:
claimName: agent-dashboard-pvc
---
apiVersion: v1
kind: Service
metadata:
name: agent-dashboard
spec:
selector:
app: agent-dashboard
ports:
- protocol: TCP
port: 80
targetPort: 4820
type: LoadBalancer
```
### Kubernetes Architecture
```mermaid
graph TB
subgraph "Kubernetes Cluster"
subgraph "LoadBalancer Service"
LB[Load Balancer<br/>:80]
end
subgraph "Pods"
Pod1[agent-dashboard-1<br/>:4820]
Pod2[agent-dashboard-2<br/>:4820]
Pod3[agent-dashboard-3<br/>:4820]
end
subgraph "Storage"
PVC[PersistentVolumeClaim]
PV[PersistentVolume]
end
end
LB --> Pod1
LB --> Pod2
LB --> Pod3
Pod1 --> PVC
Pod2 --> PVC
Pod3 --> PVC
PVC --> PV
style LB fill:#10B981
style PV fill:#F59E0B
```
---
## Process Management
### PM2 (Production Process Manager)
```bash
# Install PM2
npm install -g pm2
# Start application
pm2 start server/index.js --name agent-dashboard
# Start with environment
pm2 start server/index.js --name agent-dashboard --env production
# View logs
pm2 logs agent-dashboard
# Monitor
pm2 monit
# Restart
pm2 restart agent-dashboard
# Stop
pm2 stop agent-dashboard
# Auto-start on system boot
pm2 startup
pm2 save
```
### PM2 Ecosystem File
```javascript
// ecosystem.config.js
module.exports = {
apps: [{
name: 'agent-dashboard',
script: './server/index.js',
instances: 2,
exec_mode: 'cluster',
env: {
NODE_ENV: 'development',
PORT: 4820
},
env_production: {
NODE_ENV: 'production',
PORT: 4820,
DASHBOARD_DB_PATH: '/var/lib/agent-dashboard/dashboard.db'
},
max_memory_restart: '500M',
error_file: '/var/log/agent-dashboard/error.log',
out_file: '/var/log/agent-dashboard/out.log',
time: true
}]
};
```
```bash
# Start with ecosystem file
pm2 start ecosystem.config.js --env production
```
### Systemd Service (Linux)
```ini
# /etc/systemd/system/agent-dashboard.service
[Unit]
Description=Agent Dashboard
After=network.target
[Service]
Type=simple
User=agent-dashboard
WorkingDirectory=/opt/agent-dashboard
Environment=NODE_ENV=production
Environment=PORT=4820
Environment=DASHBOARD_DB_PATH=/var/lib/agent-dashboard/dashboard.db
ExecStart=/usr/bin/node server/index.js
Restart=on-failure
RestartSec=10
StandardOutput=journal
StandardError=journal
[Install]
WantedBy=multi-user.target
```
```bash
# Enable and start service
sudo systemctl enable agent-dashboard
sudo systemctl start agent-dashboard
# Check status
sudo systemctl status agent-dashboard
# View logs
sudo journalctl -u agent-dashboard -f
```
---
## Monitoring & Logging
### Health Checks
```bash
# Server health check
curl http://localhost:4820/api/sessions
# Expected: {"sessions": [...]}
```
### Logging Strategy
```mermaid
graph TB
subgraph "Application Logs"
App[Express Server] --> Console[Console Output]
Console --> Stdout[stdout/stderr]
end
subgraph "Log Aggregation"
Stdout --> PM2[PM2 Logs]
Stdout --> Systemd[Systemd Journal]
Stdout --> Docker[Docker Logs]
end
subgraph "Monitoring Tools"
PM2 --> File[Log Files]
Systemd --> Journalctl[journalctl]
Docker --> DockerLogs[docker logs]
end
subgraph "Analysis"
File --> Splunk[Splunk/ELK]
Journalctl --> Splunk
DockerLogs --> Splunk
end
style App fill:#000000,color:#fff
style Splunk fill:#10B981
```
### Monitoring Metrics
```javascript
// Add to server/index.js for metrics endpoint
app.get('/metrics', (req, res) => {
const metrics = {
uptime: process.uptime(),
memory: process.memoryUsage(),
cpu: process.cpuUsage(),
sessions: db.prepare('SELECT COUNT(*) as count FROM sessions').get(),
agents: db.prepare('SELECT COUNT(*) as count FROM agents').get(),
websocket_clients: wss.clients.size
};
res.json(metrics);
});
```
---
## Backup & Recovery
### Backup Strategy
```mermaid
graph TB
subgraph "Backup Process"
DB[(SQLite DB)] --> Backup[Backup Script]
Backup --> Local[Local Storage<br/>./backups/]
Backup --> S3[AWS S3]
Backup --> Cloud[Cloud Storage]
end
subgraph "Schedule"
Cron[Cron Job<br/>Daily at 2 AM]
end
subgraph "Retention"
Daily[Daily: 7 days]
Weekly[Weekly: 4 weeks]
Monthly[Monthly: 12 months]
end
Cron --> Backup
Local --> Daily
S3 --> Weekly
Cloud --> Monthly
style DB fill:#003B57,color:#fff
style S3 fill:#FF9900
```
### Backup Script
```bash
#!/bin/bash
# scripts/backup.sh
BACKUP_DIR="/var/backups/agent-dashboard"
DB_PATH="/var/lib/agent-dashboard/dashboard.db"
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
BACKUP_FILE="$BACKUP_DIR/dashboard_$TIMESTAMP.db"
# Create backup directory
mkdir -p "$BACKUP_DIR"
# Create backup (online backup with VACUUM INTO)
sqlite3 "$DB_PATH" "VACUUM INTO '$BACKUP_FILE'"
# Compress backup
gzip "$BACKUP_FILE"
# Upload to S3 (optional)
aws s3 cp "$BACKUP_FILE.gz" s3://my-backups/agent-dashboard/
# Delete old backups (keep last 7 days)
find "$BACKUP_DIR" -name "dashboard_*.db.gz" -mtime +7 -delete
echo "Backup completed: $BACKUP_FILE.gz"
```
### Restore Process
```bash
#!/bin/bash
# scripts/restore.sh
BACKUP_FILE=$1
DB_PATH="/var/lib/agent-dashboard/dashboard.db"
if [ -z "$BACKUP_FILE" ]; then
echo "Usage: ./restore.sh <backup_file.db.gz>"
exit 1
fi
# Stop application
systemctl stop agent-dashboard
# Decompress backup
gunzip -c "$BACKUP_FILE" > /tmp/restore.db
# Restore database
cp /tmp/restore.db "$DB_PATH"
chown agent-dashboard:agent-dashboard "$DB_PATH"
# Start application
systemctl start agent-dashboard
echo "Restore completed from $BACKUP_FILE"
```
---
## Security Hardening
### Security Checklist
```mermaid
graph TB
subgraph "Network Security"
Firewall[Firewall Rules<br/>Allow :4820 only from trusted IPs]
TLS[TLS/SSL<br/>HTTPS + WSS]
CORS[CORS Configuration<br/>Restrict origins]
end
subgraph "Application Security"
Validation[Input Validation]
Prepared[Prepared Statements<br/>SQL injection prevention]
Sanitize[Output Sanitization]
end
subgraph "System Security"
User[Dedicated User<br/>Non-root]
Perms[File Permissions<br/>640 for DB]
SELinux[SELinux/AppArmor]
end
style Firewall fill:#10B981
style Prepared fill:#10B981
style User fill:#10B981
```
### TLS Configuration (Nginx Reverse Proxy)
```nginx
# /etc/nginx/sites-available/agent-dashboard
server {
listen 443 ssl http2;
server_name dashboard.example.com;
ssl_certificate /etc/letsencrypt/live/dashboard.example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/dashboard.example.com/privkey.pem;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers HIGH:!aNULL:!MD5;
location / {
proxy_pass http://localhost:4820;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
location /ws {
proxy_pass http://localhost:4820/ws;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
}
# Redirect HTTP to HTTPS
server {
listen 80;
server_name dashboard.example.com;
return 301 https://$server_name$request_uri;
}
```
---
## Performance Tuning
### Node.js Optimization
```bash
# Increase memory limit
NODE_OPTIONS="--max-old-space-size=4096" node server/index.js
# Enable V8 optimizations
node --optimize-for-size server/index.js
```
### SQLite Tuning
```javascript
// server/db.js - Add these pragmas
db.pragma('journal_mode = WAL'); // Write-Ahead Logging
db.pragma('synchronous = NORMAL'); // Faster writes
db.pragma('cache_size = -64000'); // 64MB cache
db.pragma('temp_store = MEMORY'); // Temp tables in memory
db.pragma('mmap_size = 30000000000'); // Memory-mapped I/O
db.pragma('page_size = 4096'); // Optimal page size
```
### Nginx Tuning
```nginx
# /etc/nginx/nginx.conf
worker_processes auto;
worker_connections 4096;
http {
# Enable compression
gzip on;
gzip_comp_level 6;
gzip_types text/plain text/css application/json application/javascript;
# Client body buffer
client_body_buffer_size 128k;
# Keepalive
keepalive_timeout 65;
keepalive_requests 100;
# Proxy buffering
proxy_buffering on;
proxy_buffer_size 4k;
proxy_buffers 8 4k;
}
```
---
## Troubleshooting
### Common Issues
```mermaid
graph TB
subgraph "Issue Categories"
Startup[Startup Failures]
Connection[Connection Errors]
Performance[Performance Issues]
Data[Data Inconsistencies]
end
subgraph "Diagnostics"
Logs[Check Logs]
Health[Health Checks]
Metrics[Monitor Metrics]
DB[Database Integrity]
end
Startup --> Logs
Connection --> Health
Performance --> Metrics
Data --> DB
style Logs fill:#F59E0B
```
### Issue Resolution Guide
| Issue | Symptoms | Solution |
|-------|----------|----------|
| Port already in use | `EADDRINUSE: address already in use :::4820` | `lsof -i :4820` then kill process |
| Database locked | `database is locked` | Check for long-running queries, increase timeout |
| WebSocket connection fails | Clients can't connect | Check firewall, verify WebSocket upgrade headers |
| High memory usage | >500MB RAM | Enable memory limits, check for leaks |
| Slow queries | API responses >100ms | Add indexes, use EXPLAIN QUERY PLAN |
### Debug Mode
```bash
# Enable verbose logging
DEBUG=* node server/index.js
# SQLite query logging
NODE_ENV=development node server/index.js
```
---
## Summary
This deployment guide covers:
- ✅ **Multiple deployment modes** - Local, Docker, PM2, Systemd, Cloud
- ✅ **Production best practices** - Environment variables, health checks, logging
- ✅ **Process management** - PM2, systemd service files
- ✅ **Monitoring & logging** - Metrics endpoint, log aggregation
- ✅ **Backup & recovery** - Automated backups, restore procedures
- ✅ **Security hardening** - TLS, CORS, firewall rules
- ✅ **Performance tuning** - Node.js, SQLite, Nginx optimizations
- ✅ **Troubleshooting** - Common issues and resolutions
For architecture details, see [ARCHITECTURE.md](../ARCHITECTURE.md).
docs/HOOKS.md
+880
View File
@@ -0,0 +1,880 @@
# Hook System Integration Guide
Comprehensive guide to integrating with Claude Code's hook system for real-time agent monitoring.
---
## Table of Contents
- [Overview](#overview)
- [Hook Architecture](#hook-architecture)
- [Hook Installation](#hook-installation)
- [Hook Types](#hook-types)
- [Hook Handler Implementation](#hook-handler-implementation)
- [Event Processing](#event-processing)
- [Error Handling](#error-handling)
- [Performance Considerations](#performance-considerations)
- [Testing Hooks](#testing-hooks)
- [Troubleshooting](#troubleshooting)
---
## Overview
Claude Code provides a hook system that allows external tools to receive real-time events during agent execution. Agent Dashboard uses these hooks to capture session lifecycle, tool executions, and notifications.
> **Cursor (informational):** Live hooks fire from Claude Code. **Cursor** sessions that only exist as JSONL under `~/.claude` (Cursor uses the same paths locally) are still counted — they appear via startup import, continuous project sync, or remote SSH sync, not via hooks.
```mermaid
graph TB
subgraph "Claude Code Process"
CLI[Claude CLI]
Agent[Agent Execution]
Hooks[Hook System]
end
subgraph "Hook Wiring (~/.claude/settings.json)"
SessionStart[SessionStart]
UserPromptSubmit[UserPromptSubmit]
PreTool[PreToolUse]
PostTool[PostToolUse]
Stop[Stop]
SubagentStop[SubagentStop]
Notification[Notification]
SessionEnd[SessionEnd]
end
subgraph "Hook Handler"
Handler[hook-handler.js]
end
subgraph "Dashboard Server"
API[Express Server<br/>:4820]
end
Agent --> Hooks
Hooks -->|stdin JSON| SessionStart
Hooks -->|stdin JSON| UserPromptSubmit
Hooks -->|stdin JSON| PreTool
Hooks -->|stdin JSON| PostTool
Hooks -->|stdin JSON| Stop
Hooks -->|stdin JSON| SubagentStop
Hooks -->|stdin JSON| Notification
Hooks -->|stdin JSON| SessionEnd
SessionStart -->|exec| Handler
UserPromptSubmit -->|exec| Handler
PreTool -->|exec| Handler
PostTool -->|exec| Handler
Stop -->|exec| Handler
SubagentStop -->|exec| Handler
Notification -->|exec| Handler
SessionEnd -->|exec| Handler
Handler -->|HTTP POST| API
style Hooks fill:#F59E0B
style Handler fill:#10B981
style API fill:#3B82F6
```
---
## Hook Architecture
### Hook Execution Flow
```mermaid
sequenceDiagram
participant Claude as Claude Code
participant HookScript as Hook Script<br/>(Python)
participant Handler as hook-handler.js
participant Server as Dashboard Server
participant DB as SQLite
participant WS as WebSocket
Claude->>HookScript: Execute (stdin: JSON)
HookScript->>HookScript: Read stdin
HookScript->>Handler: exec node hook-handler.js
Handler->>Handler: Parse JSON
Handler->>Server: HTTP POST /hooks/{type}
Server->>DB: Insert/Update data
DB-->>Server: Success
Server->>WS: Broadcast event
WS-->>Server: Sent to clients
Server-->>Handler: 200 OK
Handler-->>HookScript: exit 0
HookScript-->>Claude: exit 0 (non-blocking)
Note over Claude: Continues execution<br/>without waiting
```
> **Security:** the hook handler POSTs to the loopback dashboard (`127.0.0.1:<port>`).
> The `/api/hooks` ingestion path is **exempt** from the optional `DASHBOARD_TOKEN`
> gate — it is a local-only write — so hooks keep working without a token even when
> one is configured for the rest of the API (GHSA-gr74-4xfh-6jw9).
### Hook System Characteristics
**Design Principles:**
```mermaid
graph TB
subgraph "Hook System Goals"
NonBlocking[Non-Blocking<br/>Never block Claude Code]
FailSafe[Fail-Safe<br/>Errors don't stop execution]
FastExec[Fast Execution<br/>&lt; 100ms per hook]
Complete[Complete Data<br/>Capture all events]
end
subgraph "Implementation"
Timeout[5s Timeout]
ErrorLog[Silent Error Logging]
Async[Async HTTP POST]
JSON[JSON Serialization]
end
NonBlocking --> Timeout
FailSafe --> ErrorLog
FastExec --> Async
Complete --> JSON
style NonBlocking fill:#10B981
style FailSafe fill:#10B981
style FastExec fill:#10B981
```
---
## Hook Installation
### Installation Script
```bash
# Install hooks
npm run install-hooks
```
> [!IMPORTANT]
> **Hooks are a host-side step.** Claude Code runs on your host, so the hook
> command must reference a `hook-handler.js` path that exists on the **host**.
> Run `npm run install-hooks` on the host — never inside a container. When run
> inside Docker/Podman, the installer **refuses** and exits non-zero (issue
> #193): a container-internal path written into a bind-mounted `~/.claude` would
> break every host hook with `MODULE_NOT_FOUND`. The host handler POSTs to
> `http://localhost:4820`, which a containerized dashboard already publishes.
> (Escape hatch for running Claude Code *inside* the same container:
> `CCAM_ALLOW_CONTAINER_HOOKS=1 npm run install-hooks`.)
This copies hook scripts from `scripts/hooks/` to `.githooks/`:
```mermaid
graph LR
Source[scripts/hooks/*.py] -->|Copy| Target[.githooks/*.py]
Handler[scripts/hook-handler.js] -->|Reference| Target
Target -->|chmod +x| Executable[Executable Hooks]
style Source fill:#3B82F6
style Executable fill:#10B981
```
### Manual Installation
```bash
#!/bin/bash
# scripts/install-hooks.js
HOOKS_DIR=".githooks"
SOURCE_DIR="scripts/hooks"
# Create hooks directory
mkdir -p "$HOOKS_DIR"
# Copy hook scripts
cp "$SOURCE_DIR/session-start.py" "$HOOKS_DIR/"
cp "$SOURCE_DIR/pre-tool-use.py" "$HOOKS_DIR/"
cp "$SOURCE_DIR/post-tool-use.py" "$HOOKS_DIR/"
cp "$SOURCE_DIR/stop.py" "$HOOKS_DIR/"
cp "$SOURCE_DIR/subagent-stop.py" "$HOOKS_DIR/"
cp "$SOURCE_DIR/notification.py" "$HOOKS_DIR/"
cp "$SOURCE_DIR/session-end.py" "$HOOKS_DIR/"
# Make executable
chmod +x "$HOOKS_DIR"/*.py
echo "Hooks installed in .githooks/"
```
### Verification
```bash
# Check hook files exist
ls -la .githooks/
# Expected output:
# session-start.py
# pre-tool-use.py
# post-tool-use.py
# stop.py
# subagent-stop.py
# notification.py
# session-end.py
```
---
## Hook Types
### 1. SessionStart
Triggered when a Claude Code session starts. The `source` field distinguishes the trigger: `startup` (fresh launch), `resume` (`--resume`/`--continue`), `clear` (`/clear`), and `compact` — which fires **mid-turn** when auto-compaction kicks in while Claude is actively working, not at a fresh prompt.
**Payload Example:**
```json
{
"type": "sessionStart",
"sessionId": "sess_abc123",
"source": "startup",
"model": "claude-sonnet-4",
"timestamp": "2024-03-18T12:00:00Z"
}
```
**Purpose:**
- Create the session and main-agent records on first contact
- Stamp `awaiting_input_since` (with `awaiting_reason` = `session_start`) so the dashboard shows the row in **Waiting** from the moment the CLI lands at a prompt — **only for `startup`/`resume`/`clear`**. A `compact`-source SessionStart fires mid-turn while Claude is working, so it leaves the awaiting flag untouched: a genuinely-active session stays **Active** (not flipped to Waiting), and a session that compacted while idle keeps its existing Waiting flag and reason
- Reactivate completed/abandoned sessions on resume
- Sweep other active sessions whose last activity is older than `DASHBOARD_STALE_MINUTES` (default 180), marking them `abandoned` with their agents `completed` (Remote Data Source sessions, `source` ≠ `local`, are exempt — their status comes from the SSH-mirror reconciliation, not local activity)
---
### 2. UserPromptSubmit
Triggered the moment the user hits enter on a prompt — fires *before* Claude does any work.
**Payload Example:**
```json
{
"type": "userPromptSubmit",
"sessionId": "sess_abc123",
"prompt": "Refactor this function...",
"timestamp": "2024-03-18T12:00:30Z"
}
```
**Purpose:**
- Clear `awaiting_input_since` (and `awaiting_reason` — both reset to NULL together) on the session and main agent
- Promote the main agent to `working` so the dashboard reflects "Claude is now thinking on this" through the entire response — including text-only replies that emit no `PreToolUse` before `Stop`
---
### 3. PreToolUse
Triggered before a tool executes.
**Payload Example:**
```json
{
"type": "preToolUse",
"sessionId": "sess_abc123",
"agentId": "agent_main_001",
"toolName": "bash",
"timestamp": "2024-03-18T12:01:00Z"
}
```
**Purpose:**
- Clear `awaiting_input_since` (and `awaiting_reason` — both reset to NULL together; Claude can only call a tool after fresh user input)
- Set agent to `working`, set `current_tool`
- Track tool execution start time
- If tool name is `Agent`, create a subagent record
---
### 4. PostToolUse
Triggered after a tool completes execution.
**Payload Example:**
```json
{
"type": "postToolUse",
"sessionId": "sess_abc123",
"agentId": "agent_main_001",
"toolName": "bash",
"durationMs": 1234,
"success": true,
"inputTokens": 1500,
"outputTokens": 800,
"timestamp": "2024-03-18T12:01:01.234Z"
}
```
**Purpose:**
- Clear `awaiting_input_since` (and `awaiting_reason` — both reset to NULL together; covers permission-prompt approval mid-tool)
- Clear `current_tool` on agent (agent stays `working`)
- Update agent token counts via shared transcript cache
- Calculate and update cost
- Rollup cost to session
**Cost Calculation Flow:**
```mermaid
graph TB
PostHook[PostToolUse Hook] --> Tokens{Has Token<br/>Counts?}
Tokens -->|Yes| Pricing[Fetch Pricing Rule]
Tokens -->|No| Skip[Skip Cost Update]
Pricing --> Calculate[Cost = <br/>input/1M * input_price +<br/>output/1M * output_price]
Calculate --> UpdateAgent[Update agent.cost]
UpdateAgent --> Rollup[Rollup to session.total_cost]
Rollup --> Broadcast[Broadcast Updates]
style Calculate fill:#10B981
style Broadcast fill:#F59E0B
```
---
### 5. Stop
Triggered when Claude finishes a turn (NOT when the session is closed).
**Payload Example:**
```json
{
"type": "stop",
"sessionId": "sess_abc123",
"stop_reason": "end_turn",
"timestamp": "2024-03-18T12:05:00Z"
}
```
**Purpose:**
- Non-error: set main agent to `idle` and stamp `awaiting_input_since` (with `awaiting_reason` = `stop`) — Claude finished its turn, ball is in the user's court. The session shows as **Waiting** until `UserPromptSubmit` / `PreToolUse` fires
- Error (`stop_reason="error"`): drop `awaiting_input_since` (and `awaiting_reason`, cleared to NULL together), mark the session `error`
- Background subagents continue running — they complete individually via `SubagentStop`, never via `Stop`
> **Note:** `Stop` does **not** fire when the user cancels a turn with `Esc` — interrupts emit no hook at all. The dashboard instead recovers cancelled turns from the transcript (see [User interrupts (Esc)](#user-interrupts-esc--no-hook-fires)).
---
### 6. SubagentStop
Triggered when a sub-agent (explore, task, etc.) completes.
**Payload Example:**
```json
{
"type": "subagentStop",
"sessionId": "sess_abc123",
"agentId": "agent_explore_002",
"agentType": "explore",
"timestamp": "2024-03-18T12:03:00Z"
}
```
**Purpose:**
- Match the finishing subagent by description, type, or task and mark it `completed`
- **Deliberately does NOT clear `awaiting_input_since`** (nor `awaiting_reason`) — a backgrounded subagent finishing tells us nothing about whether the human has responded
- **Triggers a fire-and-forget JSONL scan** (`scanAndImportSubagents` from `scripts/import-history.js`) after `res.json()` returns. The scan walks the session's `subagents/agent-*.jsonl` files, pairs each assistant `tool_use` block with the next matching user `tool_result` block by `tool_use_id`, and emits per-tool `PreToolUse` + `PostToolUse` events under the subagent's own `agent_id`. Idempotent (`data LIKE '%"tool_use_id":"X"%'` dedup) and merges into a hook-created live row when one matches by `subagent_type + started_at` within 30 s — closes the gap where subagent-internal tool calls would otherwise be invisible to the dashboard
- **Attributes per-subagent tokens to each subagent's OWN model** (issue #185). Each subagent transcript carries its own `msg.usage` under its own `msg.model`; the scan writes those token buckets to `token_usage` keyed by the real model (e.g. a Haiku QA agent under an Opus orchestrator) so cost is no longer priced at the orchestrator's rate. The subagent's resolved model is also stamped onto its agent row (`metadata.model`). Buckets whose model equals the parent session's model are deliberately **skipped** here — that bucket is owned by the main-transcript writer, and double-writing it would trip `replaceTokenUsage`'s compaction baseline-shift; same-model subagents are reconciled by the authoritative `importSession` / `reconcileTokens` path instead
- **Rebuilds the nested-subagent hierarchy** (`reconcileSubagentParents`). Subagent rows are inserted flat under the main agent because no single hook event or JSONL file carries the spawner's identity. Each subagent transcript, however, records every child it spawned via the Task tool as `toolUseResult.agentId` (surfaced by `parseSubagentFile` as `spawnedChildren`). The scan inverts these into a child→parent map and repoints `parent_agent_id` (via `setAgentParent`) so a subagent that spawns its own subagents nests under its **true** spawner instead of collapsing to a single level under main; any subagent no other subagent claims stays under main. Idempotent and additive (only rewrites `parent_agent_id`, never inserts/deletes), it also corrects the live PreToolUse-`Agent` parent heuristic's guesses once transcripts land. `scanAndImportSubagents` returns `reparented` alongside `created`; the `SubagentStop` refetch nudge fires when either is non-zero so a pure re-parent still refreshes the tree
- Imported tool events carry `imported: true, source: "subagent_jsonl"` in their JSON `data` payload so analytics can distinguish backfilled rows from live hook-captured ones if needed
---
### 7. Notification
Triggered when Claude Code sends a system notification.
**Payload Example:**
```json
{
"type": "notification",
"sessionId": "sess_abc123",
"notificationType": "backgroundTaskComplete",
"message": "Explore agent completed successfully",
"timestamp": "2024-03-18T12:03:00Z"
}
```
**Purpose:**
- Log the event for the activity feed
- If the message matches a permission/input-prompt pattern (`permission`, `waiting for input`, `needs your approval`, `awaiting your response`, …), stamp `awaiting_input_since` (with `awaiting_reason` = `notification`) so the session lands in **Waiting**
- If the message matches a compaction pattern, tag as a `Compaction` event
- Trigger a browser notification when the user has notifications enabled
---
### 8. SessionEnd
Triggered when a Claude Code session ends.
**Payload Example:**
```json
{
"type": "sessionEnd",
"sessionId": "sess_abc123",
"timestamp": "2024-03-18T14:30:00Z"
}
```
**Purpose:**
- Drop `awaiting_input_since` (and `awaiting_reason`, cleared to NULL together) on the session and any agents that still have it
- Mark all agents and the session as `completed` — **unless the session is in `error` AND that error is still unrecovered at the transcript tail** (`isErrorAtTail`: the latest API error has no successful turn after it), in which case `error` is preserved. A transient error the CLI retried past (successful assistant turns after the last error) finalizes as `completed` instead of freezing in a stale `error`
- Evict the session's transcript from the shared transcript cache
> **Stale-error self-heal.** Separately from `SessionEnd`, the 15 s watchdog now scans `error` sessions (not just `active`) and clears a session back to `active` when its transcript has progressed past the last API error (`isErrorAtTail` is false). Claude auto-retries transient API errors (e.g. "Connection closed mid-response") and keeps working, so an error followed by real turn activity has recovered — recovery previously required a live `UserPromptSubmit`/`PreToolUse` hook, leaving imported or sweep-monitored sessions pinned in `error` indefinitely.
---
## Hook Handler Implementation
### hook-handler.js Architecture
```mermaid
graph TB
CLI[CLI Args] --> Parse[Parse Hook Type]
Stdin[stdin] --> ReadJSON[Read JSON]
Parse --> HookType{Hook Type?}
ReadJSON --> Payload[Event Payload]
HookType -->|session-start| Endpoint1[POST /hooks/session-start]
HookType -->|pre-tool-use| Endpoint2[POST /hooks/pre-tool-use]
HookType -->|post-tool-use| Endpoint3[POST /hooks/post-tool-use]
HookType -->|stop| Endpoint4[POST /hooks/stop]
HookType -->|subagent-stop| Endpoint5[POST /hooks/subagent-stop]
HookType -->|notification| Endpoint6[POST /hooks/notification]
HookType -->|session-end| Endpoint7[POST /hooks/session-end]
Payload --> Endpoint1
Payload --> Endpoint2
Payload --> Endpoint3
Payload --> Endpoint4
Payload --> Endpoint5
Payload --> Endpoint6
Payload --> Endpoint7
Endpoint1 --> HTTP[HTTP POST]
Endpoint2 --> HTTP
Endpoint3 --> HTTP
Endpoint4 --> HTTP
Endpoint5 --> HTTP
Endpoint6 --> HTTP
Endpoint7 --> HTTP
HTTP --> Response{Success?}
Response -->|Yes| Exit0[exit 0]
Response -->|No| Exit1[exit 1]
style HTTP fill:#10B981
style Exit0 fill:#10B981
style Exit1 fill:#EF4444
```
### Implementation
```javascript
#!/usr/bin/env node
// scripts/hook-handler.js
const http = require('http');
const fs = require('fs');
const HOOK_TYPE = process.argv[2];
const SERVER_URL = 'http://localhost:4820';
const TIMEOUT = 5000; // 5s timeout
// Read JSON from stdin
let inputData = '';
process.stdin.setEncoding('utf8');
process.stdin.on('data', (chunk) => inputData += chunk);
process.stdin.on('end', () => {
try {
const payload = JSON.parse(inputData);
sendToServer(HOOK_TYPE, payload);
} catch (err) {
console.error('[hook-handler] JSON parse error:', err);
process.exit(1);
}
});
function sendToServer(hookType, payload) {
const postData = JSON.stringify(payload);
const options = {
hostname: 'localhost',
port: 4820,
path: `/hooks/${hookType}`,
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Content-Length': Buffer.byteLength(postData)
},
timeout: TIMEOUT
};
const req = http.request(options, (res) => {
let responseData = '';
res.on('data', (chunk) => responseData += chunk);
res.on('end', () => {
if (res.statusCode === 200) {
process.exit(0);
} else {
console.error(`[hook-handler] Server error: ${res.statusCode}`);
process.exit(1);
}
});
});
req.on('error', (err) => {
console.error('[hook-handler] Request error:', err);
process.exit(1);
});
req.on('timeout', () => {
console.error('[hook-handler] Request timeout');
req.destroy();
process.exit(1);
});
req.write(postData);
req.end();
}
```
> **Port resolution & fan-out.** The snippet above shows a single fixed `4820`
> for clarity. The real `scripts/hook-handler.js` resolves hook targets at
> runtime via `server/lib/server-info.js`:
>
> 1. If `CLAUDE_DASHBOARD_PORT` is set in the environment, the handler treats
> it as an explicit operator override and POSTs to that single port —
> no discovery, no fan-out (useful for tests and container setups).
> 2. Otherwise it reads `~/.claude/.agent-dashboard.json`, a JSON document
> that lists every dashboard server currently running on the machine.
> Each server appends its `{port, pid, startedAt, dataDir}` entry on
> startup and removes it on a clean shutdown. The handler **prunes any
> entry whose PID is no longer alive** and POSTs the hook payload to one
> port per **unique SQLite data directory** (lowest port wins when Docker
> and `npm run dev` share `~/.claude/agent-dashboard`).
> 3. If neither yields a target, the handler falls back to `4820`.
>
> Dashboards with **different** databases (e.g. the packaged desktop app using
> its own Application Support data dir alongside `npm run dev`) still each
> receive hooks. Dashboards sharing one database never double-ingest events.
---
## Event Processing
### Server-Side Hook Processing
```javascript
// server/routes/hooks.js
router.post('/session-start', (req, res) => {
try {
const { sessionId, model, agentId, agentType } = req.body;
// Upsert session
let session = stmts.findSession.get(sessionId);
if (!session) {
stmts.createSession.run(sessionId, model);
session = stmts.findSession.get(sessionId);
broadcast({ type: 'session.created', data: session });
}
// Create main agent
if (!stmts.findAgent.get(agentId)) {
stmts.createAgent.run(agentId, sessionId, agentType);
const agent = stmts.findAgent.get(agentId);
broadcast({ type: 'agent.created', data: agent });
}
// Touch session (update updated_at)
stmts.touchSession.run(sessionId);
res.json({ success: true });
} catch (err) {
console.error('session-start error:', err);
res.json({ success: false, error: err.message });
}
});
```
### Event Processing Pipeline
```mermaid
graph TB
Hook[Hook Event] --> Validate[Validate Payload]
Validate --> DB[Database Operations]
DB --> Session[Update Session]
DB --> Agent[Update Agent]
DB --> Tool[Create Tool Record]
Session --> Broadcast[Broadcast to WebSocket]
Agent --> Broadcast
Tool --> Broadcast
Broadcast --> Client1[Client 1]
Broadcast --> Client2[Client 2]
Broadcast --> ClientN[Client N]
style Validate fill:#3B82F6
style DB fill:#003B57,color:#fff
style Broadcast fill:#F59E0B
```
### Transcript-derived sync
On every event that carries a `transcript_path`, the shared `TranscriptCache` re-reads the JSONL (incrementally) and the ingestor keeps three session fields in sync with what the user is actually doing in the CLI:
- **Tokens / cost** — usage is accumulated per model bucket (compaction-aware baselines).
- **Model** — the most recent assistant entry's model keeps `sessions.model` current after a `/model` switch.
- **Name** — the session title is read from the transcript: the `custom-title` line (`/rename`, `claude -n`, picker `Ctrl+R`) always wins, otherwise the auto-generated `ai-title` fills a placeholder/auto name (so a user-chosen name is never clobbered). When neither title exists, the session's **first user prompt** (tool-result, meta/caveat, and slash-command plumbing entries skipped; whitespace-collapsed, 60-char label) fills the placeholder session name plus the main agent's placeholder name and empty task — a later `ai-title` can still replace a descriptor-filled name, and the agent fill passes the in-flight `current_tool` through so it is never wiped mid-turn. `sessions.name` is updated via a no-op-guarded statement and a `session_updated` broadcast fires only on a real change, so the dashboard reflects renames in real time. The 15 s error-detection watchdog runs the same sync for active sessions left idle right after a `/rename`.
### User interrupts (Esc) — no hook fires
Cancelling a turn with `Esc` fires **no hook at all** (a documented Claude Code limitation — there is no `Stop`, `Notification`, or other event on interrupt). Since `UserPromptSubmit` has already promoted the main agent to `working`, an un-handled cancel would leave the session stuck in `working` indefinitely. The dashboard recovers it from the transcript, via the same 15 s watchdog, two ways:
1. **Marker path** — when the cancel happens *after* some output, Claude Code appends a `[Request interrupted by user]` user entry (with an `interruptedMessageId`). `TranscriptCache` reports `pendingInterrupt`, computed from transcript ordering alone: the latest interrupt timestamp vs the latest real turn activity, both on Claude Code's clock. (It is **not** compared against the session's last hook event — those clocks differ, and for a sub-second cancel the `UserPromptSubmit` event is recorded *after* the transcript interrupt, the precise case that used to stay stuck.) The session moves to **Waiting** within ~15 s.
2. **Idle-working timeout** — when Esc is pressed *before any output*, Claude Code writes **no marker**; the only evidence is silence. When the main agent has been `working` with `current_tool` null and **neither a hook event nor the transcript mtime** has advanced for `DASHBOARD_WORKING_IDLE_SECONDS` (default `120`), the turn is treated as dead. A streaming/long-output turn (transcript still growing) and an in-flight tool call are exempt by those guards; a rare false flip self-heals on the next real hook.
Both paths land the session in **Waiting** (main agent → `waiting`, `awaiting_input_since` stamped with `awaiting_reason` = `interrupted` — identical to a non-error `Stop` aside from the reason) and log an `Interrupted` event. A resume (new prompt in the transcript) clears `pendingInterrupt` and the fresh hook keeps the session non-stale.
### Missed SessionEnd (dashboard down) — liveness reap
`SessionEnd` is the only signal that a session closed, and hooks are fire-and-forget: if the dashboard was **not running** when the user quit (Ctrl+C, terminal closed), the POST fails silently and the event is lost forever — the session previously sat in **Waiting** until the stale sweep (3 h by default). The same 15 s watchdog closes the gap with a **process-liveness probe** (`server/lib/session-liveness.js`): it enumerates running `claude` CLI processes and their working directories (`ps` + `lsof` on macOS, `/proc/<pid>/cwd` on Linux) and completes any `active` session whose `cwd` has no live claude process — the same terminal state a real `SessionEnd` produces, plus a synthetic `SessionEnd` event (`data.source = "liveness-probe"`) on the timeline.
Fail-safe guards: the probe reports "no answer" (nothing changes) on Windows, inside containers (host processes are invisible), on `ps`/`lsof` failure, or when disabled via `DASHBOARD_LIVENESS_PROBE=0` (the escape hatch for hooks arriving from another machine); the session must have a `cwd`, and that `cwd` must be **POSIX-absolute** — a household-hook-forwarded session reports the origin machine's own path (e.g. a Windows `D:\Git\ai-deck`) that this host's `/proc`/`lsof` scan can never produce, so the reap skips it rather than falsely completing every remote session (this makes a mixed local + forwarded deployment correct without the blanket `DASHBOARD_LIVENESS_PROBE=0`); **Remote Data Source sessions** (`sessions.source` ≠ `local`) are also skipped outright — their POSIX-absolute `cwd` lives on another machine reached over SSH, so this host's process probe proves nothing about them, and their status is reconciled from the SSH mirror by `remote-sync.js` (the same `source = 'local'` guard also exempts them from the watchdog's error/interrupt scan and both stale sweeps); and — on watchdog ticks only — its transcript must not have been written for at least `DASHBOARD_LIVENESS_IDLE_SECONDS` (default `60`; the last hook write is the fallback clock when no transcript exists on disk) so a mid-turn / just-resumed session never flickers out. The reap runs immediately at startup (rows from a previous run), again ~5 s after startup (rows the startup sync just imported) — both startup passes **skip the idle gate**, so a session quit even seconds before launch clears at once — and on every 15 s watchdog tick (gated) as the safety net. A false completion self-heals — the next hook event reactivates the session.
---
## Error Handling
### Error Handling Strategy
```mermaid
graph TB
Error[Error Occurs] --> Type{Error Type?}
Type -->|Network Error| Retry[Retry Once]
Type -->|Timeout| Log1[Log + Exit 1]
Type -->|Parse Error| Log2[Log + Exit 1]
Type -->|Server Error| Log3[Log + Exit 1]
Retry --> Success{Success?}
Success -->|Yes| Exit0[Exit 0]
Success -->|No| Exit1[Exit 1]
Log1 --> Exit1
Log2 --> Exit1
Log3 --> Exit1
style Exit0 fill:#10B981
style Exit1 fill:#EF4444
```
### Hook Script Error Handling
```python
#!/usr/bin/env python3
# .githooks/session-start.py
import sys
import json
import subprocess
import logging
logging.basicConfig(
filename='.githooks/hooks.log',
level=logging.ERROR,
format='%(asctime)s - %(levelname)s - %(message)s'
)
try:
data = json.load(sys.stdin)
result = subprocess.run(
['node', 'scripts/hook-handler.js', 'session-start'],
input=json.dumps(data),
text=True,
timeout=5,
capture_output=True
)
if result.returncode != 0:
logging.error(f'Hook handler failed: {result.stderr}')
except Exception as e:
logging.error(f'Hook error: {str(e)}')
# Always exit 0 to avoid blocking Claude Code
sys.exit(0)
```
---
## Performance Considerations
### Hook Execution Time
```mermaid
graph TB
subgraph "Hook Execution Breakdown"
Python[Python Script<br/>~10ms]
Handler[Node Handler<br/>~20ms]
HTTP[HTTP POST<br/>~30ms]
DB[Database Write<br/>~5ms]
WS[WebSocket Broadcast<br/>~5ms]
end
Total[Total: ~70ms]
Python --> Handler
Handler --> HTTP
HTTP --> DB
DB --> WS
WS --> Total
style Total fill:#10B981
```
**Performance Targets:**
| Phase | Target | Actual |
|-------|--------|--------|
| Hook script | < 20ms | ~10ms |
| Handler | < 30ms | ~20ms |
| HTTP POST | < 50ms | ~30ms |
| Database | < 10ms | ~5ms |
| **Total** | **< 100ms** | **~70ms** |
### Optimization Techniques
```mermaid
graph TB
subgraph "Optimizations"
Async[Async HTTP<br/>Don't wait for response]
Batch[Batch Updates<br/>Transaction batching]
Index[Database Indexes<br/>Fast lookups]
Pool[Connection Pooling<br/>Reuse connections]
end
Async --> Faster[Faster Hook Execution]
Batch --> Faster
Index --> Faster
Pool --> Faster
style Faster fill:#10B981
```
---
## Testing Hooks
### Manual Testing
```bash
# Test session-start hook
echo '{"type":"sessionStart","sessionId":"test_001","model":"claude-sonnet-4","agentId":"agent_test","agentType":"general-purpose"}' | \
python3 .githooks/session-start.py
# Test pre-tool-use hook
echo '{"type":"preToolUse","sessionId":"test_001","agentId":"agent_test","toolName":"bash"}' | \
python3 .githooks/pre-tool-use.py
# Test post-tool-use hook
echo '{"type":"postToolUse","sessionId":"test_001","agentId":"agent_test","toolName":"bash","durationMs":100,"success":true,"inputTokens":1000,"outputTokens":500}' | \
python3 .githooks/post-tool-use.py
```
### Integration Testing
```javascript
// server/__tests__/hooks.test.js
import { test } from 'node:test';
import assert from 'node:assert';
test('session-start hook creates session', async () => {
const payload = {
sessionId: 'test_session',
model: 'claude-sonnet-4',
agentId: 'test_agent',
agentType: 'general-purpose'
};
const response = await fetch('http://localhost:4820/hooks/session-start', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(payload)
});
const data = await response.json();
assert.strictEqual(data.success, true);
// Verify session exists
const session = await fetch('http://localhost:4820/api/sessions/test_session');
assert.strictEqual(session.status, 200);
});
```
---
## Troubleshooting
### Common Issues
| Issue | Symptoms | Solution |
|-------|----------|----------|
| Hooks not executing | No data in dashboard | Check `.githooks/` exists and scripts are executable |
| Timeout errors | Hooks take >5s | Check server is running, reduce timeout |
| Parse errors | JSON parse failed | Validate hook payload format |
| Permission denied | Hook script won't run | `chmod +x .githooks/*.py` |
| Server connection refused | HTTP POST fails | Start dashboard server (`npm start`) |
### Debug Mode
```bash
# Enable hook logging
export DASHBOARD_DEBUG=1
# Run hook manually with verbose output
python3 -u .githooks/session-start.py < test-payload.json
```
### Health Check
```bash
# Check server is running
curl http://localhost:4820/api/sessions
# Expected: {"sessions": [...]}
```
---
## Summary
The hook system provides:
- ✅ **Real-time event capture** - Lifecycle, tools, notifications
- ✅ **Non-blocking execution** - Never delays Claude Code
- ✅ **Fail-safe design** - Errors don't stop execution
- ✅ **Fast processing** - < 100ms per hook
- ✅ **Complete coverage** - All agent lifecycle events
- ✅ **Easy installation** - One-command setup
For server-side processing, see [server/README.md](../server/README.md).
docs/I18N.md
+314
View File
@@ -0,0 +1,314 @@
# Internationalization (i18n) Architecture and Usage
This guide documents how localization works in the Agent Dashboard, including architecture, resources, runtime behavior, testing, and rollout.
**Supported languages:** English (`en`), Chinese (`zh`), Vietnamese (`vi`), Korean (`ko`)
---
## 1) Architecture Overview
Localization is implemented in the frontend with `i18next` + `react-i18next` and browser language detection.
```mermaid
flowchart TB
subgraph Browser
User["User"]
LS["localStorage<br/>i18nextLng"]
Nav["navigator.language"]
end
subgraph ClientApp["React Client"]
Detector["i18next-browser-languagedetector"]
I18n["i18n init<br/>client/src/i18n/index.ts"]
NS["Namespace bundles<br/>common/nav/dashboard/..."]
UI["Pages + components<br/>useTranslation()"]
Format["format.ts<br/>locale-aware date/number/model-name"]
end
User --> UI
LS --> Detector
Nav --> Detector
Detector --> I18n
I18n --> NS
NS --> UI
I18n --> Format
```
**Key runtime facts**
- `supportedLngs`: `["en", "zh", "vi", "ko"]`
- `fallbackLng`: `"en"`
- `nonExplicitSupportedLngs`: `true` (e.g. `vi-VN` resolves to `vi`)
- Detection order: `localStorage` → `navigator`
---
## 2) Resource and Namespace Strategy
Translation resources are stored per language and namespace:
- `client/src/i18n/locales/en/*.json`
- `client/src/i18n/locales/zh/*.json`
- `client/src/i18n/locales/vi/*.json`
- `client/src/i18n/locales/ko/*.json`
Active namespaces:
- `common`
- `nav`
- `dashboard`
- `sessions`
- `activity`
- `analytics`
- `workflows`
- `settings`
- `kanban`
- `errors`
```mermaid
erDiagram
LANGUAGE ||--o{ NAMESPACE : contains
NAMESPACE ||--o{ KEY : defines
KEY ||--o{ TRANSLATION : maps_to
LANGUAGE {
string code "en|zh|vi"
string locale "en-US|zh-CN|vi-VN|ko-KR"
}
NAMESPACE {
string name "common|nav|dashboard|..."
string file_path "locales/{lang}/{namespace}.json"
}
KEY {
string id "dot.notation.or.leaf"
string type "string|pluralized"
}
TRANSLATION {
string value "localized text"
}
```
**Strategy notes**
- Keep namespace boundaries page/domain focused.
- Keep key parity across `en`, `zh`, `vi` files for the same namespace.
- Keep fallback behavior deterministic by ensuring `en` is always complete.
---
## 3) Key Naming Conventions
Use stable semantic keys, not English sentence literals.
### Convention rules
1. Use namespace-scoped keys: `namespace:key`
2. Use lower camelCase key segments
3. Keep terminology consistent across locales (for example, keep `Agent` / `Subagent` terms stable where required)
4. Use suffixes for plurals when needed (e.g. `_plural`)
5. Group nested concepts by domain (e.g. `time.justNow`, `time.mAgo`)
### Examples
- `nav:dashboard`
- `nav:languageNames.vi`
- `common:time.justNow`
- `common:time.mAgo`
- `kanban:agentCount`
- `kanban:agentCount_plural`
```mermaid
classDiagram
class I18nConfig {
+supportedLngs: ["en","zh","vi"]
+fallbackLng: "en"
+defaultNS: "common"
+detectionOrder: ["localStorage","navigator"]
}
class NamespaceResource {
+languageCode
+namespace
+jsonFilePath
+keys[]
}
class ReactComponent {
+useTranslation(namespace)
+t(key, params)
}
class SidebarLanguageSwitch {
+SUPPORTED_LANGUAGES
+normalizeLanguage()
+changeLanguage()
}
class FormatUtils {
+getCurrentLocale()
+formatTime()
+formatDateTime()
+fmtCostFull()
+formatModelName()
}
I18nConfig --> NamespaceResource
ReactComponent --> I18nConfig
ReactComponent --> NamespaceResource
SidebarLanguageSwitch --> I18nConfig
FormatUtils --> I18nConfig
```
---
## 4) Language Detection and Switching Flow
The sidebar language controls call `i18n.changeLanguage()` and UI updates reactively through `useTranslation`.
```mermaid
sequenceDiagram
participant U as User
participant SB as Sidebar.tsx
participant I as i18next
participant LD as LanguageDetector
participant NS as Locale Resources
participant UI as React Components
U->>SB: Click language button (EN/ZH/VI/KO)
SB->>I: changeLanguage("vi")
I->>NS: Resolve namespace bundles
NS-->>I: Return translations
I->>LD: Persist i18nextLng in localStorage
I-->>UI: Trigger rerender
UI->>UI: Re-evaluate t(...) keys
UI-->>U: Localized labels displayed
```
```mermaid
stateDiagram-v2
[*] --> Detecting
Detecting --> Loaded_en: localStorage/navigator resolves en
Detecting --> Loaded_zh: localStorage/navigator resolves zh
Detecting --> Loaded_vi: localStorage/navigator resolves vi
Detecting --> Loaded_ko: localStorage/navigator resolves ko
Detecting --> Loaded_en: unsupported locale -> fallback en
Loaded_en --> Loaded_zh: user switches to zh
Loaded_en --> Loaded_vi: user switches to vi
Loaded_zh --> Loaded_en: user switches to en
Loaded_zh --> Loaded_vi: user switches to vi
Loaded_vi --> Loaded_en: user switches to en
Loaded_vi --> Loaded_zh: user switches to zh
Loaded_en --> Loaded_ko: user switches to ko
Loaded_ko --> Loaded_en: user switches to en
```
---
## 5) Date and Number Localization Behavior
Formatting utilities are centralized in `client/src/lib/format.ts`.
- `en` → `en-US`
- `zh` → `zh-CN`
- `vi` → `vi-VN`
`formatTime`, `formatDateTime`, and `fmtCostFull` use locale-aware `toLocale*` APIs.
Timestamp parsing normalizes timezone-less SQLite datetime strings to UTC before display formatting.
`formatModelName` converts raw model identifiers (e.g. `claude-opus-4-7-20260101`, `claude-opus-4-7[1m]`) into human-friendly display names (e.g. "Claude Opus 4.7", "Claude Opus 4.7 (1M)"). This is locale-independent (brand names are proper nouns) and is applied across all UI surfaces except the Settings page (which shows raw patterns for pricing rule configuration).
```mermaid
flowchart LR
A["Raw timestamp / numeric value"] --> B["parseDate() normalization"]
B --> C["getCurrentLanguage()"]
C --> D{"Language"}
D -->|en| E["Locale en-US"]
D -->|zh| F["Locale zh-CN"]
D -->|vi| G["Locale vi-VN"]
E --> H["toLocaleTimeString / toLocaleString"]
F --> H
G --> H
H --> I["Localized date/time/number output"]
```
---
## 6) Testing Strategy
Use client tests to verify translation correctness, fallback behavior, and locale formatting:
- `client/src/i18n/__tests__/i18n.test.ts`
- `client/src/lib/__tests__/format.test.ts`
- `client/src/components/__tests__/Sidebar.test.tsx`
Run:
```bash
npm run test:client
```
### Recommended test matrix
| Area | What to verify | Example |
|---|---|---|
| Resource parity | Same key coverage across `en/zh/vi/ko` | Missing key detection in CI |
| Locale fallback | Unknown locales fall back to `en` | `vi-VN` resolves to `vi` |
| Terminology consistency | Canonical terms stay stable | `Agent`/`Subagent` expectations |
| Date/number formatting | Locale-specific output shape | `zh-CN`, `vi-VN`, `ko-KR` formatting |
| Runtime switching | UI rerenders without reload | Sidebar language toggle |
---
## 7) Troubleshooting
| Symptom | Likely cause | Resolution |
|---|---|---|
| UI stays in old language after switch | Cached key or stale component state | Confirm `i18n.changeLanguage(...)` is called and component uses `useTranslation` |
| Unexpected fallback to English | Unsupported locale code | Ensure code normalizes to `en|zh|vi|ko` and key exists in target namespace |
| Missing text on one page | Namespace file key missing | Add key to all language files for that namespace |
| Date/time looks wrong | Locale mapping or timezone parse issue | Verify `getCurrentLocale()` and `parseDate()` behavior |
| Inconsistent term translation | Manual translation drift | Enforce glossary and update locale tests |
---
## 8) Rollout Checklist
```mermaid
gantt
title i18n rollout plan
dateFormat YYYY-MM-DD
axisFormat %m/%d
section Resource Preparation
Lock key inventory :a1, 2026-01-01, 3d
Fill en/zh/vi/ko namespace files :a2, after a1, 5d
section Runtime Integration
Wire detection + persistence :b1, after a2, 2d
Validate sidebar switching :b2, after b1, 2d
Validate locale formatting :b3, after b1, 2d
section Verification
Add/refresh i18n tests :c1, after b2, 3d
Run regression suite :c2, after c1, 2d
section Release
Staged release + monitoring :d1, after c2, 2d
Post-release translation audit :d2, after d1, 3d
```
### Operational checklist
- [ ] Confirm all namespaces exist for `en`, `zh`, `vi`, `ko`
- [ ] Confirm key parity across all locale JSON files
- [ ] Confirm language switching works in collapsed and expanded sidebar modes
- [ ] Confirm fallback behavior for region tags (e.g., `vi-VN`, `zh-CN`)
- [ ] Confirm date/time/currency formatting for all supported languages
- [ ] Confirm client tests pass before release
- [ ] Confirm docs references are updated (`README`, `ARCHITECTURE`, `docs/README`)
---
## References
- `client/src/i18n/index.ts`
- `client/src/components/Sidebar.tsx`
- `client/src/lib/format.ts`
- `client/src/i18n/__tests__/i18n.test.ts`
- `client/src/lib/__tests__/format.test.ts`
docs/LANES.md
+692
View File
@@ -0,0 +1,692 @@
# Lanes: Parallel Agent Work Units
A **lane** is a durable unit of parallel agent work — one working directory, many Claude Code sessions over time. This document describes what lanes are, how to use them, and how the dashboard displays pipeline progress.
## What is a lane?
A lane persists across session restarts, which is what makes a pipeline view possible. When you run a Claude Code session, the dashboard binds it to the lane that owns its working directory (`cwd`) through **longest path-boundary prefix matching**. This allows:
- Multiple sessions on the same directory to feed into one visible lane
- Nested working directories to own their own nested lanes (deeper paths take precedence)
- A session to migrate from one directory to another without losing its lane relationship
**Lanes are keyed by `cwd`, not `session_id.** A session is ephemeral (you run it, it finishes); a lane survives restarts and shows the cumulative progress through a declared pipeline.
## Creating a lane
Create a lane bound to a working directory with `ccam lanes add`:
```bash
ccam lanes add --cwd /path/to/repo --title "My Feature"
```
The lane is now ready to track work. When you run a Claude Code session in that directory (or a subdirectory), the dashboard will bind the session to the lane automatically by matching the cwd.
**Note:** The directory must exist and be an absolute path. A lane with a cwd of `/tmp/wt` will NOT capture `/tmp/wt-sibling` (no partial prefix match) but WILL capture sessions in `/tmp/wt/subdir/` (subdirectories are owned by the parent lane unless a deeper lane exists).
To provision a dashboard-managed git worktree instead of adopting an existing directory, use a source repository:
```bash
ccam lanes add --repo /path/to/repo --title "My Feature" --base main --slug my-feature
```
`--title`, `--base`, and `--slug` are optional. The CLI waits for background provisioning to finish and reports either the ready lane or its failure notes.
## Destructive lane actions
Reset a managed worktree, remove one, or purge the lane's eligible session history with the CLI:
```bash
ccam lanes reset <id> [--force] --yes
ccam lanes remove <id> [--force] --yes
ccam lanes purge <id> --yes
```
Each command first fetches and prints its preflight counts. `reset` and `remove` show `head`, `dirty`, `untracked`, and `unpushed`; `purge` shows `sessions`, `events`, and `tokenRows`. The action refuses to run without `--yes`, and sends those exact facts back as its confirmation. Use `--force` only when the preflight reports unpushed commits. An adopted lane points at a directory you own, so the CLI refuses to `reset` it. `remove` IS allowed for an adopted lane: it drops only the dashboard's record and leaves the directory and its contents untouched.
## Lifecycle
### The two lane kinds
- **`managed`** — a git worktree the dashboard itself created (`ccam lanes add --repo`), rooted under `LANES_ROOT`. The dashboard owns its full lifecycle: it may reset the worktree to its base branch or remove it entirely.
- **`adopted`** — an existing directory you already had (`ccam lanes add --cwd`). The dashboard only ever tracks it; it never resets or deletes that worktree. "Removing" an adopted lane drops the dashboard's record of it and leaves the directory untouched.
### The three independent safety checks
Before `reset` or `remove` touches a worktree, `assertDestroyable` (`server/lib/worktree.js`) runs three checks, all of which must pass:
1. **`lane.kind === "managed"`.** An adopted lane fails immediately with `ENOTMANAGED` — this check alone is what keeps a directory you own off the destroy path.
2. **The lane's `cwd`, resolved with `fs.realpathSync`, must sit inside the resolved `LANES_ROOT` on a path boundary.** `path.relative(root, cwd)` must not start with `..` or be absolute — `EOUTSIDEROOT` otherwise. This guards against a lane row that was hand-edited or drifted to point somewhere else.
3. **The resolved path must actually appear in `git worktree list --porcelain`** for the lane's `source_repo` — `ENOTWORKTREE` otherwise. A directory that merely happens to sit under `LANES_ROOT` but was never registered as a worktree of that repo is not destroyed.
These checks run unconditionally inside `resetWorktree`/`removeWorktree` themselves, not only when a confirmation is present — there is no code path to a git-destructive operation that skips them.
One documented exception, which does not weaken them: when a managed lane's directory has already been deleted by hand, `removeWorktree` takes a **prune path** instead. Checks 1 and 2 still hold (an adopted lane is refused outright; the recorded `cwd` must still be inside `LANES_ROOT`, verified lexically because a vanished path cannot be resolved), and check 3 is what the prune replaces — a worktree git no longer lists needs no removal. That path runs `git worktree prune` plus a safe branch delete against the source repo's bookkeeping and touches no directory at all.
### Each verb
- **`add`** — either adopts an existing directory (`--cwd`, kind `adopted`) or provisions a new managed worktree (`--repo`, kind `managed`, created under `LANES_ROOT` on branch `feat/<slug>`).
- **`clear`** (UI/API action only, no CLI verb) — resets the lane's own bookkeeping (stage, status, gate/CI fields, notes) back to idle, **including the detected columns** (`detected_stage`/`detected_signal`/`detected_at`), since detection is forward-only and a surviving detection would both claim progress the reset tree no longer has and block every later detection. Touches only the `lanes` row; no git operation, no session/event data is deleted.
- **`reset`** (managed lanes only) — checks out the base branch, `git reset --hard` to it, then `git clean -fd` (no `-x`, so **gitignored files such as `node_modules` and `.env` survive**; only untracked-but-not-ignored files are deleted), then recreates the feature branch off the base and clears the lane's bookkeeping. Requires `force: true` when the preflight reports unpushed commits.
- **`remove`** — for a `managed` lane whose directory exists: `git worktree remove --force`, `git worktree prune`, then a safe branch delete, then the lane row is deleted. For a `managed` lane whose directory was deleted by hand: the prune path above (git's stale record and the branch are cleaned up, then the row is deleted) — a `missing` worktree does NOT make a lane unremovable. For an `adopted` lane: only the lane row is deleted (`removeWorktree` is never invoked for `adopted`, so the directory is never touched). The unpushed-commits force-gate only applies when the worktree is actually touched, i.e. `action === "remove" && kind === "managed"` — forgetting an adopted lane needs no `force` even with unpushed commits, since nothing on disk is at risk.
- **`purge`** — deletes the lane's own `sessions`/`events`/`token_usage` rows (excluding the lane's currently-bound or any still-active session), for its own `cwd` and subdirectories. No git operation, no worktree change; not gated on unpushed commits.
### The preflight contract
`GET /api/lanes/:id/preflight?action=reset|remove|purge` (`server/lib/lane-preflight.js`) is read-only and returns:
- **`reset`/`remove`**: `{ action, lane, kind, branch, head, dirty, untracked, unpushed, blocked[], warnings[] }`
- **`purge`**: `{ action, lane, sessions, events, tokenRows, bytesEstimate, activeSessionSkipped }`
`blocked[]` can contain: `adopted` (not a managed worktree), `missing` (the directory is gone), `unreadable` (directory exists but git failed against it), and `unpushed-commits` (unpushed count > 0 — the only one a `force: true` can override).
**Which entries actually prevent the action depends on the action, and the server is the authority.** All of them block `reset`. **None of them blocks `remove`** — the server supports removing an adopted lane (forget the row), a hand-deleted worktree (prune path), and an unreadable one: it attempts `git worktree remove --force`, and if git itself refuses (e.g. a corrupt worktree `.git` pointer, which git validates and rejects even with a second `--force`), it deregisters the worktree directly from the source repo's bookkeeping instead — the directory itself is never touched either way. A surface that greys out confirmation on any `blocked` entry regardless of action makes a documented capability unreachable; `client/src/components/lanes/DestructiveLaneModal.tsx` encodes the per-action set in `HARD_BLOCKERS` and renders the rest as context.
`unpushed` is the count of commits the action would really discard: with remotes, commits on no remote; without remotes, the commits ahead of the lane's `base_branch`, which is this lane's own work. A freshly provisioned worktree in a local-only repository reports `0`, not that repository's whole history — the `no-remote` warning, not an inflated count, is what tells the user nothing is backed up. A lane with no `base_branch` at all (an adopted one) falls back to the total commit count. `warnings[]` holds purely informational facts that never gate the action: currently just `no-remote` (no git remote configured at all — nothing here is backed up remotely, but the action proceeds normally). Both arrays are data, not an exception.
`bytesEstimate` is a **rough estimate**, not a measured disk size: it is derived as `(events + tokenRows) * 512`, a flat per-row byte guess, purely to give the confirmation dialog an order-of-magnitude sense of what purging will reclaim.
Every destructive `POST /api/lanes/:id/:action` must echo back an `expect` object with exactly the fields listed above for that action. The server re-runs preflight at execution time and rejects the request (`400 EEXPECT` if incomplete, `409 ESTALE` if any field has changed since the client read it) rather than trusting a client-supplied count.
### Environment variables
- **`LANES_ROOT`** (default `~/.claude/ccam-lanes`) — the directory managed worktrees are provisioned under; also the boundary that safety check 2 resolves paths against.
- **`LANE_DEAD_SEC`** (default `300`) — seconds of silence before a lane whose stage matches `/watch|poll/i` flips from `active` to `dead` liveness.
- **`LANE_BASE_BRANCH`** (default `main`) — the base branch a new managed worktree resolves against when a request omits `base`. An explicit `--base <branch>` (or request body `base`) always wins over this default.
- **`LANE_BRANCH_PREFIX`** (default `feat/`) — the prefix used to build a new managed worktree's feature branch name (`<prefix><slug>`).
### Limitation: a fresh worktree has no dependencies
`ccam lanes add --repo` runs `git worktree add` only. It does not run `npm install` (or any other package manager), and it does not copy the source repo's untracked local files (`.env`, IDE config, etc.) into the new worktree. A managed lane is ready for `git` immediately but needs its own dependency install and local env setup before a session can run the project inside it — bootstrapping that automatically is out of scope for this feature (see `docs/superpowers/specs/2026-07-28-worktree-lanes-design.md`).
## The Workspace page (`/run`)
The dashboard web UI merges lanes and runs into a single **Workspace** page accessible at `/run` (the old `/lanes` route now redirects here). Top to bottom:
- **Header** — the page title and four counters (`lanes`, `running`, `needs you`, `dead`), plus Add lane. The `needs you` and `dead` counters appear only when they are non-zero, so a quiet header means nothing is waiting on a human.
- **Detail panel** — the selected lane's declared stage, its inferred stage when detection leads, a full-width pipeline map, and a legend naming all five node states plus the dashed-amber inferred treatment.
- **Console** — `RunSetup`, `RunConsole` and `RunHistory` behind a disclosure that **starts collapsed**. Watching lanes is the default posture; driving one is the exception. Collapsing hides the console with CSS and never unmounts it, so a live run keeps its rendered history and scroll position.
- **Lane grid** — one card per lane, 1 column, 2 at `md`, 3 at `xl`. Each card carries the lane id, liveness dot and status, title, declared stage with a progress bar and time-on-stage, the `auto:` chip when detection leads, the kind and CI tags, the working-copy facts from `GET /api/lanes/:id/git`, the needs-you banner, and the action row.
Run history is per lane, queryable via `GET /api/run/history?laneId=<n>`.
The UI operates on a working directory (`cwd`), not a lane id. Starting a run in a `cwd` that no lane owns calls `POST /api/lanes/ensure` first, to idempotently find or adopt a lane for that path; a `cwd` an existing lane already owns is matched from the loaded lane list without a round trip. Either way the run is then started through `POST /api/lanes/:id/start` rather than directly through `POST /api/run`.
### Finding or adopting a lane: `POST /api/lanes/ensure`
The Workspace page opens on a `cwd` and needs a lane to exist for that directory without ever creating a duplicate. The `ensure` endpoint handles this:
```bash
curl -X POST http://localhost:4820/api/lanes/ensure \
-H "Content-Type: application/json" \
-d '{"cwd": "/absolute/path/to/work", "title": "My Project"}'
```
**Response:**
```json
{
"lane": { "id": 5, "cwd": "/absolute/path/to/work", "title": "My Project", ... },
"created": false
}
```
- If a lane already owns the `cwd`, returns that lane with `created: false` and HTTP `200`.
- If the `cwd` is a subdirectory of an existing lane, returns the parent lane that owns it.
- If no lane owns the `cwd`, creates an **adopted** lane and returns it with `created: true` and HTTP `201`.
- A relative or missing `cwd` returns `400 EBADCWD`.
Two concurrent calls for the same path always yield one lane: the database's `UNIQUE` constraint on `lanes.cwd` decides the winner, and the loser re-reads and returns the winning lane.
### Reading a lane's working copy: `GET /api/lanes/:id/git`
Each lane card shows the branch its working copy is actually on, the short
HEAD, that commit's subject, and how much is uncommitted:
```bash
curl http://localhost:4820/api/lanes/5/git
```
```json
{
"available": true,
"branch": "feat/rename-metric",
"head": "9b3e74a",
"subject": "free-text rule mode in the form",
"dirty": 2,
"untracked": 1
}
```
- A `cwd` that is missing, is not a git repository, or makes git fail returns
`{"available": false}` with HTTP **200**. A lane pointing at a plain directory
is a normal state, not a fault, and the card simply renders without a git row.
- A detached checkout reports the literal `HEAD` that git returns, rather than a
prettier invention.
- An unknown lane id is a `404`.
This is **not** folded into `GET /api/lanes` on purpose: it shells out to git
three times, and the lane list is polled and re-broadcast on every hook-driven
`lane_update`. Putting it there would fire a burst of subprocesses behind every
tool call an agent makes. The browser fetches it per card instead, on mount and
every 30 seconds, and a failed fetch is silent.
### Running and releasing lanes
The Workspace page starts runs through `POST /api/lanes/:id/start`, which accepts the same `mode` and `effort` parameters as `POST /api/run`:
```bash
curl -X POST http://localhost:4820/api/lanes/5/start \
-H "Content-Type: application/json" \
-d '{"mode": "conversation", "effort": "medium"}'
```
**A finished run releases its lane.** When a spawned Claude Code process exits (normally, non-zero, killed, or never spawned), the lane's `run_id` is cleared, its `status` returns to `idle`, and the change is broadcast as a `lane_update` WebSocket message. This prevents lanes from sitting at `running` with a dead child.
Runs started through a lane are recorded with the lane's id and are history-queryable:
```bash
curl http://localhost:4820/api/run/history?laneId=5
```
## Viewing lanes
List all lanes with their current status:
```bash
ccam lanes
```
This shows each lane's:
- **stage**: the declared stage (e.g., `plan`, `implement`, `review`, `done`)
- **status**: `running` (a session is active), `idle` (no active session), or `provisioning`
- **liveness**: `active` (recently updated), `dead` (silent for too long), or `idle` (not expected to be running)
- **progress**: percentage of the pipeline completed (0–100%)
- **needs_action**: a notification if the session is blocked waiting for input
A lane row also gets a `⇢ detected:<stage>` suffix when the server's inferred
stage (see "Stage detection" below) leads the declared `stage` shown in the
row — the same lead/behind comparison the web UI's `auto: <stage>` chip uses,
so the terminal and the browser never disagree about which one is the
headline. Nothing is printed when the declaration already leads or matches.
The dashboard web UI shows each lane as a card in a grid, with the selected lane's pipeline map visualized above.
## Pipeline stages and the five node states
A lane moves through stages defined in a **pipeline template** (see "Custom pipeline templates" below). The default pipeline has eight stages: `intake`, `plan`, `implement`, `tests`, `review`, `gate`, `ship`, and `done`.
The dashboard renders every node in the pipeline in one of five **states**:
| State | Color | Meaning |
|---|---|---|
| **failed** | Red | The stage recorded result `"fail"` — work was attempted and rejected |
| **current** | Blue | The lane's current declared stage |
| **done** | Green | The stage was recorded with evidence (an artifact, not just a claim) |
| **passed-no-evidence** | Amber | The stage was recorded without evidence, OR was skipped (a later stage was reached) |
| **pending** | Gray | Not reached yet |
### Why `passed-no-evidence` is amber, not green
A stage in this state was **claimed but not proven**. Two scenarios:
1. **Declared but no artifact left.** A skill called `ccam stage review` without `--evidence`, claiming the stage happened but leaving no proof that work occurred.
2. **Implicitly skipped.** The lane jumped from `plan` to `implement` (to stage index 2), leaving `intake` (index 0) in `passed-no-evidence` state.
In both cases, the stage is not **done** — it's incomplete or untested. Amber signals that the agent claimed to have done this work but the dashboard has no evidence to show it was actually completed. This matters in a pipeline: if your gate checks `done` not `passed-no-evidence`, a lane stuck in amber will block until evidence is provided.
## Reporting a stage
When a skill reaches a significant milestone, it reports the current stage to the dashboard using `ccam stage`:
```bash
ccam stage implement --evidence "Schema created, migrations run"
```
This command:
- **Declares the stage** the work is now at
- **Records evidence** (optional, but highly encouraged) — a short text description of what was completed
- **Moves the progress bar** forward in the pipeline
- **Broadcasts to the UI** so connected dashboards see the update in real-time
### Full command signature
```bash
ccam stage <stage> [--lane <id>] [--cwd <path>] [--status <s>] [--evidence <text>] [--note <text>] [--result pass|fail]
```
**Parameters:**
- `<stage>` (required): the pipeline stage name or alias. The default pipeline aliases are:
- `intake` / `assigned` / `claimed` / `start`
- `plan` / `planning` / `brainstorm` / `design`
- `implement` / `implementing` / `coding` / `build`
- `tests` / `testing` / `unit` / `gates` / `pre-push-gate`
- `review` / `reviewing` / `code-review` / `self-review`
- `gate` / `verify` / `verification` / `sr-gate` / `gate-blocked`
- `ship` / `pr` / `pr-open` / `publishing` / `commit` / `push`
- `done` / `complete` / `completed` / `merged`
- `--lane <id>` (optional): the numeric lane ID. If omitted, the command resolves the lane by `cwd`.
- `--cwd <path>` (optional): working directory to match against a lane's cwd. If omitted, uses the current working directory. Useful when calling from outside the repo.
- `--status <s>` (optional): the lane's overall status (`running`, `idle`, `provisioning`). Allows bundling a status update with the stage report.
- `--evidence <text>` (optional): a short description of what was completed at this stage. Moves the node from `passed-no-evidence` to `done`.
- `--note <text>` (optional): a longer note or comment, separate from evidence.
- `--result pass|fail` (optional): explicitly mark the stage as passed or failed. Ignored if `result` was already set; set once, never overwritten by a re-report.
### Example: a skill reporting progress
A `build` skill might call:
```bash
ccam stage implement --evidence "Built and ran tests, 42 tests passing" --status running
```
Output:
```
lane #5 → implement (66%)
```
A later call reports completion:
```bash
ccam stage tests --evidence "Pre-push gate: all linters passing" --result pass
```
## Custom pipeline templates
The default pipeline is suitable for feature work: `intake` → `plan` → `implement` → `tests` → `review` → `gate` → `ship` → `done`. For different workflows (data pipelines, research phases, deployment stages), you can provide custom templates.
### Where templates come from
1. **Built-in:** `server/data/pipelines/default.json` (always available)
2. **User override:** directory specified by the `DASHBOARD_PIPELINES_DIR` environment variable
If both exist, **files in the user directory override built-ins** with the same `id`.
### Custom template format
Create a JSON file in your custom pipeline directory (e.g., `~/.ccam/pipelines/ml-training.json`):
```json
{
"id": "ml-training",
"name": "ML Training Pipeline",
"nodes": [
{ "id": "data", "label": "data", "icon": "📊", "gate": false, "aliases": ["intake"] },
{ "id": "preprocess","label": "preprocess","icon": "⚙️", "gate": false, "aliases": ["prepare", "clean"] },
{ "id": "train", "label": "train", "icon": "🧠", "gate": false, "aliases": ["training", "fit"] },
{ "id": "eval", "label": "eval", "icon": "📈", "gate": true, "aliases": ["evaluate", "metrics"] },
{ "id": "deploy", "label": "deploy", "icon": "🚀", "gate": false, "aliases": ["ship", "release"] }
]
}
```
**Schema:**
- **id** (string, required): unique identifier for this template
- **name** (string, optional): display name shown in the UI
- **nodes** (array, required): list of pipeline stages, in order
- **id** (string): stage identifier (used in `ccam stage`)
- **label** (string, optional): display name in the UI; defaults to `id` if omitted
- **icon** (string, optional): unicode emoji or single character shown before the stage name
- **gate** (boolean, optional): `true` if this stage blocks unless explicitly passed; defaults to `false`
- **aliases** (array, optional): alternative names for the stage (e.g., `["training", "fit"]` lets you call `ccam stage training`)
Then set the environment variable:
```bash
export DASHBOARD_PIPELINES_DIR=~/.ccam/pipelines
```
When you create a new lane, you can optionally specify which pipeline to use:
```bash
ccam lanes add --cwd /path/to/ml-repo --title "Training run #1" --pipeline ml-training
```
(The pipeline is selected at creation time. Lanes created without `--pipeline` default to the `default` pipeline.)
## Stage detection
Besides the stage a skill explicitly declares with `ccam stage`, the dashboard
also **infers** a stage from the tool events every hook already delivers
(`POST /api/hooks/event`). This is a best-effort heuristic for
un-instrumented sessions that never call `ccam stage` — it fills in a lane's
pipeline map without requiring every skill to be rewritten.
### The evidence boundary
**An inferred stage is never evidence and never renders as done.** Green
(`done`) requires a stage the agent explicitly declared *with evidence*; an
inference can reach dashed amber at most — visually distinct from both solid
green `done` and solid amber `passed-no-evidence`
(`client/src/components/lanes/PipelineMap.tsx`). This boundary exists because
detection is a guess from tool *names* and a handful of string fields, not
proof that the agent actually finished the stage — a `Bash` call matching
`npm run test` could be a flaky retry, not a passing suite. Treating it as
proof would let a lane's pipeline map lie about what actually happened.
The node the agent *declared* itself on is never flagged as detected, so it
keeps its blue `current` ring — including when the declaration came in through
an alias (`ccam stage coding` resolves to the `implement` node, and `stages` is
keyed by the raw declared word, not the node id).
The same rule governs `ccam lanes`: the inferred stage is printed only when it
leads the declared one (see "Viewing lanes" below), and `LaneCard.tsx`'s
`detectionLeadsDeclaration` shows its `auto: <stage>` chip under the identical
condition — the terminal and the browser never disagree about which one is
the headline.
### Which lane a signal is credited to
A hook's `cwd` is the **session's** directory, not the directory the command ran
in. Measured on a real install: 325 of 400 events carried the session's cwd
while the edits and test runs happened in another repo reached with
`cd <other> && …`. The lane doing the work detected nothing; the lane the
terminal started in absorbed all of it.
So `resolveLaneForWork` (`server/lib/lanes.js`) prefers a lane named by the
tool's own input — the `file_path` being edited, an absolute path a command
`cd`s into — and falls back to the session's `cwd` when the input names no other
lane. Deepest match wins, same rule as `resolveLaneByCwd`.
Only the stage inference follows the work. `session_id` and `needs_action` stay
on the session's own lane, because those really are session-scoped facts: the
session is bound to the directory it started in even while it operates
elsewhere.
### What signals are read
`server/lib/stage-detect.js` inspects every incoming hook's `tool_name` and
`tool_input` (`touchLaneFromHook` in `server/routes/hooks.js` calls `detect()`
on each hook). Detection runs ahead of the lane's `needs_action`/`session_id`
bookkeeping — that block returns early when there is nothing to patch, so
detection could not live after it — and therefore sits in a try/catch of its
own: a throwing `recordDetection` (`ENOLANE` on a lane deleted mid-hook,
`SQLITE_BUSY`) must never cost the bookkeeping that predates it. Only these
`tool_input` fields are read — never `old_string`/`new_string` or other editor payloads that can
contain whole code blocks:
- `command`
- `file_path`
- `skill`
- `prompt`
- `pattern`
- `description`
- `subagent_type`
The signal shown to the user is the span the rule's regex actually matched,
plus 24 characters of context on each side, with an ellipsis marking either end
that was trimmed. A shell line like
`cd /home/very/long/path && npm run test:server 2>&1` therefore reports
`` `…path && npm run test:server 2>&1` `` rather than the whole command. A rule
with no `match` fired on the tool name alone and has no span, so it falls back
to the flattened input. Either way the result is capped at 120 characters
(collapsing whitespace first) and stored as `detected_signal` — this is why a
lane's tooltip sometimes shows a file path or a skill name rather than a shell
command: it's whichever of the fields above the matching rule's `tool` carried.
### Detection expires
Forward-only would otherwise park a lane at the highest stage it ever touched:
a session that ran the test suite reaches `tests` and can never show
`implement` again, even while the agent is back editing code.
So a standing detection holds the forward-only floor only while it is fresh.
Once `detected_at` is older than `DETECTION_TTL_MS` (default `300000`, i.e. 5
minutes), the comparison against `detected_stage` is skipped and the new signal
wins regardless of direction. A `detected_at` that is NULL or unparseable counts
as stale — an unknown age cannot be proven fresh.
Five minutes, not thirty. A session cycles implement → tests → ship → implement
→ tests within one sitting, and a thirty-minute hold pinned the lane at the
furthest stage it ever touched: one push left a lane reading `ship` while the
agent was demonstrably back to running tests. Five minutes is still far longer
than a burst of tool calls, so the anti-flap property is unaffected.
Two things the expiry deliberately does NOT do:
- **It does not weaken declared-wins.** An agent's own claim never expires: a
lane declared at `review` still refuses an `implement` detection, stale
standing detection or not.
- **It does not let inference render `done`.** Everything in "An inferred stage
is never evidence" above still holds. The TTL changes which detection is
current, never what a detection is worth.
Set `DETECTION_TTL_MS` in the server's environment to change the window; it is
read per call, so no restart is needed for a test harness that sets it.
### Where the rules live
Detection rules are not code — they live in the pipeline template itself, as
a `detect` array on each node. The built-in `default.json`
(`server/data/pipelines/default.json`) ships these rules today, node by node:
| Node | Detect rules |
|---|---|
| `intake` | none — see below |
| `plan` | `Skill` matching `brainstorming\|writing-plans`; `Write` matching `docs/.*plan.*\.md` |
| `implement` | `Edit` (any); `Write` to any path NOT under a `docs/` directory (`^(?!.*(?:^\|/)docs/)`) |
| `tests` | `Bash` matching `\b(npm (run )?test\|pytest\|vitest\|jest\|go test\|cargo test\|node\s+--test)\b` |
| `review` | `Skill` matching `code-review\|requesting-code-review`; `Bash` matching `\bgh\b[^;&\|]*\bpr diff\b` |
| `gate` | none — see below |
| `ship` | `Bash` matching a `git push` **invocation** — `git`, then option groups, then `push` as the first non-flag token — or `gh … pr create` |
| `done` | none — see below |
**`git diff` is deliberately NOT a review signal.** It was, and measured on a
real session four `git diff --stat` runs — used to verify a generated file —
outranked 81 test runs and 30 edits and pinned the lane at `review`, because
`detect()` takes the LAST matching node and `recordDetection` is forward-only.
Reading a diff is constant background activity, so it carries no stage
information. `gh pr diff` stays: that one really is a review act.
**The `ship` pattern matches a git *invocation*, not the word `push`.** Two
weaker forms were tried and both failed against real commands:
- `git[^;&|]*push` (same shell segment) **misses** this repo's own push, because
`git -c credential.helper='!f() { … }; f' push` puts a `;` between the two
inside a quoted value. Six real pushes went undetected that way.
- `git[\s\S]*push` (anywhere after `git`) **over-matches**, and did so on a live
lane within minutes: `git log --oneline -1 && echo "… push …"` pinned the lane
at `ship`. Since `ship` sits near the end of the pipeline and detection is
forward-only, that single over-read stuck — the same failure `git diff` caused
for `review`.
So the pattern consumes option groups explicitly, treating a quoted option value
as one unit, and then requires `push` to be the first non-flag token. That
accepts `git push`, `git -c core.hooksPath=/dev/null push` and the credential
-helper form, while rejecting `git log … "push"`, `git commit -m "don't push"`,
`docker push`, and `npm run push-docs`.
`intake`, `gate`, and `done` deliberately have no rules. `intake` is where a
lane starts — there is no tool event that means "just claimed," so there is
nothing to detect. `gate` and `done` are explicitly out of scope for
inference (see the task's "Out of scope" list): a gate's pass/fail is a human
or skill decision, and `done` is the one state detection must never reach on
its own — inferring either would let a heuristic assert an outcome instead of
observing activity.
To add a rule, drop a template into the directory named by the
`DASHBOARD_PIPELINES_DIR` environment variable (see "Custom pipeline
templates" above) with a `detect` array on the node you want to match:
```json
{ "id": "implement", "label": "implement", "detect": [
{ "tool": "Bash", "match": "docker build" }
] }
```
`match` is a JavaScript regular expression tested against the flattened
`tool_input` string; omit `match` entirely to match on `tool` name alone. An
invalid regex is skipped rather than blocking hook ingestion.
### Forward-only, write-on-change
Detection never rewinds a lane and never fights a declaration:
- **Forward-only.** A detection only overwrites `detected_stage` when its
node index is strictly greater than the current `detected_stage`'s index —
reading a file after editing it must not drag a lane back to `plan`.
- **Declared beats detected.** A detection is written only when the lane's
*declared* stage index is strictly less than the detection's. A lane
already declared at `review` ignores an `implement` detection outright.
- **Write-on-change only.** The database write (and the `lane_update`
WebSocket broadcast) happens only when `recordDetection` reports an actual
change. A real install saw 29,470 `Bash` events alone from one project —
writing to the DB and broadcasting on every matching hook, rather than only
on a genuine advance, would turn routine ingestion into a DB write and a WS
frame per tool call.
Detection writes only `detected_stage`, `detected_signal`, and `detected_at`
(`server/lib/lanes.js`'s `recordDetection`) — it **never** writes `lanes.stage`,
the declared field. The agent's own declaration is the only path to a stage
being recorded as `done`.
## Liveness: detecting dead lanes
The dashboard watches for silent sessions using two rules:
### The liveness rule
A lane's **liveness** state (shown as `active`, `idle`, or `dead`) depends on:
- **Lane status** (`running`, `idle`, `provisioning`)
- **Stage name** (does it match `watch|poll` regex?)
- **Age** (how long since the last event?)
**Idle lanes are never dead.** If a lane's status is `idle`, it's at rest by design — no session is expected to be running. A session might connect later.
**Watching sessions die after silence.** When a lane's stage matches the pattern `/watch|poll/i` (e.g., stage `watch-for-updates`), the session is expected to emit heartbeat events. If it goes silent for longer than `LANE_DEAD_SEC` (default 300 seconds), the dashboard marks it as `dead`. A watcher with no events is a dead process; an idle session with no events is just idle.
### Configuring the timeout
Set the environment variable to change when a watcher is considered dead:
```bash
export LANE_DEAD_SEC=600 # 10 minutes instead of 5
```
## Lane actions
The web UI and CLI provide these actions on a lane:
### start
Launch a new Claude Code session bound to the lane. If the lane has a recorded `session_id` from a previous run, you can resume it instead with `--resume`.
```bash
curl -X POST http://localhost:4820/api/lanes/5/start \
-H "Content-Type: application/json" \
-d '{"prompt": "continue the work", "resume": true}'
```
`mode` accepts the same two values as `POST /api/run` — `"conversation"` (the
default: multi-turn, `message` keeps working) or `"headless"` (one shot, the
prompt goes in argv and the process exits when the turn finishes). Unlike
`POST /api/run`, an unknown value is refused with `400 EBADMODE` rather than
silently treated as a conversation.
**A finished run releases its lane.** When the child truly exits — normally,
non-zero, killed, or never spawned at all — the lane's `run_id` is cleared, its
status returns to `idle`, and the change is broadcast as `lane_update`. So a
lane never sits at `running` behind a dead run, and `message` reports
`409 ENORUN` instead of targeting one. Runs started this way are recorded with
the lane's id and are listable via `GET /api/run/history?laneId=<n>`.
Currently the `prompt` field does not populate the input field in the UI (see "Known limitations" below).
### message
Send input to a running session. The lane must have a live `run_id` (active session). The `needs_action` flag clears automatically when the message is delivered.
```bash
curl -X POST http://localhost:4820/api/lanes/5/message \
-H "Content-Type: application/json" \
-d '{"text": "approved, proceed"}'
```
### stop
Terminate the running session. If the lane has no live run, this is a no-op (no error).
```bash
curl -X POST http://localhost:4820/api/lanes/5/stop
```
### clear
Reset the lane's pipeline progress, stages, notes, and inferred stage. The lane itself persists; only its recorded history is cleared.
```bash
curl -X POST http://localhost:4820/api/lanes/5/clear
```
### remove
Delete the lane entirely, including all its history. **Requires confirmation** (`confirm: true`):
```bash
curl -X POST http://localhost:4820/api/lanes/5/remove \
-H "Content-Type: application/json" \
-d '{"confirm": true}'
```
**Actions gated behind confirmation:** `remove` requires the `confirm` flag to prevent accidental deletion.
## Orchestration: what CCAM does NOT do
**CCAM does not chain, queue, retry, or evaluate gates.**
The driving Claude Code session is in control. Here's what that means:
- **No automatic chaining:** The lane doesn't automatically move from one stage to the next. A skill declares the stage explicitly with `ccam stage`.
- **No queue:** There is no work queue. When you run a session, it runs immediately; no waiting for dependencies or prior stages.
- **No retry logic:** If a stage fails (marked with `--result fail`), the lane stays at that stage. The next session must be told to reattempt it (the skill logic decides).
- **No gate evaluation:** A gate stage (marked `gate: true` in the template) doesn't block the UI, stop a session, or prevent a transition. It's just a visual marker. The skill must check the gate condition and decide whether to proceed.
**Example:** In a feature pipeline, if the `tests` stage is a gate, and tests fail:
1. A skill calls `ccam stage tests --result fail --evidence "5 tests failing"`
2. The UI shows `tests` in red (failed state)
3. The next session runs and a human/skill must explicitly decide: re-run tests, or skip to ship?
4. If re-running: `ccam stage tests --evidence "2 tests still failing"`
5. If skipping: `ccam stage ship --evidence "manual gate override"`
The **dashboard shows the history; the driving session makes the decisions.**
## Known limitations
### Lanes page snapshot
The Lanes screen currently shows only the empty state in the dashboard's design. Full design and implementation of the populated view (multiple lanes, interactions, pipeline map) is deferred to a follow-up.
### Lane card UI
The lane card UI currently has no prompt/message input field. When you click `start`, it opens a promptless conversation run in Claude Code (no initial input). The `message` action exists server-side and via the API but is not exposed in the web UI yet. To send messages, use the API directly or the CLI.
## Related commands
```bash
ccam lanes # List all lanes
ccam lanes add --cwd <path> --title <text> # Adopt an existing directory
ccam lanes add --repo <path> [--title <text>] [--base <branch>] [--slug <slug>]
ccam lanes reset|remove|purge <id> [--force] --yes
ccam stage <stage> [--evidence "..."] # Report the current stage
```
## See also
- `server/lib/lanes.js` — lane storage and lifecycle
- `server/lib/worktree.js` — git worktree creation/reset/removal and the three-check destroy guard
- `server/lib/lane-preflight.js` — the read-only preflight facts behind reset/remove/purge
- `server/lib/pipelines.js` — pipeline templates and node state derivation
- `server/lib/stage-detect.js` — the stage-detection matcher, field allowlist, and signal cap
- `server/routes/lanes.js` — REST API surface
- `bin/ccam.js` — CLI commands for lanes and stage reporting
- `server/routes/hooks.js` — hook-to-lane binding by cwd, and where detection is invoked
docs/MCP.md
+822
View File
@@ -0,0 +1,822 @@
# MCP Integration Guide
Model Context Protocol (MCP) server integration for programmatic dashboard access.
---
## Table of Contents
- [Overview](#overview)
- [MCP Architecture](#mcp-architecture)
- [Setup & Installation](#setup--installation)
- [Available Tools](#available-tools)
- [Client Configuration](#client-configuration)
- [Usage Examples](#usage-examples)
- [Tool Reference](#tool-reference)
- [Error Handling](#error-handling)
- [Performance](#performance)
- [Development](#development)
- [Deployment](#deployment)
---
## Overview
The Agent Dashboard MCP server exposes dashboard functionality as tools that can be used by Claude Desktop, Cline, and other MCP clients.
```mermaid
graph TB
subgraph "MCP Clients"
Claude[Claude Desktop]
Cline[Cline IDE]
Custom[Custom MCP Client]
end
subgraph "MCP Server"
MCPServer[Agent Dashboard<br/>MCP Server]
Tools[Tool Registry]
end
subgraph "Dashboard API"
API[Express API<br/>:4820]
DB[(SQLite DB)]
end
Claude -->|stdio| MCPServer
Cline -->|stdio| MCPServer
Custom -->|stdio| MCPServer
MCPServer --> Tools
Tools -->|HTTP| API
API --> DB
style MCPServer fill:#0f766e
style API fill:#3B82F6
style DB fill:#003B57,color:#fff
```
**Key Benefits:**
- 🤖 **AI-Native** - Claude can query sessions, agents, and costs
- 🔌 **Standardized** - Works with any MCP-compatible client
- 🚀 **Easy Setup** - One-command installation
- 🔒 **Local-First** - No cloud dependencies
---
## MCP Architecture
### MCP Protocol Flow
```mermaid
sequenceDiagram
participant Client as MCP Client<br/>(Claude Desktop)
participant Server as MCP Server
participant API as Dashboard API
participant DB as SQLite
Client->>Server: Initialize connection
Server-->>Client: Server info + capabilities
Client->>Server: List tools
Server-->>Client: Tool definitions
Client->>Server: Call tool (get_sessions)
Server->>API: GET /api/sessions
API->>DB: Query sessions
DB-->>API: Results
API-->>Server: JSON response
Server-->>Client: Tool result
Client->>Client: Process result
```
### MCP Server Structure
```mermaid
graph TB
subgraph "MCP Server (mcp/)"
Index[index.ts<br/>Entry point]
Config[config/app-config.ts<br/>Configuration]
Tools[tools/<br/>Tool implementations]
Types[types.ts<br/>TypeScript types]
end
subgraph "Tool Categories"
Sessions[Session Tools<br/>get_sessions, get_session]
Agents[Agent Tools<br/>get_agents, get_agent]
Pricing[Pricing Tools<br/>get_pricing, create_rule]
Stats[Stats Tools<br/>get_stats]
end
Index --> Config
Index --> Tools
Tools --> Sessions
Tools --> Agents
Tools --> Pricing
Tools --> Stats
style Index fill:#0f766e
style Tools fill:#10B981
```
---
## Setup & Installation
### Prerequisites
- Node.js >= 18.0.0
- Dashboard server running on `localhost:4820`
### Installation
```bash
# Install MCP server dependencies
npm run mcp:install
# Build MCP server
npm run mcp:build
# Test MCP server
npm run mcp:start
```
### Directory Structure
```
mcp/
├── src/
│ ├── index.ts # MCP server entry point
│ ├── config/
│ │ └── app-config.ts # Configuration + validation
│ ├── tools/
│ │ ├── sessions.ts # Session-related tools
│ │ ├── agents.ts # Agent-related tools
│ │ ├── pricing.ts # Pricing management tools
│ │ └── stats.ts # Statistics tools
│ └── types.ts # TypeScript type definitions
│
├── dist/ # Compiled JavaScript (gitignored)
├── package.json
├── tsconfig.json
└── README.md
```
---
## Available Tools
### Tool Catalog
```mermaid
graph TB
subgraph "Session Management"
GetSessions[get_sessions<br/>List all sessions]
GetSession[get_session<br/>Get session details]
end
subgraph "Agent Management"
GetAgents[get_agents<br/>List session agents]
GetAgent[get_agent<br/>Get agent details]
GetTools[get_tools<br/>List agent tools]
end
subgraph "Pricing"
GetPricing[get_pricing<br/>List pricing rules]
CreateRule[create_pricing_rule<br/>Add custom rule]
DeleteRule[delete_pricing_rule<br/>Remove rule]
end
subgraph "Statistics"
GetStats[get_stats<br/>Dashboard statistics]
end
style GetSessions fill:#3B82F6
style GetAgents fill:#10B981
style GetPricing fill:#F59E0B
style GetStats fill:#8B5CF6
```
---
## Client Configuration
### Claude Desktop
Add to `~/Library/Application Support/Claude/claude_desktop_config.json` (macOS):
```json
{
"mcpServers": {
"agent-dashboard": {
"command": "node",
"args": ["/path/to/agent-dashboard/mcp/dist/index.js"],
"env": {
"MCP_DASHBOARD_BASE_URL": "http://localhost:4820"
}
}
}
}
```
**Linux:**
```
~/.config/Claude/claude_desktop_config.json
```
**Windows:**
```
%APPDATA%\Claude\claude_desktop_config.json
```
### Cline (VS Code Extension)
Add to VS Code settings (`.vscode/settings.json`):
```json
{
"cline.mcpServers": {
"agent-dashboard": {
"command": "node",
"args": ["/path/to/agent-dashboard/mcp/dist/index.js"],
"env": {
"MCP_DASHBOARD_BASE_URL": "http://localhost:4820"
}
}
}
}
```
### Environment Variables
| Variable | Default | Description |
|----------|---------|-------------|
| `MCP_DASHBOARD_BASE_URL` | `http://localhost:4820` | Dashboard API base URL |
**URL Validation:**
```mermaid
graph TB
URL[MCP_DASHBOARD_BASE_URL] --> Validate{Valid?}
Validate -->|Invalid Protocol| Error1[Throw: Only http/https allowed]
Validate -->|Invalid Hostname| Error2[Throw: Only loopback allowed]
Validate -->|Valid| Accept[Accept URL]
subgraph "Valid Hostnames"
H1[127.0.0.1]
H2[localhost]
H3[::1]
end
Accept --> H1 & H2 & H3
style Error1 fill:#EF4444
style Error2 fill:#EF4444
style Accept fill:#10B981
```
---
## Usage Examples
### Example 1: List Recent Sessions
**User Prompt:**
> "Show me the 5 most recent Claude Code sessions"
**Tool Call:**
```json
{
"name": "get_sessions",
"arguments": {
"limit": 5
}
}
```
**Response:**
```json
{
"sessions": [
{
"session_id": "sess_abc123",
"model": "claude-sonnet-4",
"status": "active",
"total_cost": 1.23,
"agent_count": 3,
"tool_count": 12,
"created_at": "2024-03-18T12:00:00Z"
}
]
}
```
---
### Example 2: Analyze Session Cost
**User Prompt:**
> "What was the cost breakdown for session sess_abc123?"
**Tool Sequence:**
```mermaid
sequenceDiagram
participant User
participant Claude
participant MCP as MCP Server
participant API as Dashboard API
User->>Claude: Analyze session cost
Claude->>MCP: get_session(sess_abc123)
MCP->>API: GET /api/sessions/sess_abc123
API-->>MCP: Session data
MCP-->>Claude: Session result
Claude->>MCP: get_agents(sess_abc123)
MCP->>API: GET /api/sessions/sess_abc123/agents
API-->>MCP: Agents data
MCP-->>Claude: Agents result
Claude->>User: Analysis:<br/>Total: $1.23<br/>3 agents:<br/>- Main: $0.85<br/>- Explore: $0.25<br/>- Task: $0.13
```
---
### Example 3: Create Custom Pricing Rule
**User Prompt:**
> "Add a pricing rule for my-custom-model with input $5/1M and output $20/1M"
**Tool Call:**
```json
{
"name": "create_pricing_rule",
"arguments": {
"pattern": "my-custom-model",
"input_cost_per_1m": 5.0,
"output_cost_per_1m": 20.0
}
}
```
**Response:**
```json
{
"rule": {
"id": 10,
"pattern": "my-custom-model",
"input_cost_per_1m": 5.0,
"output_cost_per_1m": 20.0,
"created_at": "2024-03-18T14:30:00Z"
}
}
```
---
## Tool Reference
### get_sessions
List all sessions with optional filters.
**Input Schema:**
```typescript
{
limit?: number; // Max sessions to return (1-1000)
status?: 'active' | 'completed';
}
```
**Output Schema:**
```typescript
{
sessions: Session[];
total: number;
}
interface Session {
session_id: string;
model: string;
status: 'active' | 'completed';
total_cost: number;
agent_count: number;
tool_count: number;
created_at: string;
updated_at: string;
}
```
---
### get_session
Get single session details.
**Input Schema:**
```typescript
{
session_id: string; // Required
}
```
**Output Schema:**
```typescript
{
session: Session;
}
```
**Errors:**
- `404` - Session not found
---
### get_agents
List agents for a session.
**Input Schema:**
```typescript
{
session_id: string; // Required
}
```
**Output Schema:**
```typescript
{
agents: Agent[];
}
interface Agent {
agent_id: string;
session_id: string;
agent_type: string;
status: 'running' | 'completed' | 'failed';
current_tool: string | null;
input_tokens: number;
output_tokens: number;
cost: number;
tool_count: number;
created_at: string;
updated_at: string;
}
```
---
### get_agent
Get single agent details.
**Input Schema:**
```typescript
{
agent_id: string; // Required
}
```
**Output Schema:**
```typescript
{
agent: Agent;
}
```
**Errors:**
- `404` - Agent not found
---
### get_tools
List tool executions for an agent.
**Input Schema:**
```typescript
{
agent_id: string; // Required
}
```
**Output Schema:**
```typescript
{
tools: ToolExecution[];
}
interface ToolExecution {
id: number;
agent_id: string;
tool_name: string;
duration_ms: number;
success: boolean;
error_message: string | null;
created_at: string;
}
```
---
### get_pricing
List pricing rules.
**Input Schema:**
```typescript
{} // No parameters
```
**Output Schema:**
```typescript
{
rules: PricingRule[];
}
interface PricingRule {
id: number;
pattern: string;
input_cost_per_1m: number;
output_cost_per_1m: number;
is_default: boolean;
created_at: string;
}
```
---
### create_pricing_rule
Create custom pricing rule.
**Input Schema:**
```typescript
{
pattern: string; // Model pattern
input_cost_per_1m: number; // USD per 1M input tokens
output_cost_per_1m: number; // USD per 1M output tokens
}
```
**Output Schema:**
```typescript
{
rule: PricingRule;
}
```
**Errors:**
- `400` - Invalid input
- `409` - Pattern already exists
---
### delete_pricing_rule
Delete pricing rule.
**Input Schema:**
```typescript
{
pattern: string; // Pattern to delete
}
```
**Output Schema:**
```typescript
{
deleted: true;
}
```
**Errors:**
- `404` - Pattern not found
- `403` - Cannot delete default rule
---
### get_stats
Get dashboard statistics.
**Input Schema:**
```typescript
{} // No parameters
```
**Output Schema:**
```typescript
{
total_sessions: number;
active_sessions: number;
total_agents: number;
total_tools: number;
total_cost: number;
avg_session_cost: number;
}
```
---
## Error Handling
### Error Response Format
```typescript
interface MCPError {
code: string;
message: string;
details?: any;
}
```
### Error Codes
| Code | Description | Resolution |
|------|-------------|------------|
| `INVALID_INPUT` | Invalid tool arguments | Check input schema |
| `API_ERROR` | Dashboard API error | Check server is running |
| `NOT_FOUND` | Resource not found | Verify ID exists |
| `TIMEOUT` | Request timeout | Increase timeout, check network |
| `CONFIG_ERROR` | Invalid configuration | Check `MCP_DASHBOARD_BASE_URL` |
### Error Handling Flow
```mermaid
graph TB
Request[Tool Request] --> Validate{Input<br/>Valid?}
Validate -->|No| Error1[Return INVALID_INPUT]
Validate -->|Yes| API[Call API]
API --> Success{HTTP 200?}
Success -->|No| HTTPCode{Status Code}
HTTPCode -->|404| Error2[Return NOT_FOUND]
HTTPCode -->|500| Error3[Return API_ERROR]
HTTPCode -->|Timeout| Error4[Return TIMEOUT]
Success -->|Yes| Parse[Parse JSON]
Parse --> Result[Return Result]
style Error1 fill:#EF4444
style Error2 fill:#F59E0B
style Error3 fill:#EF4444
style Error4 fill:#EF4444
style Result fill:#10B981
```
---
## Performance
### Tool Execution Time
```mermaid
graph TB
subgraph "Execution Breakdown"
Validation[Input Validation<br/>~1ms]
HTTP[HTTP Request<br/>~20ms]
API[API Processing<br/>~5ms]
DB[Database Query<br/>~5ms]
Response[Response Serialization<br/>~5ms]
end
Total[Total: ~36ms]
Validation --> HTTP
HTTP --> API
API --> DB
DB --> Response
Response --> Total
style Total fill:#10B981
```
**Performance Benchmarks:**
| Tool | Avg Time | 95th Percentile |
|------|----------|-----------------|
| `get_sessions` | 25ms | 40ms |
| `get_session` | 15ms | 25ms |
| `get_agents` | 20ms | 35ms |
| `get_tools` | 30ms | 50ms |
| `get_pricing` | 10ms | 20ms |
| `get_stats` | 40ms | 60ms |
---
## Development
### Building from Source
```bash
# Install dependencies
cd mcp && npm install
# Build TypeScript
npm run build
# Watch mode (auto-rebuild)
npm run dev
# Type checking
npm run typecheck
```
### Adding New Tools
```typescript
// mcp/src/tools/my-tool.ts
import { z } from 'zod';
import { fetchFromAPI } from '../utils';
export const myTool = {
name: 'my_tool',
description: 'Description of what this tool does',
inputSchema: z.object({
param1: z.string(),
param2: z.number().optional()
}),
async execute(args: { param1: string; param2?: number }) {
const response = await fetchFromAPI(`/api/my-endpoint?param=${args.param1}`);
return response.data;
}
};
```
Register in `index.ts`:
```typescript
import { myTool } from './tools/my-tool';
server.setRequestHandler(CallToolRequestSchema, async (request) => {
switch (request.params.name) {
case 'my_tool':
return await myTool.execute(request.params.arguments);
// ... other tools
}
});
```
---
## Deployment
### Docker Deployment
```dockerfile
# mcp/Dockerfile
FROM node:18-alpine
WORKDIR /app
# Install dependencies
COPY package*.json ./
RUN npm ci --production
# Copy built files
COPY dist ./dist
CMD ["node", "dist/index.js"]
```
```bash
# Build Docker image
npm run mcp:docker:build
# Run container
docker run -e MCP_DASHBOARD_BASE_URL=http://localhost:4820 agent-dashboard-mcp:local
```
### Podman Deployment
```bash
# Build with Podman
npm run mcp:podman:build
# Run with Podman
podman run -e MCP_DASHBOARD_BASE_URL=http://localhost:4820 localhost/agent-dashboard-mcp:local
```
---
## Summary
The MCP server provides:
- ✅ **AI-native interface** - Claude can query dashboard data naturally
- ✅ **Complete tool coverage** - Sessions, agents, tools, pricing, stats
- ✅ **Type-safe** - Full TypeScript types with Zod validation
- ✅ **Standards-compliant** - Implements MCP protocol specification
- ✅ **Easy setup** - One-command installation and configuration
- ✅ **Local-first** - No cloud dependencies, runs entirely locally
- ✅ **Docker-ready** - Containerized deployment support
For API details, see [docs/API.md](./API.md).
docs/PLUGINS.md
+321
View File
@@ -0,0 +1,321 @@
# Claude Code Agent Monitor — Plugin Marketplace
Official Claude Code plugins for the Agent Monitor dashboard. **10 plugins** extend Claude Code with skills, agents, slash commands, hooks, and CLI tools for deep analytics, cost guardrails, productivity automation, developer tools, AI-powered insights, session forensics, workflow/fleet intelligence, reliability & SLOs, config & memory governance, and dashboard connectivity.
Every plugin is powered by the local Agent Monitor REST API at `http://localhost:4820`. They are read-only advisors unless a skill explicitly documents a mutating endpoint (and those preview + confirm before acting).
## Quick Start
### Add the marketplace
```bash
claude plugin marketplace add Smartgift-AI/Claude-Code-Monitor
```
### Install a plugin
```bash
claude plugin install ccam-analytics@smartgift-claude-code-monitor
claude plugin install ccam-cost-guard@smartgift-claude-code-monitor
claude plugin install ccam-productivity@smartgift-claude-code-monitor
claude plugin install ccam-devtools@smartgift-claude-code-monitor
claude plugin install ccam-insights@smartgift-claude-code-monitor
claude plugin install ccam-sessions@smartgift-claude-code-monitor
claude plugin install ccam-workflows@smartgift-claude-code-monitor
claude plugin install ccam-quality@smartgift-claude-code-monitor
claude plugin install ccam-config@smartgift-claude-code-monitor
claude plugin install ccam-dashboard@smartgift-claude-code-monitor
```
### Or install locally during development
```bash
# From the repo root, test a plugin locally
claude --plugin-dir plugins/ccam-analytics
```
## Prerequisites
- [Claude Code](https://docs.anthropic.com/en/docs/claude-code) installed and authenticated
- Agent Monitor dashboard running at `http://localhost:4820` (see [SETUP.md](../SETUP.md))
- Hooks installed: `npm run setup` from the Agent Monitor project
Skills and commands are invoked as `/ccam-<plugin>:<name>`. Agents are dispatched automatically by Claude Code (or named explicitly).
## Available Plugins
### 1. `ccam-analytics` — Analytics & Monitoring
Deep analytics on sessions, token usage, costs, cache efficiency, model mix, and productivity.
| Skill | Command | Purpose |
|-------|---------|---------|
| Session Report | `/ccam-analytics:session-report` | Per-model tokens (input/output/cache_read/cache_write + baselines), cost, agent hierarchy, tool activity, timeline |
| Cost Breakdown | `/ccam-analytics:cost-breakdown` | Per-model cost via the pricing engine, daily trends, cache efficiency, optimization opportunities |
| Usage Trends | `/ccam-analytics:usage-trends` | 365-day session/event trends, token volume, tool rankings, model distribution, event-type ratios |
| Productivity Score | `/ccam-analytics:productivity-score` | Weighted scorecard: completion, token efficiency, tool effectiveness, velocity, cost efficiency |
| Cache Efficiency | `/ccam-analytics:cache-efficiency` | Cache hit rate, write-vs-read reuse, sessions with poor cache reuse |
| Model Mix | `/ccam-analytics:model-mix` | Share of tokens and cost per model family; expensive models doing cheap work |
**Commands:** `/ccam-analytics:cost-today` · `/ccam-analytics:top-spenders` · `/ccam-analytics:burn-rate`
**Agents:** `analytics-advisor` (full advisor incl. workflow intelligence) · `token-economist` (token economics & reduction tactics)
**Hooks:** Logs `Stop` / `SubagentStop` events. **CLI:** `ccam-stats` — terminal stats (sessions, cost, tokens).
---
### 2. `ccam-cost-guard` — Budget Guardrails
Spend limits, forecasting, cost alerts, and model-routing savings.
| Skill | Command | Purpose |
|-------|---------|---------|
| Budget Set | `/ccam-cost-guard:budget-set` | Define a budget and (optionally) arm a `token_threshold` alert rule; explains the $→token conversion |
| Spend Forecast | `/ccam-cost-guard:spend-forecast` | Project week/month-end spend from the daily trend (moving average × remaining days) |
| Cost Alert | `/ccam-cost-guard:cost-alert` | Review alert rules and fired alerts; explain exactly what tripped |
| Model Savings | `/ccam-cost-guard:model-savings` | Estimate $ saved by routing eligible work to a cheaper model family |
| Daily Budget Check | `/ccam-cost-guard:daily-budget-check` | Today's spend vs a daily budget, pace vs target, projected overage |
**Commands:** `/ccam-cost-guard:budget` · `/ccam-cost-guard:forecast` · `/ccam-cost-guard:overspend`
**Agent:** `budget-sentinel` — watches spend vs target, projects month-end, recommends cuts. **Hooks:** fail-safe `Stop` event POST so budget tracking sees session ends.
---
### 3. `ccam-productivity` — Productivity & Workflows
Standups, weekly/monthly reviews, sprint tracking, focus analysis, and workflow optimization.
| Skill | Command | Purpose |
|-------|---------|---------|
| Daily Standup | `/ccam-productivity:daily-standup` | Standup from recent sessions — work by project (cwd), costs, tools, errors, velocity |
| Weekly Report | `/ccam-productivity:weekly-report` | Daily session/event trends, per-session costs, token volumes, tool top-20, completion rates |
| Sprint Summary | `/ccam-productivity:sprint-summary` | Per-project + per-model costs, token efficiency, subagent effectiveness, retrospective data |
| Workflow Optimizer | `/ccam-productivity:workflow-optimizer` | Tool-flow transitions, effectiveness, delegation, error propagation, concurrency, compaction |
| Monthly Review | `/ccam-productivity:monthly-review` | Month-over-month sessions, cost, tokens, completion, top projects, notable shifts |
| Time of Day | `/ccam-productivity:time-of-day` | Activity/productivity bucketed by hour and day-of-week; peak vs low-output windows |
**Commands:** `/ccam-productivity:standup` · `/ccam-productivity:whats-next` · `/ccam-productivity:focus-report`
**Agents:** `productivity-coach` (work-pattern review) · `focus-analyst` (deep-work / focus blocks). **Hooks:** session start/end timing.
---
### 4. `ccam-devtools` — Developer Tools
Debugging, data-integrity inspection, event tracing, transcript search, diagnostics, export, and health checks.
| Skill | Command | Purpose |
|-------|---------|---------|
| Session Debug | `/ccam-devtools:session-debug` | Full event chain, agent hierarchy, token usage with baselines, workflow intelligence |
| Hook Diagnostics | `/ccam-devtools:hook-diagnostics` | Hook install, connectivity, handler validation, event delivery, data freshness |
| Data Export | `/ccam-devtools:data-export` | Export sessions/events/analytics/costs as JSON/CSV/Markdown |
| Health Check | `/ccam-devtools:health-check` | API, SQLite (WAL), WebSocket, endpoints, hooks, disk, data freshness |
| Event Trace | `/ccam-devtools:event-trace` | Ordered event timeline for a session, highlighting gaps/failures |
| Transcript Grep | `/ccam-devtools:transcript-grep` | Search a session transcript for a string/pattern with context |
**Commands:** `/ccam-devtools:doctor` · `/ccam-devtools:export` · `/ccam-devtools:tail-events`
**Agents:** `issue-triager` (cross-component triage) · `db-inspector` (data-integrity inspection). **CLI:** `ccam-doctor`, `ccam-export`.
---
### 5. `ccam-insights` — AI-Powered Insights
Pattern detection, anomaly alerting, forecasting, regression watch, benchmarking, optimization, and comparison.
| Skill | Command | Purpose |
|-------|---------|---------|
| Pattern Detect | `/ccam-insights:pattern-detect` | Tool-flow transitions, recurring sequences, agent co-occurrence, delegation habits |
| Anomaly Alert | `/ccam-insights:anomaly-alert` | Cost/token/event-ratio/complexity outliers (statistical) |
| Optimization Suggest | `/ccam-insights:optimization-suggest` | Model downgrades, cache optimization, compaction reduction, tool reliability |
| Session Compare | `/ccam-insights:session-compare` | Side-by-side tokens, costs, complexity, tool-flow, metadata deltas |
| Regression Watch | `/ccam-insights:regression-watch` | Rising error rate, falling cache hits, growing compaction, climbing cost/session |
| Benchmark | `/ccam-insights:benchmark` | Benchmark a session vs the rolling average; show percentile |
**Commands:** `/ccam-insights:insights` · `/ccam-insights:compare` · `/ccam-insights:anomalies`
**Agents:** `insights-advisor` (strategic analysis) · `trend-forecaster` (near-future cost/usage projection).
---
### 6. `ccam-sessions` — Session Forensics
Search, timeline, transcript replay, per-project rollups, and lifecycle management.
| Skill | Command | Purpose |
|-------|---------|---------|
| Session Search | `/ccam-sessions:session-search` | Find sessions by project/model/status/date; rank by cost or recency |
| Session Timeline | `/ccam-sessions:session-timeline` | Ordered timeline of one session's events with durations and tool names |
| Transcript Replay | `/ccam-sessions:transcript-replay` | Walk a transcript turn-by-turn, summarizing each message |
| CWD Rollup | `/ccam-sessions:cwd-rollup` | Roll up sessions by working directory: counts, cost, tokens, last-active |
| Session Cleanup | `/ccam-sessions:session-cleanup` | Identify stale/empty sessions; preview before the cleanup endpoint deletes (confirm required) |
**Commands:** `/ccam-sessions:find-session` · `/ccam-sessions:replay` · `/ccam-sessions:recent`
**Agent:** `session-investigator` — end-to-end investigation of a single session.
---
### 7. `ccam-workflows` — Orchestration & Fleet Intelligence
Multi-agent structure analysis using the workflow intelligence API and Workflow-tool run journals.
| Skill | Command | Purpose |
|-------|---------|---------|
| DAG Map | `/ccam-workflows:dag-map` | Orchestration DAG: parent→child subagent edges, depth, fan-out |
| Delegation Audit | `/ccam-workflows:delegation-audit` | Model delegation + subagent effectiveness; wasted delegations |
| Concurrency Report | `/ccam-workflows:concurrency-report` | Concurrency lanes, parallelism, serialization bottlenecks |
| Error Propagation | `/ccam-workflows:error-propagation` | Trace failures by depth and how they cascade across subagents |
| Fleet Runs | `/ccam-workflows:fleet-runs` | Summarize Workflow-tool fleet runs (no-hook fleets ingested from run journals) |
**Commands:** `/ccam-workflows:workflow` · `/ccam-workflows:dag` · `/ccam-workflows:runs`
**Agent:** `orchestration-analyst` — analyzes the 11 workflow datasets + fleet runs.
---
### 8. `ccam-quality` — Reliability & SLOs
Error monitoring, hook-delivery health, SLO tracking with error budgets, and regression alerts.
| Skill | Command | Purpose |
|-------|---------|---------|
| Error Scan | `/ccam-quality:error-scan` | Scan events for APIError + failure signals; group by tool/model; rank by frequency |
| API Error Report | `/ccam-quality:api-error-report` | APIError detail: counts over time, affected sessions/models, likely causes |
| Hook Failure Audit | `/ccam-quality:hook-failure-audit` | PreToolUse/PostToolUse balance, missing terminators, stale ingestion |
| SLO Check | `/ccam-quality:slo-check` | Completion rate, tool success rate, error rate; error budget remaining |
| Regression Alert | `/ccam-quality:regression-alert` | Compare this period's error/failure rates to the prior period; optional alert rule |
**Commands:** `/ccam-quality:errors` · `/ccam-quality:slo` · `/ccam-quality:health`
**Agent:** `reliability-engineer` — treats Claude Code usage as a service with an error budget.
---
### 9. `ccam-config` — Config & Memory Governance
Audit your Claude Code configuration and curate the file-based memory store via the Config Explorer API.
| Skill | Command | Purpose |
|-------|---------|---------|
| Config Audit | `/ccam-config:config-audit` | Counts per surface (user vs project), duplicate skills/agents, shell-running hooks |
| Memory Review | `/ccam-config:memory-review` | CLAUDE.md + per-project auto-memory files grouped by project; flag stale/oversized facts |
| Skill Inventory | `/ccam-config:skill-inventory` | Installed skills + contributing plugins; overlap with your own skills |
| MCP Audit | `/ccam-config:mcp-audit` | MCP servers (user + project): transport, command/args/env names, source file |
| Hook Inventory | `/ccam-config:hook-inventory` | Hooks across settings + the hooks scripts dir; flag network/arbitrary-command hooks |
**Commands:** `/ccam-config:audit-config` · `/ccam-config:memory` · `/ccam-config:inventory`
**Agent:** `config-auditor` — audits config sprawl, duplication, risky hooks, and stale memory.
> Memory Review can also edit the per-project memory store: auto-memory files are mutable via `PUT`/`DELETE /api/cc-config/file` with `{ scope: "auto-memory", type: "auto-memory", project, name }` (always backed up first).
---
### 10. `ccam-dashboard` — Dashboard Connector
Direct MCP integration, quick status, live watch, and endpoint probing.
| Skill | Command | Purpose |
|-------|---------|---------|
| Dashboard Status | `/ccam-dashboard:dashboard-status` | Health: API connectivity, session/event counts, hook status, data freshness |
| Quick Stats | `/ccam-dashboard:quick-stats` | One-line metrics: active sessions, total cost, events, top tool, cache efficiency |
| Live Watch | `/ccam-dashboard:live-watch` | Poll a few times to show live deltas (active sessions/agents, events, ws connections) |
| Endpoint Probe | `/ccam-dashboard:endpoint-probe` | Probe each major API route and report reachability/shape |
**Commands:** `/ccam-dashboard:status` · `/ccam-dashboard:ping` · `/ccam-dashboard:open-dashboard`
**Agent:** `dashboard-operator` — verifies the dashboard is up and guides start/restart/import. **MCP Server:** direct tool access to the Agent Monitor API. **Settings:** default agent model.
---
## Data Model Reference
These plugins query the Agent Monitor API at `http://localhost:4820`. Key data shapes:
### Token Tracking
- **4 token types**: `input_tokens`, `output_tokens`, `cache_read_input_tokens`, `cache_creation_input_tokens`
- **4 baselines**: `baseline_input`, `baseline_output`, `baseline_cache_read`, `baseline_cache_write` (preserve pre-compaction tokens)
- **Effective total** = current + baseline (the `/api/analytics` totals are pre-summed)
### Cost Calculation
- Formula: `(tokens / 1,000,000) × rate_per_mtok` for each token type
- Model matching: longest `model_pattern` wins (e.g., `claude-sonnet-4-5%` beats `claude-sonnet-4%`)
- Pre-seeded rates for Opus, Sonnet, Haiku families
### Session Metadata (JSON)
- `thinking_blocks`: count of extended thinking blocks
- `turn_count`: number of conversation turns
- `total_turn_duration_ms`: cumulative turn processing time
- `usage_extras`: `{ service_tiers[], speeds[], inference_geos[] }`
### Event Types
`PreToolUse`, `PostToolUse`, `Stop`, `SubagentStop`, `SessionStart`, `SessionEnd`, `Notification`, `Compaction`, `APIError`, `TurnDuration`, `ToolError`, `Interrupted`
### Workflow Intelligence API (`/api/workflows/{sessionId}`)
11 datasets: `stats`, `orchestration` (DAG), `toolFlow` (transitions), `effectiveness` (subagent success), `patterns` (recurring sequences), `modelDelegation`, `errorPropagation` (by depth), `concurrency` (lanes), `complexity` (score), `compaction` (impact), `cooccurrence` (agent pairs)
### Alert Rules (`/api/alerts/rules`)
Rule types: `token_threshold` (`{ total_tokens }` — the spend-relevant guardrail), `event_pattern`, `inactivity`, `status_duration`.
### Config Explorer (`/api/cc-config/*`)
Read every Claude Code surface (skills, agents, commands, output-styles, plugins, marketplaces, mcp, hooks, settings, keybindings, statusline, memory). `memory` includes the per-project file-based store with `scope: "auto-memory"` (carrying `project`, `name`, `isIndex`, `frontmatter`); those files plus `CLAUDE.md` are mutable via `PUT`/`DELETE /api/cc-config/file` with always-on timestamped backups.
## Plugin Development
To create your own plugins for the Agent Monitor, see the [Claude Code plugin documentation](https://docs.anthropic.com/en/docs/claude-code/plugins).
### Plugin structure
```
my-plugin/
├── .claude-plugin/
│ └── plugin.json # Required: name (== dir name), description, version
├── skills/
│ └── my-skill/
│ └── SKILL.md # Skill (description-only frontmatter; uses $ARGUMENTS)
├── agents/
│ └── my-agent.md # Agent (name == filename, model, tools, instructions)
├── commands/
│ └── my-command.md # Slash command (description, optional argument-hint)
├── hooks/
│ └── hooks.json # Event hooks (fail-safe, non-blocking)
├── bin/
│ └── my-cli-tool # CLI scripts (added to PATH)
├── .mcp.json # MCP server configuration
└── settings.json # Plugin settings
```
Structure is validated by `server/__tests__/plugins-marketplace.test.js`, which enforces the marketplace↔directory bijection, `plugin.json` shape, name/dir agreement, and required frontmatter on every agent / skill / command.
### Testing locally
```bash
claude --plugin-dir /path/to/my-plugin # then use /my-plugin:my-skill some args
```
## Troubleshooting
### Dashboard not reachable
```bash
cd /path/to/Claude-Code-Agent-Monitor
npm start # or: npm run dev
```
### Hooks not installed
```bash
cd /path/to/Claude-Code-Agent-Monitor
npm run setup
```
### Plugin not found
```bash
claude plugin marketplace list
claude plugin marketplace add Smartgift-AI/Claude-Code-Monitor
```
## License
Internal SmartGift build — all rights reserved.
docs/README.md
+494
View File
@@ -0,0 +1,494 @@
# Documentation Index
Comprehensive documentation for the Agent Dashboard project.
---
## Quick Links
- [Architecture Overview](../ARCHITECTURE.md) - System design and technical reference
- [I18N Architecture](./I18N.md) - Internationalization architecture and usage guide
- [CLI Reference](./CLI.md) - The `ccam` terminal CLI: every command, discovery, safety model
- [Setup Guide](../SETUP.md) - Installation and configuration
- [Installation](../INSTALL.md) - Detailed installation instructions
---
## Documentation Sections
### 📘 Core Documentation
```mermaid
graph TB
Start[Start Here] --> Setup[SETUP.md<br/>Installation & Config]
Start --> Architecture[ARCHITECTURE.md<br/>System Design]
Setup --> Client[Client README<br/>React UI docs]
Setup --> Server[Server README<br/>Backend docs]
Architecture --> API[API.md<br/>REST & WebSocket]
Architecture --> Database[DATABASE.md<br/>Schema reference]
Architecture --> Hooks[HOOKS.md<br/>Hook system integration]
Architecture --> MCP[MCP.md<br/>MCP server integration]
Setup --> Deploy[DEPLOYMENT.md<br/>Production deployment]
style Start fill:#3B82F6
style Setup fill:#10B981
style Architecture fill:#F59E0B
```
---
### 📋 Documentation Catalog
| Document | Description | Audience |
|----------|-------------|----------|
| [client/README.md](../client/README.md) | React frontend architecture, components, state management | Frontend developers |
| [server/README.md](../server/README.md) | Express backend, database, WebSocket, API | Backend developers |
| [API.md](./API.md) | REST API endpoints (sessions, agents, events, stats, analytics, hooks, pricing, workflows, settings, import history, **cc-config**, **run**), WebSocket protocol (including `run_stream` / `run_status` / `run_input_ack` for the Run page) | Integration developers |
| [DATABASE.md](./DATABASE.md) | SQLite schema, queries, performance | Database administrators |
| [HOOKS.md](./HOOKS.md) | Claude Code hook system integration | Hook developers |
| [MCP.md](./MCP.md) | MCP server setup and tool reference | MCP integrators |
| [DEPLOYMENT.md](./DEPLOYMENT.md) | Production deployment strategies | DevOps engineers |
| [I18N.md](./I18N.md) | Language architecture, locale strategy, and rollout checklist | Frontend and product teams |
| [CLI.md](./CLI.md) | `ccam` command reference — monitoring, browsing, insights, alerts, pricing, import, administration | Terminal users and CI scripting |
| [monitoring/README.md](../monitoring/README.md) | Prometheus + Grafana stack (`npm run monitoring:up` or Docker) | DevOps / observability |
---
## Getting Started
### For New Users
```mermaid
graph LR
A[New to Project] --> B[Read SETUP.md]
B --> C[Install Dependencies]
C --> D[Run npm run dev]
D --> E[Open localhost:5173]
style A fill:#3B82F6
style E fill:#10B981
```
**Quick Start:**
1. Read [SETUP.md](../SETUP.md)
2. Run `npm run setup`
3. Run `npm run dev`
4. Open browser to `http://localhost:5173`
---
### For Frontend Developers
```mermaid
graph TB
FE[Frontend Developer] --> ClientDocs[client/README.md]
ClientDocs --> Components[Component Architecture]
ClientDocs --> State[State Management]
ClientDocs --> WebSocket[WebSocket Integration]
style FE fill:#61DAFB
```
**Key Documents:**
- [client/README.md](../client/README.md) - Complete frontend guide
- [API.md](./API.md#websocket-api) - WebSocket protocol
- Component source: `client/src/components/`
---
### For Backend Developers
```mermaid
graph TB
BE[Backend Developer] --> ServerDocs[server/README.md]
ServerDocs --> Routes[API Routes]
ServerDocs --> DB[Database Design]
ServerDocs --> WS[WebSocket Server]
style BE fill:#339933
```
**Key Documents:**
- [server/README.md](../server/README.md) - Complete backend guide
- [DATABASE.md](./DATABASE.md) - Schema and queries
- [HOOKS.md](./HOOKS.md) - Hook processing
- API source: `server/routes/`
---
### For DevOps Engineers
```mermaid
graph TB
DevOps[DevOps Engineer] --> Deploy[DEPLOYMENT.md]
Deploy --> Docker[Docker Setup]
Deploy --> PM2[PM2 Process Manager]
Deploy --> Cloud[Cloud Deployment]
Deploy --> Monitoring[Monitoring & Logging]
style DevOps fill:#F59E0B
```
**Key Documents:**
- [DEPLOYMENT.md](./DEPLOYMENT.md) - Complete deployment guide
- [DATABASE.md](./DATABASE.md#backup-strategies) - Backup strategies
- [server/README.md](../server/README.md#performance) - Performance tuning
---
### For Integration Developers
```mermaid
graph TB
Integration[Integration Developer] --> API[API.md]
API --> REST[REST Endpoints]
API --> WebSocket[WebSocket Events]
Integration --> MCP[MCP.md]
MCP --> Tools[MCP Tools]
MCP --> Config[Client Configuration]
style Integration fill:#8B5CF6
```
**Key Documents:**
- [API.md](./API.md) - Complete API reference
- [MCP.md](./MCP.md) - MCP server integration
- [HOOKS.md](./HOOKS.md) - Custom hook integration
---
## Architecture Overview
### System Components
```mermaid
graph TB
subgraph "Frontend"
React[React + TypeScript<br/>Vite + Tailwind]
end
subgraph "Backend"
Express[Express Server<br/>Node.js 20+]
DB[(SQLite Database)]
WS[WebSocket Server]
end
subgraph "Integration"
Hooks[Claude Code Hooks]
MCP[MCP Server]
end
subgraph "Clients"
Browser[Web Browser]
Claude[Claude Desktop]
Custom[Custom Clients]
end
Browser --> React
React -->|HTTP/WS| Express
Express --> DB
Express --> WS
Hooks -->|HTTP POST| Express
Claude -->|stdio| MCP
MCP -->|HTTP| Express
Custom -->|HTTP| Express
style React fill:#61DAFB
style Express fill:#000000,color:#fff
style DB fill:#003B57,color:#fff
style MCP fill:#0f766e
```
**Technology Stack:**
| Layer | Technology |
|-------|------------|
| **Frontend** | React 18, TypeScript 5.7, Vite 6, Tailwind CSS |
| **Backend** | Node.js 20+, Express 4.21, WebSocket |
| **Database** | SQLite 3 (better-sqlite3 or node:sqlite) |
| **Integration** | Claude Code Hooks, MCP Server |
### Internationalization Support (en/zh/vi/ko)
```mermaid
flowchart LR
A["User language preference<br/>en / zh / vi / ko"] --> B["i18next detector<br/>localStorage + navigator"]
B --> C["Namespace JSON resources"]
C --> D["React useTranslation hooks"]
D --> E["Localized UI + a11y labels"]
E --> F["Locale-aware date/number formatting"]
F --> G["formatModelName() — human-friendly model display"]
```
Supported language codes are explicitly `en`, `zh`, and `vi`. Use [I18N.md](./I18N.md) for architecture details, naming conventions, language switching flow, localization behavior, and rollout guidance.
---
## Feature Documentation
### Real-Time Updates
```mermaid
sequenceDiagram
participant Hook as Claude Code Hook
participant Server as Dashboard Server
participant DB as SQLite
participant WS as WebSocket
participant Client as Browser
Hook->>Server: POST /hooks/post-tool-use
Server->>DB: Update data
DB-->>Server: Success
Server->>WS: Broadcast event
WS->>Client: { type: 'tool.executed', data }
Client->>Client: Update UI
Note over Client: No polling required!
```
**Documentation:**
- [WebSocket Protocol](./API.md#websocket-api)
- [Client Integration](../client/README.md#websocket-integration)
- [Server Broadcasting](../server/README.md#websocket-protocol)
---
### Pricing System
```mermaid
graph TB
Model[Model Name] --> Match[Pattern Matching]
Match --> Custom{Custom<br/>Rule?}
Custom -->|Yes| UseCustom[Use Custom Pricing]
Custom -->|No| UseDefault[Use Default Pricing]
UseCustom --> Calculate[Calculate Cost]
UseDefault --> Calculate
Calculate --> Result[input_cost + output_cost]
style Calculate fill:#10B981
```
**Documentation:**
- [Pricing API](./API.md#pricing)
- [Database Schema](./DATABASE.md#pricing_rules)
- [Server Implementation](../server/README.md#pricing-calculation)
---
### Hook System
```mermaid
graph LR
Claude[Claude Code] -->|stdin| Hook[Hook Script]
Hook -->|exec| Handler[hook-handler.js]
Handler -->|HTTP POST| Server[Dashboard Server]
Server --> DB[(Database)]
Server --> WS[WebSocket]
style Hook fill:#F59E0B
style Handler fill:#10B981
```
**Documentation:**
- [Hook System Guide](./HOOKS.md)
- [Hook Processing](../server/README.md#hook-processing)
- [Installation](../SETUP.md#install-hooks)
---
## API Documentation
### REST API Summary
| Endpoint | Method | Description |
|----------|--------|-------------|
| `/api/sessions` | GET | List sessions |
| `/api/sessions/:id` | GET | Get session |
| `/api/sessions/:id/agents` | GET | List session agents |
| `/api/agents/:id` | GET | Get agent |
| `/api/agents/:id/tools` | GET | List agent tools |
| `/api/pricing` | GET | List pricing rules |
| `/api/pricing` | POST | Create pricing rule |
| `/api/pricing/:pattern` | DELETE | Delete pricing rule |
**Full Reference:** [API.md](./API.md#rest-api)
---
### WebSocket Events
| Event Type | Triggered By |
|------------|--------------|
| `session.created` | SessionStart hook |
| `session.updated` | Any session update |
| `agent.created` | New agent started |
| `agent.updated` | Agent status/cost change |
| `tool.executed` | Tool execution completed |
| `notification.received` | System notification |
**Full Reference:** [API.md](./API.md#websocket-api)
---
## Database Schema
### Entity Relationships
```mermaid
erDiagram
sessions ||--o{ agents : "has many"
agents ||--o{ tool_executions : "has many"
sessions ||--o{ notifications : "has many"
sessions {
text session_id PK
text model
text status
real total_cost
datetime updated_at
}
agents {
text agent_id PK
text session_id FK
text agent_type
text status
text current_tool
int input_tokens
int output_tokens
real cost
}
```
**Full Reference:** [DATABASE.md](./DATABASE.md)
---
## Deployment Options
```mermaid
graph TB
subgraph "Development"
Dev[npm run dev<br/>Hot reload]
end
subgraph "Production"
Docker[Docker Compose<br/>Containerized]
PM2[PM2<br/>Process manager]
Systemd[Systemd Service<br/>Linux systems]
Cloud[Cloud Platform<br/>AWS, Azure, GCP]
end
Dev -.->|Build| Docker
Dev -.->|Build| PM2
Dev -.->|Build| Systemd
Dev -.->|Build| Cloud
style Dev fill:#3B82F6
style Docker fill:#2496ED
style PM2 fill:#10B981
style Systemd fill:#F59E0B
style Cloud fill:#8B5CF6
```
**Full Reference:** [DEPLOYMENT.md](./DEPLOYMENT.md)
---
## Performance Metrics
### Benchmarks
| Metric | Target | Actual |
|--------|--------|--------|
| Hook processing | < 100ms | ~70ms |
| API response time | < 50ms | ~30ms |
| WebSocket latency | < 10ms | ~5ms |
| Database query | < 10ms | ~5ms |
| Session list (50) | < 20ms | ~10ms |
**Optimization Details:**
- [Server Performance](../server/README.md#performance)
- [Database Tuning](./DATABASE.md#performance-optimization)
- [Client Performance](../client/README.md#performance)
---
## Contributing
### Development Workflow
```mermaid
graph LR
Fork[Fork Repository] --> Clone[Clone Locally]
Clone --> Branch[Create Feature Branch]
Branch --> Code[Write Code]
Code --> Test[Run Tests]
Test --> Commit[Commit Changes]
Commit --> Push[Push to Fork]
Push --> PR[Create Pull Request]
style Fork fill:#3B82F6
style PR fill:#10B981
```
**Before submitting:**
1. Run tests: `npm test` (server `node --test` + client Vitest, including per-screen render snapshots — regenerate intentional UI changes with `cd client && npx vitest run -u`)
2. Check formatting: `npm run format:check`
3. Build: `npm run build`
4. Update docs if needed
---
## Support & Resources
### Getting Help
- **Issues:** [GitHub Issues](https://github.com/your-org/agent-dashboard/issues)
- **Discussions:** [GitHub Discussions](https://github.com/your-org/agent-dashboard/discussions)
- **Documentation:** This folder
### Additional Resources
- [Claude Code Documentation](https://docs.anthropic.com/claude/docs)
- [Model Context Protocol (MCP)](https://modelcontextprotocol.io/)
- [SQLite Documentation](https://sqlite.org/docs.html)
- [React Documentation](https://react.dev/)
- [Express Documentation](https://expressjs.com/)
---
## License
Internal SmartGift build — all rights reserved.
---
## Summary
This documentation covers:
- ✅ **Complete architecture** - Frontend, backend, database, integrations
- ✅ **API reference** - REST endpoints, WebSocket events
- ✅ **Deployment guides** - Docker, PM2, systemd, cloud
- ✅ **Performance tuning** - Database, server, client optimizations
- ✅ **Integration guides** - Hooks, MCP, custom clients
- ✅ **Internationalization** - Language resources, switching flow, locale formatting, rollout checklist
- ✅ **Development guides** - Setup, testing, contributing
**Start with:** [SETUP.md](../SETUP.md) for installation, then explore specific areas based on your role.
docs/screenshots/dashboard-local-fonts.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 516 KiB

docs/superpowers/logs/2026-07-27-lanes-pipeline-build-log.md
+53
View File
@@ -0,0 +1,53 @@
# SDD ledger — plan: docs/superpowers/plans/2026-07-27-lanes-pipeline.md
Base commit: 8e41e80 (branch feat/lanes-pipeline)
Task 1: review — spec MET, quality SOUND. 1 Important, 1 Minor.
Task 1: ruled — Important (.gitignore `data/` -> `/data/`) does NOT enter fix loop. Plan's Global Constraints say "Preserve existing behavior. Additive schema only"; a gitignore anchor changes no behavior, and the reviewer's "purely additive" alternative edits the same file AND cannot work alone (git cannot re-include a path under an excluded directory without also negating the parent). `/data/` is the correct minimal fix. Stands.
Task 1: minor (deferred): lanes-lib.test.js test title says "amber" where it means "without evidence" (wording inherited from the plan).
Task 1: complete (commits 8e41e80..8b9e477, review clean after ruling)
Task 2: review — spec MET except one deviation; all 5 load-bearing behaviors verified correct. 1 Important, 1 Minor.
Task 2: ruled — Important (lane SQL prepared inline in lanes.js instead of added to db.js `stmts`) does NOT enter fix loop: the plan contradicted itself (File Structure said "stmts entries", the Task 2 code block and its Interfaces line "Owns all SQL for lanes" say inline). The code block is authoritative; the stale File Structure line has been corrected in the plan so later reviews do not re-raise it.
Task 2: minor (deferred): lanes-lib.test.js @file comment still says it only covers pipelines.js.
Task 2: complete (commits 8b9e477..f29d904, review clean after ruling)
Task 3: review — spec MET; 1 "Critical" (WS payload asymmetry) ruled plan-mandated, 3 Important, 3 Minor.
Task 3: ruled — delete broadcasting `{removed: id}` instead of `{lane}` is deliberate and consumed by Task 7's Lanes.tsx; the plan's Produces bullet was stale prose and has been corrected. Not a defect.
Task 3: ruled — per-lane `SELECT MAX(created_at)` stands: events(session_id) is indexed and lanes number in the dozens (one per worktree), not thousands.
Task 3: ruled — broadcast-before-response stands: `broadcast()` in server/websocket.js is already defensive.
Task 3: minor (deferred): double payload() call per mutation (broadcastLane re-reads the lane); lanes-api.test.js tests share state via an outer laneId.
Task 3: fix round 1/5 (2 addressed, 0 open — 409 now branches on SQLITE_CONSTRAINT_UNIQUE with message fallback; new WS test asserts lane_update on create and delete; commits 8e76c90..76c6f26)
Task 3: complete (commits a69b3d8..76c6f26, review clean)
Task 4: review — spec FAIL (1 Critical: needs_action cleared by any session), 2 Important, 1 Minor.
Task 4: ruled — the Critical was real. The plan's own contract said "cleared on the next non-Notification hook for that lane"; amended to "from the session currently bound to that lane, evaluated before rebinding" so two agents sharing one worktree cannot cancel each other's "needs you".
Task 4: fix round 1/5 (3 addressed, 0 open — clear now gated on the pre-existing lane.session_id; cross-session + default-message tests added, verified to fail against the buggy code; per-hook lane-scan ceiling documented, no cache; commits 83aa655..3f7121d)
Task 4: complete (commits 76c6f26..3f7121d, review clean) — 780 server tests pass
Task 5: first attempt reported DONE_WITH_CONCERNS claiming the sandbox blocks loopback — WRONG, and nothing was committed (pre-commit test gate held). Real cause: the plan's own test harness used blocking spawnSync while the test HTTP server ran in the same process, so the event loop stalled and the CLI child's request was never served. Verified loopback works between processes (detached node server + curl + separate node fetch, all exit 0). Plan's code block corrected to async spawn.
Task 5: review — spec PASS, 2 Important (ccam lanes add untested; temp DB leaked), 3 Minor.
Task 5: fix round 1/5 (2 addressed, 1 open — lanes-add test added, health poll replaced the 100ms sleep, DB cleanup added but not exception-safe; commits 19dd31b..3deb420)
Task 5: fix round 2/5 (1 addressed, 0 open — after() teardown wrapped in try/finally; commit 35138b4; verified inline)
Task 5: complete (commits 3f7121d..35138b4, review clean) — 783 server tests pass. `ccam lanes add` was an authorised addition so the CLI's empty-state hint names a command that exists.
Task 6: review — spec MET; security verified adversarially (cross-origin POST to /:id/start returns 403; prompt travels via stdin, model/effort/resumeSessionId are separate argv with no shell; /:id/stage not captured by /:id/:action; unknown action rejected before any lookup or mutation). 1 Important, 2 Minor.
Task 6: fix round 1/5 (2 addressed, 0 open — message now consults getRun and returns 409 for a recorded-but-dead run instead of 500; unknown-action test asserts stage/status unchanged; new 409 test; commits f225fe8..b3f9e2f)
Task 6: minor (deferred): lane.cwd is not re-validated at action time — a vanished directory fails cleanly at spawn.
Task 6: complete (commits 35138b4..b3f9e2f, review clean) — 788 server tests pass
Task 7: review — 4 Critical, 2 Important, 2 Minor. Three of the Criticals were the PLAN's fault (it mandated wrapping LaneCard in a <button>, an unconfirmed remove, and hardcoded English strings). Review was right; plan was wrong.
Task 7: fix round 1/5 (7 addressed, 0 open — selection is now a keyboard-operable role="button" div with stopPropagation on actions; remove goes through the existing ConfirmModal; every string i18n'd with real zh/vi/ko translations; Lanes case added to screens.snapshot.test.tsx; unknown-lane lane_update refetches so counters stay server-truthful; amber-vs-green test now asserts the colour tokens; start's empty-prompt behaviour documented in a tooltip; commits f29f597..a05065d)
Task 7: minor (deferred): the Lanes screen snapshot captures the empty state only — a populated card + pipeline map is not snapshotted.
Task 7: minor (deferred): the card cannot send a prompt — driving a lane from the UI needs a prompt/message input; today `start` opens a promptless conversation run and `message` has no input field. Follow-up feature, not a defect.
Task 7: complete (commits b3f9e2f..a05065d, review clean) — verified by controller: 279/279 client tests, `npm run build` clean
Task 8: review — every documented command/flag/env var/endpoint/state-rule fact-checked against the shipped code and correct, EXCEPT one Critical: docs advertised `ccam lanes add --pipeline <id>` which the CLI never parsed.
Task 8: ruled — fix the CODE, not the docs: a pipeline template you cannot select from the CLI is a template nobody uses, and the server already accepted the field. Authorised a scoped code change inside the docs task.
Task 8: fix round 1/5 (1 addressed, 0 open — --pipeline parsed and forwarded only when provided, usage string updated, two tests via a DASHBOARD_PIPELINES_DIR fixture with cleanup, docs parenthetical corrected; commits 6cca59c..3b196ef)
Task 8: minor (deferred): README VN/CN/KO mirrors are now behind the English README; the repo's update-project-docs convention expects them synced.
Task 8: complete (commits a05065d..3b196ef, review clean) — 790 server tests pass
Final whole-branch review (Opus): READY — no Critical/Important/Minor findings. Cross-cutting types, CLI-vs-server cwd resolution, WS payload branches, migration safety on an existing DB, the same-origin guard on the spawning routes, and both confirmation gates all verified. All 8 deferred/parked items triaged acceptable-to-defer, none load-bearing.
Controller end-to-end smoke test (real server, real CLI, real hook POSTs) — lane created, `ccam stage plan --evidence` then `ccam stage review` reported: node states came back intake=passed-no-evidence, plan=done, implement/tests=passed-no-evidence, review=current, rest=pending; progress 57%; stage_seconds live; `ccam lanes` table and counters correct; a Notification hook bound the lane and a later hook from the newly-bound session cleared needs_action exactly as the amended contract specifies.
Side effect found and REVERTED: starting the fork's server auto-configured Claude Code hooks in ~/.claude/settings.json (8 entries pointing at this fork). Removed surgically; the pre-existing `rtk hook claude` PreToolUse entry was left untouched. Polluted copy kept at /tmp/settings.before-ccam-cleanup.json.
docs/superpowers/logs/2026-07-28-stage-detection-build-log.md
+31
View File
@@ -0,0 +1,31 @@
# SDD ledger — plan: docs/superpowers/plans/2026-07-28-stage-detection.md
Base commit: d0044d1 (branch feat/stage-detection)
Task 1 (B1): written by Codex terra (foreground; four earlier background runs were killed by the harness's background wall-clock limit). Review — spec MET, all 9 scenarios covered by 5 tests; totality verified by a reviewer that actually threw null/numbers/arrays/circular refs/symbols/malformed rules/a throwing property getter at it (zero throws), and regex compilation confirmed once-per-pipeline (1 RegExp construction across 1000 events, WeakMap keyed on pipeline identity). 2 Important, 1 Minor.
Task 1: fix round 1/5 (3 addressed — the signal was unbounded at 5002 chars for a real Bash command and is now whitespace-collapsed and capped at 120 with an ellipsis; `flattenInput` had concatenated EVERY top-level string, so a real Edit event's `old_string`/`new_string` (whole code blocks) would have been matched against and then shown as the reason for the inference — now restricted to an allowlist of identifying fields; compileRules gained direct totality tests. Totality re-verified by the fixer with its own script.) A Codex attempt at this fix hit the 540 s wall and produced nothing, so it was done by a Claude subagent.
Task 1: complete (869 server tests) — commits 6bb445e + the fix commit
Tooling note: Codex completed 3 of 10 attempted runs in this environment; when it finishes, its work is good (it was the only implementer that ran clause-deletion experiments unprompted), but each failure costs ~9 minutes, so implementation moved to Claude subagents. The useful half of the Codex protocol was kept for every implementer: the agent runs only its own focused test file and does NOT commit; the controller runs the full suite and commits, so the 861-test pre-commit hook runs once per task instead of twice.
Task 2 (B2): rules on plan/implement/tests/review/ship in the default template (intake/gate/done deliberately bare — a gate is a judgement, `done` is a claim); three per-column probes for detected_stage/detected_signal/detected_at; `recordDetection` with the forward-only + declared-wins guard; `detected` decorating nodeStates' output without touching how any state is computed.
Task 2: the implementer found an integration bug in the PLAN, not in its own work, and flagged it instead of fixing outside its file list: `loadAll()` in pipelines.js normalises each node to {id,label,icon,gate,aliases} and DROPPED the new `detect` array, so every rule shipped in the template was inert. Controller reproduced it directly — raw template matched `tests`, `getPipeline('default')` matched null. Neither B1's review nor B2's own tests could have caught it: B1 tested the matcher with raw fixtures that bypass the loader, and B2 tested the rules as JSON. Fixed by preserving `detect` defensively in the loader, plus an END-TO-END test that calls detect(getPipeline('default'), event) through the real loader — the pin that was missing.
Task 2: controller verified through the real path afterwards: rules live, an Edit's signal is its file_path (not the code from old_string/new_string), `gate` still uninferrable, and getPipeline() still returns a stable object so stage-detect's WeakMap regex cache keeps compiling once per pipeline.
Task 2: complete (878 server tests)
Task 3 (B3): detect + recordDetection + broadcast placed inside the existing fail-safe try/catch in touchLaneFromHook, right after the lane is resolved; broadcast only when recordDetection reports written:true, with a comment naming the 29,470-Bash-event volume behind that rule. 5 new API tests including the premise guard at the API level.
Task 3: controller smoke-tested on a real server. First attempt looked like a total failure (detected_stage null) — cause was NOT the code: a server from an earlier branch still held port 4820, so the new instance refused to start and the probes hit stale code. After stopping it: detected_stage=tests with the right signal; 15 identical events plus one backward Edit left updated_at completely unchanged (write-on-change holds under real traffic); the backward Edit did not drag the lane back; every node still `pending` with a `detected` flag and no node `done`.
Task 3: complete (883 server tests)
Task 4 (B4): detected nodes render amber-dashed via a class that REPLACES the state-driven one, so even a hypothetical {detected:true, state:"done"} payload renders amber and never green — the safe direction. `auto:` chip only when the detection leads the declaration. tsc (not vitest) caught a missing field in an unrelated test fixture, which is why `npm run build` is in every client task's verification list.
Task 4: complete (308 client tests)
Task 5 (B5): `ccam lanes` gains a detected suffix using the same lead-comparison as the card, so terminal and browser never disagree; docs/LANES.md documents the signals, the field allowlist, the 120-char cap, the rules node by node, the deliberate blanks on intake/gate/done, forward-only + write-on-change, and the evidence boundary with its reason. The implementer flagged rather than papered over the fact that GET /api/lanes has NO OpenAPI path docs at all and no Lane schema — it added a standalone schema for the three fields instead of inventing either.
Task 5: complete (885 server tests)
FINAL whole-branch review (Opus): READY WITH FIXES — 1 Critical, 5 Important, 7 Minor. The evidence boundary itself held: no lane state could be constructed in which a detection renders green or writes lanes.stage, proved by mutation and live probing rather than by reading, with per-event cost measured (1 µs for an ordinary Bash event, zero queries when nothing changes).
FINAL Critical — clearLane() did not reset the detection columns. A lane reset back to base kept claiming `auto: ship` with 7 of 8 nodes amber for an empty tree, and then refused every subsequent detection with `behind-detected` — and since no node past `ship` carries a rule, detection was dead for that lane permanently.
FINAL corrected MY OWN RULING: I had called the ENOLANE race in the hook path a non-issue "because the outer try/catch swallows it". Right conclusion, wrong reasoning — entering the catch skips the rest of the function, so `needs_action` was left lit and a rebound `session_id` unwritten. The fix is ordering/isolation, not error handling.
FINAL found a second inert rule, the same defect class as the loader bug: `plan`'s `Write docs/.*plan.*\.md` was shadowed by `implement`'s unconditional `Write` because detect() takes the LAST match, so writing a plan document reported `implement`. Documented as live, could never fire.
FINAL proved the boundary was unpinned: removing `&& !stages[n.id]` from withDetected left 56/56 tests passing, because both tests calling themselves PREMISE GUARD sat on lanes with no declarations, making "no node is done" vacuously true.
Fix wave (one commit, 9da58ea, Opus): all 9 items fixed with seven verify-by-deletion experiments and their exact failure messages. It DECLINED my instruction to move the detection block below the bookkeeping, correctly: that block ends in `if (!Object.keys(patch).length) return;`, so detection would have become unreachable on the common hook. It used an inner try/catch instead and corrected the docs sentence. `implement`'s Write rule is now constrained with a negative lookahead, verified at the boundaries (mydocs/, docs2/ still implement).
Fix-wave re-review: READY. Re-ran the G-1 mutation itself in a throwaway copy (32 pass / 1 fail, same tally), rebuilt the I-2 resolution table independently, and traced the alias mechanism to confirm the I-3 test would fail against the old code. All three of the fixer's open items judged non-blocking.
B complete: 10 commits c78e7f7..9da58ea, 892 server tests, 308 client tests.
docs/superpowers/logs/2026-07-28-worktree-lanes-build-log.md
+59
View File
@@ -0,0 +1,59 @@
# SDD ledger — plan: docs/superpowers/plans/2026-07-28-worktree-lanes.md
Base commit: 7195741 (branch feat/worktree-lanes)
Task 1: first attempt committed with --no-verify after misdiagnosing a hook failure as "test isolation". Controller reproduced it via `git commit --amend`: git hooks export GIT_DIR/GIT_INDEX_FILE, every git child inherited them, and in a worktree `.git` is a FILE so `.git/index` gave ENOTDIR. Real bug in worktree.js, not the environment. Fixed by scrubbing 9 GIT_* vars (+ GIT_TERMINAL_PROMPT=0) in the git() helper and the test fixture, with a regression test that sets bogus GIT_DIR/GIT_INDEX_FILE. Recommitted through the hook.
Task 1: review — spec MET (one accepted deviation: reset-in-place instead of delete+recreate); adversarial checks all passed: path containment defeats symlink/prefix/`..`/missing-path attacks via realpath on both sides + path.relative boundary; branch deletion cannot be tricked into main/master/base even by a lying lane row; env scrub complete; no vacuous tests. 2 Important.
Task 1: fix round 1/5 (2 addressed, 0 open — ERESETBRANCH verifies the worktree really landed on the feature branch; ENOBASE verifies the base ref before ANY mutation, with a test proving the worktree and its dirty files are untouched on that path; commits 436adfa..03a6e63)
Task 1: complete (commits 7195741..03a6e63, review clean) — 801 server tests pass, every commit through the pre-commit gate
Task 2: review — spec met on the surface, 2 Critical underneath. (a) the migration probed only `kind` while adding four columns, so a crash after the first ALTER left three columns permanently missing on a real install — the plan's own fault; (b) `kind` was validated in createLane only, so updateLane could silently corrupt the boundary that decides whether CCAM may delete a directory. Plus 3 Important (lock Map never pruned, two missing tests).
Task 2: fix round 1/5 (5 addressed, 0 open — per-column independent probes that self-heal a partial migration; shared validateKind() used by both create and update, rejecting before any write so a mixed patch cannot half-apply; lock entry deleted when its chain settles if still current; regression tests for update-kind, per-lane (not global) locking, and an old-schema database plus a simulated mid-migration crash; commits fdf4da4..6138944)
Task 2: complete (commits 03a6e63..6138944, review clean) — 808 server tests pass
Task 3: review — read-only confirmed, shapes/route order correct. 3 Important: purge WHERE clause duplicated between counter and deleter; the purge test could not fail (every session it created was `active`, so all three exclusions were deletable with the test still green); `unpushed: 0` was a confident lie when no upstream is configured — the exact under-report the preflight exists to prevent.
Task 3: fix round 1/5 (2 addressed, 1 NOT — shared `purgeCandidateSessions()` now the single expression of the rule; `unpushedCount` counts commits on no remote via `--not --remotes` and surfaces a distinct `no-remote` fact; but the purge test got WEAKER, not stronger: the implementer deleted the sessions entirely and asserted zeros against an empty table, while its report claimed it had added direct DB inserts. Controller verified: zero INSERTs in the file. commits 51c1c4c..85aa098)
Task 3: fix round 2/5 (1 addressed, 0 open — handed to Codex gpt-5.6-terra effort medium, which built the discriminating fixture (counted / active / bound / sibling-prefix sessions + seeded events and token rows), ran all three clause-deletion experiments and reported the failure each produced; commit d247c37)
Task 3: controller re-verified independently — removed the bound-session exclusion by hand, test failed `2 !== 1`, file restored, `git diff server/lib/lanes.js` empty. 7/7 lifecycle tests, 815 server tests.
Task 3: complete (commits 6138944..d247c37, review clean)
Tooling note: Codex cannot run through `codex-rescue` here — the subagent's Bash sandbox makes Codex's own bwrap fail with `loopback: Failed RTM_NEWADDR`. Working invocation, with the user's explicit approval to drop the sandbox for it: `codex exec --dangerously-bypass-approvals-and-sandbox -m gpt-5.6-terra -c model_reasoning_effort=medium "<prompt>"` run from the controller's Bash with dangerouslyDisableSandbox.
Task 4: implemented by Codex gpt-5.6-terra (effort medium) via `codex exec`, review by Claude. Review — no Critical; same-origin guard genuinely applied, slug cannot escape LANES_ROOT, failure path leaves the lane managed/failed/removable with git's stderr and no orphan directory (reviewer reproduced it against a real repo). Codex also updated ARCHITECTURE.md, docs/API.md, server/README.md and added an OpenAPI fragment — ruled NOT scope creep: .claude/skills/update-project-docs mandates exactly those files for an API change. 1 Important, 3 Minor.
Task 4: fix round 1/5 (4 addressed, 0 open — boot sweep marks any still-'provisioning' lane failed with an explanatory note (a lane killed mid-provision could otherwise never age into 'dead', since a fresh managed lane has no session_id for classifyLiveness to measure); 409 EDUPCWD and the `base` default now documented in both OpenAPI and docs/API.md; directory-suffix loop capped at 50 with a 409 and a test that pre-creates exactly the 51 colliding directories needed to reach it; commits d524cd3..cf8ebbf)
Task 4: re-review traced the boot ordering — the sweep runs as a microtask off `server.listen`'s resolve, before the poll phase can dispatch a connection, and is one synchronous better-sqlite3 UPDATE with no yield point, so no request can slip a new provisioning lane into the sweep. Confirmed it writes only status/notes/updated_at, leaving kind/cwd/branch/source_repo/base_branch/slug intact.
Task 4: minor (deferred): recoverInterruptedProvisioning writes raw SQL instead of going through updateLane/PATCHABLE; the boot handler logs err.message without the stack.
Task 4: complete (commits d247c37..cf8ebbf, review clean) — 821 server tests pass
Task 5: implemented by Codex terra, reviewed by Claude Opus. Review — guard work (three checks, force gating, purge honesty, error mapping, lock) clean, but 2 Critical: (a) "kill the run and await its exit" was vacuous — killRun sets status='killed' synchronously after SIGTERM, so the poll on status returned instantly and `git clean -fd` / `worktree remove --force` ran milliseconds later while a live Claude could still be writing into that directory; (b) `remove` had been silently narrowed to managed lanes, breaking the shipped Remove button for adopted lanes with no client path at all (api.ts had no DELETE method), and the existing test that encoded the old contract was rewritten instead of the regression being reported.
Task 5: fix round 1/5 (7 addressed, 2 NEW breakages — added `actualExitedAt`, written only in the child's exit handler, polled with a 7.5 s deadline (> killRun's 5 s escalation) and failing loudly via ERUNTIMEOUT before any git; restored `remove` for both kinds (managed tears down the worktree, adopted deletes the row only); made `expect` mandatory with the full field set; guarded DELETE; ESTALE now carries expected/current. commits 24841d4..58a94e8)
Task 5: fix round 2/5 (4 addressed, 0 open — the mandatory `expect` (my requirement) had broken the Remove button for EVERY lane with 400 EEXPECT, so the client now fetches preflight and echoes it, and surfaces errors instead of swallowing them; a child that fails to spawn emits only `error`, never `exit`, so `actualExitedAt` also set there — otherwise `claude` missing from PATH froze every destructive action for the whole reap window; killRun's SIGKILL escalation tested `!child.killed`, which Node sets true on a successful SIGTERM, so it could never fire — now keyed on real exit; PATCH guarded. commits 58a94e8..fa77725)
Task 5: controller committed one leftover line Codex left uncommitted (`preflight: r({})` in the screens-snapshot API mock, cac0d2a) and re-ran the client suite from a clean tree: 279/279.
Task 5: minor (deferred to C7): LaneCard has no Force affordance, so an unpushed reset/remove 409s with no way to retry from the UI; reset and purge are not on the card at all yet; adopted `remove` still demands force when the adopted directory has unpushed commits even though nothing is destroyed.
Task 5: complete (commits cf8ebbf..cac0d2a, review clean) — 834 server tests, 279 client tests
Task 6: written by Codex terra, whose process was killed by a wall-clock limit TWICE before it could run a single test or commit — the work survived staged. A Claude subagent then read the staged diff, verified it and committed it unchanged (00fc9fa). So the code reached review having never been run by its author; the reviewer was told that and given permission to run the CLI test file itself.
Task 6: review — no Critical, no Important. Verified field-by-field that the CLI's LANE_PREFLIGHT_FIELDS matches the server's expectedFields exactly and that `expect` is built from the freshly fetched preflight (not client-guessed); `--yes` cannot be bypassed (returns with exit 1 before any POST is constructed); `--force` only ever unlocks the unpushed gate; the adopted-lane refusal is plain language and its test asserts the file's CONTENTS survive; no spawnSync, no bare sleeps, no vacuous tests. Reviewer independently ran the CLI suite: 10/10.
Task 6: minor (deferred): `ccam help` shows `[--yes]` while the prose shows it unbracketed; no test covers the ESTALE 409 or the provisioning failed/timeout print paths; the destructive tests share a mutable `managedLane` across describe blocks.
Task 6: complete (commits cac0d2a..00fc9fa, review clean) — 839 server tests
Task 7: Codex wrote the component work and was killed by a wall-clock limit a third time, leaving it uncommitted and missing both the modal tests and all the documentation; a Claude agent audited it, wrote those, and committed (2a58f5a).
Task 7: that agent also found and fixed a real pre-existing bug OUTSIDE its brief: client/src/i18n/index.ts never registered the `lanes` namespace, so no string on the Lanes page had ever resolved — every locale, including the vi/ko/zh translations shipped in the earlier plan's Task 7, rendered as raw keys. Both that task's reviewer and its re-reviewer had passed i18n as complete; both had only checked that the locale files contained the keys, never that the namespace was loaded.
Task 7: review — spec MET throughout; docs fact-checked line by line against the code (three safety checks, `clean -fd` without `-x`, preflight field lists, bytesEstimate derivation) and found truthful, including correctly documenting that LANE_BASE_BRANCH / LANE_BRANCH_PREFIX do NOT exist rather than inventing support. 1 Critical.
Task 7: Critical (open, goes to C8) — `no-remote` sits in the same `blocked[]` array as hard blockers, so a managed lane in a repo with no remote can never be reset or removed from the UI: unpushedCount counts every commit when there is no remote, giving `blocked = ["unpushed-commits","no-remote"]`, the modal treats anything but `unpushed-commits` as a hard block, and its Force checkbox only appears when `blocked.length === 1`. The server gates nothing on `no-remote` and the CLI works fine — the UI locks only itself, permanently, for a common case.
Task 7: minor (goes to C8): LaneCard renders `t("status.<status>")` but no locale has any `status.*` key, and i18next returns the key on a miss, so the `||` fallback never fires — every lane shows literal text like `status.active` right now.
Task 7: complete (commits 00fc9fa..2a58f5a, 1 Critical carried into C8) — 287 client tests, 840 server tests
Task 8 (added after Task 7's Critical): implemented by a Claude subagent. Root-cause fix rather than a patch — `blocked[]` had been conflating three kinds of thing, so it now carries only the action-preventing conditions (adopted, missing, unreadable) plus `unpushed-commits` (the one Force overrides), while purely informational facts (`no-remote`) moved to a new `warnings[]`. The modal needed no logic change at all: it was locked out purely because the server mislabelled `no-remote` as blocking. Also landed: status.* i18n keys for all four locales, real support for LANE_BASE_BRANCH and LANE_BRANCH_PREFIX (the spec had promised them, the code had hardcoded main and feat/), CLI tests for the ESTALE print path and for failed provisioning, and `--yes` presented as mandatory everywhere.
Task 8: review — APPROVE, no Critical or Important. Reviewer enumerated every blocked[] combination the server can emit and confirmed the modal and the server now agree on all eight; ran the ESTALE test three times (12/12 each, ~300 ms) and confirmed it is structural rather than timing-based, since GET /preflight takes no lock while POST does; verified the env-var tests actually set the variables and observe their values rather than asserting a default that would pass anyway; fact-checked the docs against the code and found no inaccurate sentence.
Task 8: minor (deferred, for final-review triage): `lanes.status` is not validated at the API layer — `POST /:id/stage` and `ccam stage --status <s>` accept an arbitrary string, so a rogue value would reproduce the raw-key badge bug this task fixed. The robust fix is a t() defaultValue in LaneCard rather than restricting what an agent may declare.
Task 8: complete (commits 2a58f5a..3139de9, review clean) — 844 server tests, 292 client tests
FINAL whole-branch review (Opus): READY WITH FIXES — 2 Critical, 6 Important, 9 Minor. Safety core sound: every caller of resetWorktree/removeWorktree/purgeLaneSessions/deleteLane enumerated across server/, bin/, scripts/, mcp/ (the CLI goes over HTTP, so nothing outside the routes reaches the git layer), the guard sits INSIDE both destructive functions rather than only upstream, and the migration was verified against real SQLite including a simulated mid-migration crash.
FINAL: the reviewer earned its verdict by mutation — it deleted `await assertDestroyable(lane)` from removeWorktree and all 844 tests stayed green, proving the remove-path guard was pinned by nothing. The same deletion in resetWorktree failed a test.
FINAL: controller raised the reviewer's Important I3 to Critical — `cwd LIKE ? || '/%'` was unescaped and `_` is a LIKE wildcard, while every managed lane directory this branch creates is named `<repo>__<slug>`. A lane at /root/myrepo__feat-foo purged sessions belonging to /root/myrepoXXfeat-foo, and the preflight counted the victims too, so the confirmation was consistently wrong rather than detectably wrong. The old fixture (/tmp/wt-purge) had no underscore, so nothing could have caught it.
Fix wave (one commit, 513235a, Opus): all 16 findings addressed, +997/-151 across 28 files, 844→857 server tests, 292→297 client tests. Deletion experiments run and reported for C2, C3, I1 and I2. The fixer also found a FOURTH instance of "the UI refusing what the server permits" in bin/ccam.js and disclosed it rather than fixing it silently.
Fix-wave re-review (Opus): READY. Verified 857/857 itself, re-ran the C2 mutation in a throwaway copy, and built the four-way table (adopted/missing/unreadable/no-remote × reset/remove/purge × modal/server/CLI) — no square where the UI is stricter than the server, no third instance of the defect.
FINAL: the re-reviewer found what neither the fix wave nor I had: a FOURTH refusal shape for C2 that git does NOT incidentally refuse — an adopted lane legitimately pointing at a worktree the user registered themselves. With the guard removed it destroyed the directory silently with no error at all. Check 1 is the sole protection for that case, so the guard was load-bearing well beyond the three shapes the tests cover.
FINAL: it also disproved the fix wave's own reasoning on `unreadable` + `remove` by reproducing a corrupt-.git worktree: `git worktree remove --force` and even `--force --force` both refuse (code 128), so the promise written into `destructive.notice.unreadable` in all four locales and into two doc lines — that removal is forced and git's entry cleared — is FALSE for that shape. Decision to unblock stands (blocking recreates the defect class), but the copy over-promises.
FINAL: registered as a known fact about the branch, not a surprise — `removeWorktree`'s new prune path deliberately does NOT call assertDestroyable; it runs check 1 plus a lexical check 2 and substitutes `worktree prune` for check 3, fires only when the directory does not exist, and is pinned by a test that still refuses a missing cwd outside LANES_ROOT.
Open follow-ups, none blocking, surfaced to the user: (1) reword the `unreadable` notice + two doc lines to promise an attempt rather than success, add a prune-style fallback so an unreadable lane is genuinely removable, add the two missing modal tests; (2) wrap `start` in withLaneLock so its atomicity is an invariant rather than a property of the current await-free code; (3) add an openapi.yaml drift check to CI; (4) separate triage for the pre-existing unguarded `POST /api/lanes/` and `POST /api/lanes/:id/stage`.
docs/superpowers/logs/2026-07-28-worktree-lanes-followups.md
+204
View File
@@ -0,0 +1,204 @@
# Worktree-lanes: final-review follow-ups
Base: bf312c6 (branch `feat/worktree-lanes`). One commit for the whole set.
## Follow-up 1: unreadable worktrees were genuinely unremovable
### (a) Made an unreadable managed lane genuinely removable
`server/lib/worktree.js`: `removeWorktree` unconditionally called
`git worktree remove --force`, which git refuses outright — even with a
second `--force` — when the worktree's OWN `.git` pointer fails its own
validation (a corrupt/garbage `.git` file). All three `assertDestroyable`
checks pass for this shape (the directory exists, resolves inside
`LANES_ROOT`, and is still listed by the source repo), so the lane was stuck:
`removeWorktree` threw, the route 500'd, the lane row survived.
Fix: wrapped the `git worktree remove --force` call in a try/catch. On
failure, `findWorktreeAdminDir(sourceRepo, cwd)` locates the worktree's
administrative directory under the source repo's common dir
(`<common>/worktrees/<name>`) by reading each entry's `gitdir` file — that
file's content is the absolute path to the worktree's own `.git` file, read
from the *source repo's* side, so it still resolves correctly even though the
worktree's own `.git` file is corrupt. If found, `fs.rmSync` deletes only
that administrative directory (never the worktree directory itself), which
deregisters the worktree from `git worktree list`. Branch delete and lane-row
deletion then proceed exactly as for every other remove. If no matching
admin directory is found (paranoid case — should not happen for a lane that
passed all three checks), the original git error is rethrown rather than
silently continuing, so the failure is never swallowed.
This is a genuine deregistration, not a workaround: it never touches the
worktree's own files, and after it runs, `git worktree remove --force` on
the same path correctly reports `'...' is not a working tree` — proof the
repo no longer thinks it manages that directory.
### Experiment (raw output)
Built a real corrupt worktree under a throwaway repo and ran the exact
sequence a reviewer had already reproduced, then the new fallback:
```
=== remove --force (expect fail) ===
fatal: validation failed, cannot remove working tree: '/tmp/tmp.4kZLeHvThU/lanes/src__corrupt/.git' is not a .git file, error code 5
exit=128
=== manually delete admin dir ===
(no output)
=== worktree list after manual admin removal ===
worktree /tmp/tmp.4kZLeHvThU/src
HEAD 0a1ed2bb01a8ac88b758180d569997b8c0bb22e8
branch refs/heads/main
=== branch still exists? ===
feat/corrupt
=== worktree directory + corrupted .git file still present? ===
total 16
drwxrwxr-x 2 smartgiftailab smartgiftailab 4096 ... .
drwxrwxr-x 3 smartgiftailab smartgiftailab 4096 ... ..
-rw-rw-r-- 1 smartgiftailab smartgiftailab 8 ... .git
-rw-rw-r-- 1 smartgiftailab smartgiftailab 6 ... README.md
garbage
=== git worktree remove again now (expect: not a working tree, confirms deregistered) ===
fatal: '/tmp/tmp.4kZLeHvThU/lanes/src__corrupt' is not a working tree
exit=128
```
This confirms: (1) `remove --force` genuinely refuses a corrupt worktree,
matching the earlier report exactly (error code 5 here vs 7 in the original
report — git version difference, same refusal); (2) deleting only
`.git/worktrees/<name>` deregisters the worktree from `git worktree list`
without touching the worktree directory or its files; (3) after
deregistration git itself confirms the path is no longer a working tree.
The actual code path (`removeWorktree` against a real corrupt worktree
inside `LANES_ROOT`, through `findWorktreeAdminDir`) was then exercised by
the new server test below and produced the same result end to end.
### (b) Reworded the false promise
`destructive.notice.unreadable` in `client/src/i18n/locales/{en,zh,vi,ko}/lanes.json`
no longer promises forced success. New English text: "The lane directory
cannot be read as a Git worktree. Removal is attempted; if Git itself
refuses, its worktree record is cleared directly instead. Either way, the
directory itself is never touched." zh/vi/ko were translated with the same
meaning (not machine-transliterated word-for-word), matching each locale's
existing terminology for "lane"/"worktree"/"record" already used elsewhere
in the same file.
`docs/API.md` (blocked/remove paragraph) and `docs/LANES.md` (the three
safety checks section) both had the same "force-removes an unreadable one"
claim; both now describe the attempt-then-fallback behavior and name the
corrupt-`.git`-pointer case explicitly.
### (c) Added the two missing modal tests
`client/src/components/lanes/__tests__/DestructiveLaneModal.test.tsx`:
- `disables RESET when the worktree directory is unreadable` — pins
`unreadable` staying in `HARD_BLOCKERS.reset`.
- `ENABLES remove when the worktree directory is unreadable` — pins
`unreadable` staying absent from `HARD_BLOCKERS.remove`, and that the
reworded notice text renders and the modal still echoes back the exact
`expect` block on confirm.
### Regression test for (a)
`server/__tests__/worktree.test.js`: new test builds a real worktree, writes
garbage into its `.git` file (reproducing the exact corruption), asserts
`git status` inside it fails (sanity), calls `removeWorktree` directly, then
asserts: the source repo no longer lists it, its branch is gone, **and** the
directory plus a file written into it still exist afterward untouched.
## Follow-up 2: `start` was atomic by accident, not by invariant
`server/routes/lanes.js`: the `start` case read `lane.run_id` and later wrote
a new one with no `await` in between — atomic today only because nothing
yields the event loop in that stretch. Wrapped the whole check-then-spawn in
`withLaneLock(lane.id, async () => {...})`, re-fetching the lane inside the
lock (a concurrent `remove` could have deleted the row while queued, so a
`missing` case now returns 404 `ENOLANE` instead of dereferencing a null
lane — a real edge case introduced by the lock itself, not present before).
The `409 ERUNLIVE` code and message are unchanged.
Regression test (`server/__tests__/lane-lifecycle.test.js`,
`"start is serialized behind the per-lane lock..."`): holds the same lane's
lock directly from the test (`withLaneLock(lane.id, () => new Promise(...))`),
fires `POST .../start` while that lock is held, confirms the request has NOT
settled 50ms later and that no run_id was written, then releases the held
lock and confirms `start` only proceeds (spawns, returns 200) after release.
This proves the route genuinely shares the per-lane lock rather than proving
only that the current zero-await code happens to be atomic.
## Follow-up 3: openapi.yaml drift check in CI
`.github/workflows/ci.yml`: added a step to the existing "🧹 Check Formatting"
job — `npm run openapi:yaml` (regenerate) followed by
`git diff --exit-code openapi.yaml`. No new job, no git hook (the pre-commit
hook is already slow, per instruction). Verified locally: regenerating
produces zero diff against the currently committed file.
## Follow-up 4: missing `sameOriginGuard` on two mutating routes
`server/routes/lanes.js`: added `sameOriginGuard` to `POST /api/lanes/`
(lane creation) and `POST /api/lanes/:id/stage` (stage reporting) —
previously the only two mutating lane routes without it.
Checked both callers before changing anything:
- `bin/ccam.js`'s `post()` helper (backing `ccam lanes add` and `ccam stage`)
only ever sends a `Content-Type` header, never `Origin` or `Referer` — the
guard passes any request with no Origin header through unconditionally
(same rule already relied on by every other guarded lane route). Verified
end to end by re-running `server/__tests__/lanes-cli.test.js` (13/13) after
the change — both `ccam lanes add --repo` and the destructive-lifecycle
CLI flows (which call `stage` indirectly via `clear`) still pass.
- `grep -rn "api/lanes" mcp/ scripts/` — no hits. Nothing else calls these
routes.
No legitimate caller was broken; no need to weaken the guard or stop and
ask.
Regression tests added to `server/__tests__/lanes-api.test.js`:
`"rejects cross-origin lane creation"` and `"rejects cross-origin stage
reporting"`, mirroring the existing cross-origin PATCH/DELETE tests exactly
(assert `403 EBADORIGIN`, and for stage, that the lane's stage was not
changed).
## Commands run, with tallies
- `node --test server/__tests__/worktree.test.js` → 19/19 (was 18)
- `node --test server/__tests__/lane-lifecycle.test.js` → 29/29 (was 28)
- `node --test server/__tests__/lanes-api.test.js` → 22/22 (was 20)
- `node --test server/__tests__/lanes-cli.test.js` → 13/13 (unchanged, re-run
as a caller check for follow-up 4)
- `npm run test:server` → **861/861** (baseline 857)
- `cd client && npx vitest run src/components/lanes/__tests__/DestructiveLaneModal.test.tsx`
→ 16/16 (was 14)
- `npm run test:client` → **299/299** (baseline 297)
- `npm run build` → succeeded (client build, `tsc -b && vite build`)
- `bash .claude/skills/file-headers/scripts/check-headers.sh` → exit 0
- `node scripts/generate-openapi-yaml.js && git diff --exit-code openapi.yaml`
→ exit 0, no diff
- `npm run format:check` → all files pass
## Left out / deferred, with reasons
- Did not touch the "Each verb → remove" bullet in `docs/LANES.md` (a
different paragraph from the one named in the brief) — it only describes
the ordinary managed-removal steps and makes no claim about the
corrupt-`.git` case, so it was not inaccurate and editing it would have
been unrequested scope.
- Did not add a CLI-level test reproducing the corrupt-`.git` removal
through `ccam lanes remove` — the brief asked for "a test proving (a)"
which the new `worktree.test.js` case already does directly against
`removeWorktree` (the same function the route and CLI both call); adding a
second, slower end-to-end CLI version would duplicate coverage without
proving anything new.
- Did not add a real concurrent-`claude`-process test for follow-up 2 (two
genuine `POST /start` calls racing against real spawned processes) — the
codebase's own existing tests avoid ever spawning `claude` twice
concurrently for exactly this reason (slow, and the real `claude` binary's
behavior isn't what's under test). The lock-holding test proves the same
invariant deterministically without that cost.
docs/superpowers/plans/2026-03-16-remove-native-sqlite-dependency.md
+240
View File
@@ -0,0 +1,240 @@
# Remove Native SQLite Dependency — Implementation Plan
> **For agentic workers:** REQUIRED: Use superpowers:subagent-driven-development (if subagents available) or superpowers:executing-plans to implement this plan. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Replace `better-sqlite3` (native C++ module requiring Python/build tools) with a compatibility layer over Node.js built-in `node:sqlite`, so `npm install` succeeds on any machine without native compilation tools.
**Architecture:** Create `server/compat-sqlite.js` — a thin wrapper that gives `DatabaseSync` (from `node:sqlite`) the same API as `better-sqlite3`. Move `better-sqlite3` to `optionalDependencies` so it's preferred when prebuilds are available but doesn't block install. The `server/db.js` loader tries `better-sqlite3` first, falls back to the compat wrapper. Update minimum Node version to 22.
**Tech Stack:** Node.js `node:sqlite` (DatabaseSync), existing Express/WS server
---
## File Structure
| File | Action | Responsibility |
|------|--------|---------------|
| `server/compat-sqlite.js` | **Create** | Wrapper class: DatabaseSync → better-sqlite3 API |
| `server/db.js` | **Modify** (line 1) | Try better-sqlite3, fallback to compat wrapper |
| `scripts/clear-data.js` | **Modify** (line 8) | Same fallback import |
| `package.json` | **Modify** | Move better-sqlite3 to optionalDependencies, bump engines to >=22 |
| `server/__tests__/api.test.js` | **Modify** (lines 952-959) | Fix `db.pragma()` calls to work with both backends |
---
## Chunk 1: Core Implementation
### Task 1: Create `server/compat-sqlite.js`
**Files:**
- Create: `server/compat-sqlite.js`
- [ ] **Step 1: Write the compat wrapper**
The wrapper must bridge these API differences:
| better-sqlite3 | node:sqlite (DatabaseSync) |
|----------------|---------------------------|
| `new Database(path)` | `new DatabaseSync(path)` |
| `db.pragma("key = value")` | `db.exec("PRAGMA key = value")` |
| `db.pragma("key")` → value | `db.prepare("PRAGMA key").get()` → `{key: value}` |
| `db.pragma("key", { simple: true })` → value | same as above, extract single value |
| `db.transaction(fn)` → wrapper fn | manual `BEGIN`/`COMMIT`/`ROLLBACK` |
| `db.prepare(sql)` → stmt with `.run()`, `.get()`, `.all()` | identical API |
| `db.exec(sql)` | identical |
| `db.close()` | identical |
```js
// server/compat-sqlite.js
const { DatabaseSync } = require("node:sqlite");
class Database {
constructor(filePath) {
this._db = new DatabaseSync(filePath);
}
exec(sql) {
this._db.exec(sql);
return this;
}
pragma(str, options) {
if (str.includes("=")) {
this._db.exec(`PRAGMA ${str}`);
return undefined;
}
const row = this._db.prepare(`PRAGMA ${str}`).get();
if (!row) return undefined;
const keys = Object.keys(row);
if (options?.simple || keys.length === 1) return row[keys[0]];
return row;
}
prepare(sql) {
return this._db.prepare(sql);
}
transaction(fn) {
const db = this._db;
const wrapper = (...args) => {
db.exec("BEGIN");
try {
const result = fn(...args);
db.exec("COMMIT");
return result;
} catch (err) {
db.exec("ROLLBACK");
throw err;
}
};
return wrapper;
}
close() {
this._db.close();
}
}
module.exports = Database;
```
- [ ] **Step 2: Verify the wrapper works standalone**
Run: `node -e "const DB = require('./server/compat-sqlite'); const db = new DB(':memory:'); db.pragma('journal_mode = WAL'); db.exec('CREATE TABLE t (id INTEGER PRIMARY KEY, v TEXT)'); const s = db.prepare('INSERT INTO t (v) VALUES (?)'); console.log(s.run('hi')); console.log(db.prepare('SELECT * FROM t').all()); const tx = db.transaction((items) => { for (const i of items) s.run(i); }); tx(['a','b','c']); console.log(db.prepare('SELECT COUNT(*) as c FROM t').get()); db.close(); console.log('OK')"`
Expected: `OK` printed at the end with correct query results.
- [ ] **Step 3: Commit**
```bash
git add server/compat-sqlite.js
git commit -m "feat: add node:sqlite compat wrapper for better-sqlite3 API"
```
---
### Task 2: Update `server/db.js` to use fallback import
**Files:**
- Modify: `server/db.js:1`
- [ ] **Step 1: Replace the import**
Change line 1 from:
```js
const Database = require("better-sqlite3");
```
To:
```js
let Database;
try {
Database = require("better-sqlite3");
} catch {
Database = require("./compat-sqlite");
}
```
- [ ] **Step 2: Verify server starts**
Run: `node -e "process.env.DASHBOARD_DB_PATH = require('path').join(require('os').tmpdir(), 'test-fallback-' + Date.now() + '.db'); const { db, stmts } = require('./server/db'); console.log('stmts keys:', Object.keys(stmts).length); stmts.insertSession.run('test-1', 'Test', 'active', null, null, null); console.log(stmts.getSession.get('test-1')); db.close(); console.log('OK')"`
Expected: Prints statement count (39), session row, and `OK`.
- [ ] **Step 3: Commit**
```bash
git add server/db.js
git commit -m "feat: fallback to node:sqlite when better-sqlite3 unavailable"
```
---
### Task 3: Update `scripts/clear-data.js` to use fallback import
**Files:**
- Modify: `scripts/clear-data.js:8`
- [ ] **Step 1: Replace the import**
Change line 8 from:
```js
const Database = require("better-sqlite3");
```
To:
```js
let Database;
try {
Database = require("better-sqlite3");
} catch {
Database = require("../server/compat-sqlite");
}
```
- [ ] **Step 2: Commit**
```bash
git add scripts/clear-data.js
git commit -m "fix: use fallback sqlite import in clear-data script"
```
---
### Task 4: Update `package.json`
**Files:**
- Modify: `package.json`
- [ ] **Step 1: Move better-sqlite3 to optionalDependencies, bump engines**
Move `"better-sqlite3": "^11.7.0"` from `dependencies` to `optionalDependencies`.
Change engines from `"node": ">=18.0.0"` to `"node": ">=22.0.0"`.
- [ ] **Step 2: Commit**
```bash
git add package.json
git commit -m "chore: make better-sqlite3 optional, require Node >= 22"
```
---
### Task 5: Fix test pragma calls
**Files:**
- Modify: `server/__tests__/api.test.js:952-959`
- [ ] **Step 1: Fix pragma calls in Database Integrity tests**
The tests call `db.pragma("journal_mode", { simple: true })` and `db.pragma("foreign_keys", { simple: true })`. The compat wrapper supports `{ simple: true }`, so these should work as-is. However, WAL mode isn't available for in-memory databases (returns "memory"). The test creates a file-based DB via `TEST_DB`, so WAL should work.
No change needed — verify by running tests.
- [ ] **Step 2: Run full test suite**
Run: `node --test server/__tests__/api.test.js`
Expected: All tests pass.
- [ ] **Step 3: Run setup to verify npm install succeeds without Python**
Run: `npm run setup`
Expected: Install succeeds (better-sqlite3 may warn but won't fail since it's optional).
---
### Task 6: Update documentation
**Files:**
- Modify: `SETUP.md` (if it mentions better-sqlite3 or Python requirements)
- [ ] **Step 1: Check and update SETUP.md**
Remove any mentions of Python or build tools as requirements. Note that Node >= 22 is required.
- [ ] **Step 2: Commit all remaining changes**
```bash
git add -A
git commit -m "docs: update setup requirements for native-free SQLite"
```
docs/superpowers/plans/2026-03-20-jsonl-reading-performance.md
+875
View File
@@ -0,0 +1,875 @@
# JSONL Reading Performance Optimization
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Eliminate redundant full-file reads of JSONL transcript files by caching extracted token data and using incremental reads.
**Architecture:** Add a lightweight in-memory cache keyed by `(transcriptPath, mtime, size)` that stores the extracted `{tokensByModel, compaction}` result. On each hook event, stat the file first — if unchanged, return cached result. For files that did change, use byte-offset tracking to only read new lines appended since last parse. The periodic compaction scanner shares this same cache.
**Tech Stack:** Node.js `fs.statSync`, in-memory `Map` cache, byte-offset tracking via `fs.openSync`/`fs.readSync`.
---
## Performance Problem Analysis
### Current Behavior
Three code paths read JSONL files **fully, synchronously, with zero caching**:
| Path | File | Trigger | Frequency |
|------|------|---------|-----------|
| `extractTokensFromTranscript()` | `server/routes/hooks.js:15-62` | Every POST `/api/hooks/event` with `transcript_path` | 1-10x/min per active session |
| `findCompactionsInFile()` | `scripts/import-history.js:658-674` | 2-minute periodic scan | Every 2 min × active sessions |
| `parseSessionFile()` | `scripts/import-history.js:22-131` | Server startup import | Once per JSONL file at startup |
### Why This Hurts
1. **`extractTokensFromTranscript` is the hot path.** Called on *every* hook event. For a session producing 5 events/min with a 10K-line JSONL (typical long session), that's 5 full file reads + 50K `JSON.parse` calls per minute.
2. **JSONL files are append-only** (until compaction rewrites them). Between hook events, only a few new lines are appended. Reading the entire file to re-sum tokens that haven't changed is pure waste.
3. **`readFileSync` blocks the event loop.** Long sessions (50K+ lines, several MB) block the Express request handler for tens of milliseconds, stalling concurrent hook ingestion and API responses.
4. **Periodic scanner duplicates work.** `findCompactionsInFile` re-reads the same files that `extractTokensFromTranscript` already parsed seconds ago.
### Quantified Impact (estimated)
| Session Length | Lines | File Size | Parse Time (sync) | Events/min | Wasted CPU/min |
|---------------|-------|-----------|--------------------|------------|----------------|
| Short (30min) | 500 | ~100KB | ~2ms | 3 | ~6ms |
| Medium (2hr) | 5,000 | ~1MB | ~15ms | 5 | ~75ms |
| Long (8hr+) | 20,000 | ~4MB | ~50ms | 8 | ~400ms |
| Marathon (24hr) | 50,000+ | ~10MB+ | ~120ms+ | 10 | ~1.2s |
With multiple concurrent sessions, this compounds. The 2-minute scanner adds another full read per active session on top.
---
## File Structure
| File | Responsibility | Action |
|------|---------------|--------|
| `server/lib/transcript-cache.js` | In-memory cache + incremental reader for JSONL files | **Create** |
| `server/lib/__tests__/transcript-cache.test.js` | Unit tests for cache + incremental read logic | **Create** |
| `server/routes/hooks.js` | Hook event handler — swap `extractTokensFromTranscript` to use cache | **Modify** (lines 15-62, 353-354) |
| `scripts/import-history.js` | Periodic compaction scanner — swap `findCompactionsInFile` to use cache | **Modify** (lines 658-674) |
| `server/index.js` | Wire cache into periodic scanner; add cache stats to settings | **Modify** (lines 104-128) |
| `server/routes/settings.js` | Expose cache stats in `/api/settings/info` | **Modify** |
---
## Task 1: Create the Transcript Cache Module
**Files:**
- Create: `server/lib/transcript-cache.js`
- Test: `server/lib/__tests__/transcript-cache.test.js`
### Design
```
Cache entry = {
mtime: number, // file modification time (ms)
size: number, // file size in bytes
bytesRead: number, // how far we've read into the file
tokensByModel: {}, // accumulated token sums
compaction: null|{}, // compaction entries found so far
}
On read request:
1. fs.statSync(path) → get mtime + size
2. Cache hit? (same mtime + size) → return cached result
3. File shrunk or mtime changed with smaller size? → compaction rewrite → full re-read, reset cache
4. File grew? (size > bytesRead) → incremental read from bytesRead → parse new lines → merge into cached totals
5. Store updated entry, return result
```
- [ ] **Step 1: Create test file with first test — cache miss triggers full read**
```javascript
// server/lib/__tests__/transcript-cache.test.js
const { describe, it, beforeEach, afterEach } = require("node:test");
const assert = require("node:assert/strict");
const fs = require("fs");
const path = require("path");
const os = require("os");
let tmpDir;
let TranscriptCache;
function writeJsonl(filePath, entries) {
fs.writeFileSync(filePath, entries.map((e) => JSON.stringify(e)).join("\n") + "\n");
}
describe("TranscriptCache", () => {
beforeEach(() => {
tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), "tc-test-"));
// Fresh require to reset module-level state
delete require.cache[require.resolve("../../lib/transcript-cache")];
TranscriptCache = require("../../lib/transcript-cache");
});
afterEach(() => {
fs.rmSync(tmpDir, { recursive: true, force: true });
});
it("should extract tokens on first read (cache miss)", () => {
const file = path.join(tmpDir, "session.jsonl");
writeJsonl(file, [
{ message: { model: "claude-sonnet-4-20250514", usage: { input_tokens: 100, output_tokens: 50 } } },
{ message: { model: "claude-sonnet-4-20250514", usage: { input_tokens: 200, output_tokens: 75 } } },
]);
const cache = new TranscriptCache();
const result = cache.extract(file);
assert.deepStrictEqual(result.tokensByModel, {
"claude-sonnet-4-20250514": { input: 300, output: 125, cacheRead: 0, cacheWrite: 0 },
});
assert.strictEqual(result.compaction, null);
});
});
```
- [ ] **Step 2: Run test to verify it fails**
Run: `node --test server/lib/__tests__/transcript-cache.test.js`
Expected: FAIL — module not found
- [ ] **Step 3: Implement TranscriptCache with full-read path**
```javascript
// server/lib/transcript-cache.js
const fs = require("fs");
class TranscriptCache {
constructor() {
this._cache = new Map();
}
/**
* Extract token usage and compaction data from a JSONL transcript file.
* Uses stat-based caching — returns cached result if file hasn't changed.
* Returns null if file doesn't exist or has no data.
*/
extract(transcriptPath) {
if (!transcriptPath) return null;
try {
const stat = fs.statSync(transcriptPath);
const key = transcriptPath;
const cached = this._cache.get(key);
// Cache hit: file unchanged
if (cached && cached.mtimeMs === stat.mtimeMs && cached.size === stat.size) {
return cached.result;
}
// Full read (cache miss or file was rewritten/compacted)
const result = this._fullRead(transcriptPath);
this._cache.set(key, {
mtimeMs: stat.mtimeMs,
size: stat.size,
bytesRead: stat.size,
tokensByModel: result ? { ...result.tokensByModel } : null,
compaction: result ? result.compaction : null,
result,
});
return result;
} catch {
return null;
}
}
_fullRead(filePath) {
const content = fs.readFileSync(filePath, "utf8");
return this._parseContent(content);
}
_parseContent(content) {
const tokensByModel = {};
let compaction = null;
for (const line of content.split("\n")) {
if (!line) continue;
try {
const entry = JSON.parse(line);
if (entry.isCompactSummary) {
if (!compaction) compaction = { count: 0, entries: [] };
compaction.count++;
compaction.entries.push({
uuid: entry.uuid || null,
timestamp: entry.timestamp || null,
});
}
const msg = entry.message || entry;
const model = msg.model;
if (!model || model === "<synthetic>" || !msg.usage) continue;
if (!tokensByModel[model]) {
tokensByModel[model] = { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 };
}
tokensByModel[model].input += msg.usage.input_tokens || 0;
tokensByModel[model].output += msg.usage.output_tokens || 0;
tokensByModel[model].cacheRead += msg.usage.cache_read_input_tokens || 0;
tokensByModel[model].cacheWrite += msg.usage.cache_creation_input_tokens || 0;
} catch {
continue;
}
}
const hasTokens = Object.keys(tokensByModel).length > 0;
if (!hasTokens && !compaction) return null;
return { tokensByModel: hasTokens ? tokensByModel : null, compaction };
}
/** Number of entries currently cached */
get size() {
return this._cache.size;
}
/** Remove a specific path from cache (e.g. when session ends) */
invalidate(transcriptPath) {
this._cache.delete(transcriptPath);
}
/** Clear all cached entries */
clear() {
this._cache.clear();
}
/** Return cache stats for diagnostics */
stats() {
return {
entries: this._cache.size,
paths: [...this._cache.keys()],
};
}
}
module.exports = TranscriptCache;
```
- [ ] **Step 4: Run test to verify it passes**
Run: `node --test server/lib/__tests__/transcript-cache.test.js`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add server/lib/transcript-cache.js server/lib/__tests__/transcript-cache.test.js
git commit -m "feat: add TranscriptCache module with stat-based caching for JSONL reads"
```
---
## Task 2: Add Cache Hit and Compaction Detection Tests
**Files:**
- Modify: `server/lib/__tests__/transcript-cache.test.js`
- [ ] **Step 1: Add test — second read with unchanged file returns cached result**
```javascript
it("should return cached result when file is unchanged", () => {
const file = path.join(tmpDir, "session.jsonl");
writeJsonl(file, [
{ message: { model: "claude-sonnet-4-20250514", usage: { input_tokens: 100, output_tokens: 50 } } },
]);
const cache = new TranscriptCache();
const r1 = cache.extract(file);
const r2 = cache.extract(file);
assert.deepStrictEqual(r1, r2);
// Same object reference proves cache hit (no re-parse)
assert.strictEqual(r1, r2);
});
```
- [ ] **Step 2: Add test — detects appended lines after file grows**
```javascript
it("should detect new data when file grows", (t) => {
const file = path.join(tmpDir, "session.jsonl");
writeJsonl(file, [
{ message: { model: "claude-sonnet-4-20250514", usage: { input_tokens: 100, output_tokens: 50 } } },
]);
const cache = new TranscriptCache();
const r1 = cache.extract(file);
assert.strictEqual(r1.tokensByModel["claude-sonnet-4-20250514"].input, 100);
// Append more data (simulates Claude writing to transcript)
fs.appendFileSync(
file,
JSON.stringify({ message: { model: "claude-sonnet-4-20250514", usage: { input_tokens: 200, output_tokens: 75 } } }) + "\n"
);
const r2 = cache.extract(file);
assert.strictEqual(r2.tokensByModel["claude-sonnet-4-20250514"].input, 300);
assert.strictEqual(r2.tokensByModel["claude-sonnet-4-20250514"].output, 125);
});
```
- [ ] **Step 3: Add test — detects compaction (file shrinks)**
```javascript
it("should do full re-read when file shrinks (compaction rewrite)", () => {
const file = path.join(tmpDir, "session.jsonl");
writeJsonl(file, [
{ message: { model: "claude-sonnet-4-20250514", usage: { input_tokens: 500, output_tokens: 200 } } },
{ message: { model: "claude-sonnet-4-20250514", usage: { input_tokens: 300, output_tokens: 100 } } },
]);
const cache = new TranscriptCache();
cache.extract(file);
// Simulate compaction — file is rewritten with fewer entries + summary
writeJsonl(file, [
{ isCompactSummary: true, uuid: "abc-123", timestamp: "2026-03-20T10:00:00Z" },
{ message: { model: "claude-sonnet-4-20250514", usage: { input_tokens: 50, output_tokens: 20 } } },
]);
const r2 = cache.extract(file);
assert.strictEqual(r2.tokensByModel["claude-sonnet-4-20250514"].input, 50);
assert.strictEqual(r2.compaction.count, 1);
assert.strictEqual(r2.compaction.entries[0].uuid, "abc-123");
});
```
- [ ] **Step 4: Add test — returns null for missing file**
```javascript
it("should return null for non-existent file", () => {
const cache = new TranscriptCache();
assert.strictEqual(cache.extract("/nonexistent/file.jsonl"), null);
assert.strictEqual(cache.extract(null), null);
assert.strictEqual(cache.extract(""), null);
});
```
- [ ] **Step 5: Add test — compaction-only extraction (for findCompactionsInFile replacement)**
```javascript
it("should expose compaction entries via extractCompactions()", () => {
const file = path.join(tmpDir, "session.jsonl");
writeJsonl(file, [
{ message: { model: "claude-sonnet-4-20250514", usage: { input_tokens: 100, output_tokens: 50 } } },
{ isCompactSummary: true, uuid: "c1", timestamp: "2026-03-20T09:00:00Z" },
{ message: { model: "claude-sonnet-4-20250514", usage: { input_tokens: 50, output_tokens: 20 } } },
{ isCompactSummary: true, uuid: "c2", timestamp: "2026-03-20T10:00:00Z" },
]);
const cache = new TranscriptCache();
const compactions = cache.extractCompactions(file);
assert.strictEqual(compactions.length, 2);
assert.strictEqual(compactions[0].uuid, "c1");
assert.strictEqual(compactions[1].uuid, "c2");
});
```
- [ ] **Step 6: Run all tests**
Run: `node --test server/lib/__tests__/transcript-cache.test.js`
Expected: All pass
- [ ] **Step 7: Commit**
```bash
git add server/lib/__tests__/transcript-cache.test.js
git commit -m "test: add cache hit, compaction, and edge case tests for TranscriptCache"
```
---
## Task 3: Add Incremental Read (Byte-Offset Tracking)
**Files:**
- Modify: `server/lib/transcript-cache.js`
- Modify: `server/lib/__tests__/transcript-cache.test.js`
This is the key optimization. JSONL files are append-only between compactions. Instead of re-reading the full file, read only the bytes appended since our last read.
- [ ] **Step 1: Add test — incremental read only parses new bytes**
```javascript
it("should only read new bytes on incremental update (not full file)", () => {
const file = path.join(tmpDir, "session.jsonl");
const line1 = JSON.stringify({ message: { model: "m1", usage: { input_tokens: 100, output_tokens: 50 } } }) + "\n";
fs.writeFileSync(file, line1);
const cache = new TranscriptCache();
cache.extract(file);
// Append a second line
const line2 = JSON.stringify({ message: { model: "m1", usage: { input_tokens: 200, output_tokens: 75 } } }) + "\n";
fs.appendFileSync(file, line2);
// Spy: check bytesRead advanced by only line2 length
const r2 = cache.extract(file);
assert.strictEqual(r2.tokensByModel["m1"].input, 300);
const entry = cache._cache.get(file);
assert.strictEqual(entry.bytesRead, Buffer.byteLength(line1 + line2, "utf8"));
});
```
- [ ] **Step 2: Update `extract()` to use incremental read path**
In `server/lib/transcript-cache.js`, update the `extract` method:
```javascript
extract(transcriptPath) {
if (!transcriptPath) return null;
try {
let stat;
try {
stat = fs.statSync(transcriptPath);
} catch {
return null;
}
const key = transcriptPath;
const cached = this._cache.get(key);
// Cache hit: file unchanged
if (cached && cached.mtimeMs === stat.mtimeMs && cached.size === stat.size) {
return cached.result;
}
// File shrunk or was rewritten (compaction) → full re-read
if (!cached || stat.size < cached.bytesRead) {
const result = this._fullRead(transcriptPath);
this._cache.set(key, {
mtimeMs: stat.mtimeMs,
size: stat.size,
bytesRead: stat.size,
tokensByModel: result ? this._cloneTokens(result.tokensByModel) : null,
compaction: result ? this._cloneCompaction(result.compaction) : null,
result,
});
return result;
}
// File grew → incremental read from last position
const newBytes = this._readFrom(transcriptPath, cached.bytesRead, stat.size);
if (newBytes) {
const incremental = this._parseContent(newBytes);
const merged = this._merge(cached, incremental);
const result = {
tokensByModel: Object.keys(merged.tokensByModel).length > 0 ? merged.tokensByModel : null,
compaction: merged.compaction,
};
if (!result.tokensByModel && !result.compaction) {
this._cache.set(key, { ...cached, mtimeMs: stat.mtimeMs, size: stat.size, bytesRead: stat.size, result: null });
return null;
}
this._cache.set(key, {
mtimeMs: stat.mtimeMs,
size: stat.size,
bytesRead: stat.size,
tokensByModel: this._cloneTokens(result.tokensByModel),
compaction: this._cloneCompaction(result.compaction),
result,
});
return result;
}
// newBytes was empty (e.g. only newlines appended)
this._cache.set(key, { ...cached, mtimeMs: stat.mtimeMs, size: stat.size, bytesRead: stat.size });
return cached.result;
} catch {
return null;
}
}
_readFrom(filePath, offset, totalSize) {
const len = totalSize - offset;
if (len <= 0) return null;
const buf = Buffer.alloc(len);
const fd = fs.openSync(filePath, "r");
try {
fs.readSync(fd, buf, 0, len, offset);
} finally {
fs.closeSync(fd);
}
return buf.toString("utf8");
}
_merge(cached, incremental) {
const tokensByModel = cached.tokensByModel ? { ...cached.tokensByModel } : {};
if (incremental && incremental.tokensByModel) {
for (const [model, tokens] of Object.entries(incremental.tokensByModel)) {
if (!tokensByModel[model]) {
tokensByModel[model] = { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 };
}
tokensByModel[model].input += tokens.input;
tokensByModel[model].output += tokens.output;
tokensByModel[model].cacheRead += tokens.cacheRead;
tokensByModel[model].cacheWrite += tokens.cacheWrite;
}
}
let compaction = cached.compaction ? this._cloneCompaction(cached.compaction) : null;
if (incremental && incremental.compaction) {
if (!compaction) compaction = { count: 0, entries: [] };
compaction.count += incremental.compaction.count;
compaction.entries.push(...incremental.compaction.entries);
}
return { tokensByModel, compaction };
}
_cloneTokens(tokensByModel) {
if (!tokensByModel) return null;
const clone = {};
for (const [model, t] of Object.entries(tokensByModel)) {
clone[model] = { ...t };
}
return clone;
}
_cloneCompaction(compaction) {
if (!compaction) return null;
return { count: compaction.count, entries: compaction.entries.map((e) => ({ ...e })) };
}
```
- [ ] **Step 3: Run tests**
Run: `node --test server/lib/__tests__/transcript-cache.test.js`
Expected: All pass
- [ ] **Step 4: Add `extractCompactions()` convenience method**
```javascript
/**
* Extract only compaction entries from a JSONL file (replacement for findCompactionsInFile).
* Uses the same cache — no duplicate reads.
*/
extractCompactions(transcriptPath) {
const result = this.extract(transcriptPath);
if (!result || !result.compaction) return [];
return result.compaction.entries;
}
```
- [ ] **Step 5: Run all tests**
Run: `node --test server/lib/__tests__/transcript-cache.test.js`
Expected: All pass
- [ ] **Step 6: Commit**
```bash
git add server/lib/transcript-cache.js server/lib/__tests__/transcript-cache.test.js
git commit -m "feat: add incremental byte-offset reads and extractCompactions to TranscriptCache"
```
---
## Task 4: Wire Cache into Hook Handler
**Files:**
- Modify: `server/routes/hooks.js` (lines 1-62, 353-354)
Replace the standalone `extractTokensFromTranscript` function with the shared `TranscriptCache` instance.
- [ ] **Step 1: Create shared cache instance and replace function**
At the top of `server/routes/hooks.js`, replace:
```javascript
// OLD (lines 15-62): the entire extractTokensFromTranscript function
```
With:
```javascript
const TranscriptCache = require("../lib/transcript-cache");
const transcriptCache = new TranscriptCache();
```
- [ ] **Step 2: Update the call site at line 353-354**
Replace:
```javascript
const result = extractTokensFromTranscript(data.transcript_path);
```
With:
```javascript
const result = transcriptCache.extract(data.transcript_path);
```
- [ ] **Step 3: Export the cache instance for use by periodic scanner**
At the bottom of hooks.js, change:
```javascript
module.exports = router;
```
To:
```javascript
module.exports = router;
module.exports.transcriptCache = transcriptCache;
```
Wait — that overwrites the router export. Instead, attach it to the router:
```javascript
router.transcriptCache = transcriptCache;
module.exports = router;
```
- [ ] **Step 4: Run existing server tests to verify no regression**
Run: `npm run test:server`
Expected: All existing tests pass
- [ ] **Step 5: Commit**
```bash
git add server/routes/hooks.js
git commit -m "refactor: replace extractTokensFromTranscript with TranscriptCache in hook handler"
```
---
## Task 5: Wire Cache into Periodic Compaction Scanner
**Files:**
- Modify: `server/index.js` (lines 86, 104-128)
The 2-minute periodic scanner currently calls `findCompactionsInFile()` which does its own full synchronous read. Replace it with the shared cache from the hooks router.
- [ ] **Step 1: Update import and use shared cache**
In `server/index.js`, in the `if (!isTest)` block where the periodic scanner is set up (~line 85):
Replace the import:
```javascript
const { importCompactions, findCompactionsInFile } = require("../scripts/import-history");
```
With:
```javascript
const { importCompactions } = require("../scripts/import-history");
const { transcriptCache } = require("./routes/hooks");
```
- [ ] **Step 2: Replace `findCompactionsInFile` calls with cache**
Replace (inside the setInterval, ~line 113):
```javascript
const compactions = findCompactionsInFile(row.tp);
```
With:
```javascript
const compactions = transcriptCache.extractCompactions(row.tp);
```
- [ ] **Step 3: Run server tests**
Run: `npm run test:server`
Expected: All pass
- [ ] **Step 4: Commit**
```bash
git add server/index.js
git commit -m "refactor: periodic compaction scanner uses shared TranscriptCache instead of standalone file reads"
```
---
## Task 6: Cache Eviction for Ended Sessions
**Files:**
- Modify: `server/routes/hooks.js`
When a session completes, its JSONL file won't be read again. Evict it from cache to prevent unbounded memory growth.
- [ ] **Step 1: Add test for cache invalidation**
Add to `server/lib/__tests__/transcript-cache.test.js`:
```javascript
it("should remove entry on invalidate()", () => {
const file = path.join(tmpDir, "session.jsonl");
writeJsonl(file, [
{ message: { model: "m1", usage: { input_tokens: 100, output_tokens: 50 } } },
]);
const cache = new TranscriptCache();
cache.extract(file);
assert.strictEqual(cache.size, 1);
cache.invalidate(file);
assert.strictEqual(cache.size, 0);
});
```
- [ ] **Step 2: Run test**
Run: `node --test server/lib/__tests__/transcript-cache.test.js`
Expected: Pass (invalidate was already implemented in Task 1)
- [ ] **Step 3: Add eviction when session ends in hooks.js**
In `server/routes/hooks.js`, find the Stop event handler section. After the session is updated to "completed", add:
```javascript
// Evict transcript from cache — session is done, no more reads expected
if (data.transcript_path) {
transcriptCache.invalidate(data.transcript_path);
}
```
Place this right after the `stmts.updateSession.run(...)` call for the Stop event that sets status to "completed".
- [ ] **Step 4: Run server tests**
Run: `npm run test:server`
Expected: All pass
- [ ] **Step 5: Commit**
```bash
git add server/routes/hooks.js server/lib/__tests__/transcript-cache.test.js
git commit -m "feat: evict transcript cache entry when session completes"
```
---
## Task 7: Expose Cache Stats in Settings API
**Files:**
- Modify: `server/routes/settings.js`
Add cache stats to the `/api/settings/info` endpoint for observability.
- [ ] **Step 1: Import cache and add stats to info response**
In `server/routes/settings.js`, add to the `GET /api/settings/info` handler:
```javascript
const { transcriptCache } = require("./hooks");
```
In the response object, add:
```javascript
transcript_cache: transcriptCache.stats(),
```
- [ ] **Step 2: Run server tests**
Run: `npm run test:server`
Expected: All pass
- [ ] **Step 3: Commit**
```bash
git add server/routes/settings.js
git commit -m "feat: expose transcript cache stats in settings info endpoint"
```
---
## Task 8: Integration Smoke Test
**Files:**
- Modify: `server/__tests__/api.test.js`
Add a test that simulates the full hook event flow with transcript file reads to verify the cache integration works end-to-end.
- [ ] **Step 1: Add integration test for cached transcript reading**
Add a new describe block to `server/__tests__/api.test.js`:
```javascript
describe("transcript cache integration", () => {
it("should extract tokens from transcript file via hook event", async () => {
// Create a temp JSONL transcript file
const tmpTranscript = path.join(os.tmpdir(), `test-transcript-${Date.now()}.jsonl`);
const entries = [
JSON.stringify({ message: { model: "claude-sonnet-4-20250514", usage: { input_tokens: 100, output_tokens: 50, cache_read_input_tokens: 10, cache_creation_input_tokens: 5 } } }),
JSON.stringify({ message: { model: "claude-sonnet-4-20250514", usage: { input_tokens: 200, output_tokens: 75, cache_read_input_tokens: 20, cache_creation_input_tokens: 10 } } }),
];
fs.writeFileSync(tmpTranscript, entries.join("\n") + "\n");
try {
// Send hook event with transcript_path
const sessionId = `cache-test-${Date.now()}`;
const res = await post("/api/hooks/event", {
hook_type: "Stop",
data: {
session_id: sessionId,
transcript_path: tmpTranscript,
cwd: "/tmp",
},
});
assert.strictEqual(res.status, 200);
// Verify tokens were stored
const costRes = await fetch(`/api/pricing/cost/${sessionId}`);
if (costRes.status === 200 && costRes.body.breakdown) {
const sonnet = costRes.body.breakdown.find((b) => b.model.includes("sonnet"));
if (sonnet) {
assert.strictEqual(sonnet.input_tokens, 300);
assert.strictEqual(sonnet.output_tokens, 125);
}
}
} finally {
fs.unlinkSync(tmpTranscript);
}
});
});
```
- [ ] **Step 2: Run full server test suite**
Run: `npm run test:server`
Expected: All pass
- [ ] **Step 3: Commit**
```bash
git add server/__tests__/api.test.js
git commit -m "test: add integration smoke test for transcript cache via hook events"
```
---
## Task 9: Final Build Verification
- [ ] **Step 1: Run all server tests**
Run: `npm run test:server`
Expected: All pass
- [ ] **Step 2: Run client build to check nothing broke**
Run: `npm run build`
Expected: Clean build, no errors
- [ ] **Step 3: Manual smoke test**
Start the dev server (`npm run dev`) and verify:
1. Hook events still process correctly
2. Token counts update in the UI
3. `/api/settings/info` shows `transcript_cache` stats
4. No errors in server console
- [ ] **Step 4: Final commit if any cleanup needed**
---
## Summary of Expected Impact
| Metric | Before | After |
|--------|--------|-------|
| File reads per hook event | 1 full read (every line) | 0 reads (cache hit) or partial read (new bytes only) |
| Parse calls per hook event | N lines × JSON.parse | 0 (cache hit) or K new lines only |
| Periodic scanner file reads | 1 full read per active session every 2min | 0 (shared cache already has data) |
| Memory overhead | None | ~1KB per active session (tokens + metadata) |
| Event loop blocking | Up to 120ms for large files | <1ms (stat only) on cache hit |
For a typical long session (20K lines, 4MB), this reduces per-event CPU cost from ~50ms to <1ms — a **50x improvement** on the hot path.
docs/superpowers/plans/2026-05-22-fix-transcript-cache-leak-plan.md
+1289
View File
File diff suppressed because it is too large Load Diff
docs/superpowers/plans/2026-07-27-lanes-pipeline.md
+1882
View File
File diff suppressed because it is too large Load Diff
docs/superpowers/plans/2026-07-28-stage-detection.md
+117
View File
@@ -0,0 +1,117 @@
# Stage Auto-Detection Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Infer a lane's stage from the hook stream the dashboard already ingests, and surface it as an explicitly-inferred amber node that can never read as done.
**Architecture:** Design doc: `docs/superpowers/specs/2026-07-28-stage-detection-design.md` — read it once before Task 1. Rules live in the pipeline template JSON, not in code. One pure function (`server/lib/stage-detect.js`) turns an event into a candidate node; the existing fail-safe block in `touchLaneFromHook` applies it under a forward-only, write-on-change guard; three additive columns hold the result; the client renders it dashed-amber and never green.
**Tech Stack:** Node 18+, Express, better-sqlite3, `node:test` (server), React 18 + TypeScript + Vitest (client).
## Global Constraints
- Branch: `feat/stage-detection`, cut from the head of `feat/worktree-lanes`. Never work on `master`.
- Every `.js/.ts/.tsx` file created or modified MUST start with a file overview comment plus the exact line `@author Nguyễn Ngọc Trí Vĩ <vinnt@smartgift.vn>`. Verify with `bash .claude/skills/file-headers/scripts/check-headers.sh` (must exit 0).
- **Detection never writes `lanes.stage`.** It writes only `detected_stage`, `detected_signal`, `detected_at`. Declared stage keeps its exact current meaning.
- **Inference never renders `done`.** A detected node reaches `passed-no-evidence` at most.
- **The hook path must never throw.** Everything added to `server/routes/hooks.js` lives inside the existing try/catch that already swallows lane bookkeeping errors. Claude Code waits on `POST /api/hooks/event`.
- Schema changes are additive with one probe per column (`try { SELECT col } catch { ALTER } `), so a crash mid-migration self-heals on the next boot.
- Preserve existing behavior: no existing route, response field, WebSocket type, or CLI command changes meaning. `lane_update` stays the only lane WS type.
- Server CommonJS. No new npm dependencies. Server tests `node:test` + `node:assert/strict`; client tests Vitest + Testing Library. Exact-value assertions; no bare sleeps.
- Docs move with behavior: `docs/LANES.md`, `docs/API.md`, `server/openapi-extra/lanes.js` (+ regenerate `openapi.yaml`), `server/README.md`, `ARCHITECTURE.md` as applicable.
- The pre-commit hook runs Prettier plus both suites and takes minutes. Let it finish. NEVER `--no-verify` — on the previous branch it caught a real bug that had been dismissed as an environment quirk.
- Baseline at branch point: 857 server tests, 297 client tests, all passing. Each task must leave `git status --short` empty.
---
## Task 1 (B1): the rule matcher
Pure logic, no DB, no HTTP. Everything else depends on its shape.
**Files:** Create `server/lib/stage-detect.js`; create `server/__tests__/stage-detect.test.js`.
**Produces:**
- `flattenInput(toolInput): string` — a searchable string from a tool's input object (concatenate string values one level deep, plus `command`, `file_path`, `skill`, `prompt` if present). Must tolerate `null`, a string, an array, and deeply nested objects without throwing.
- `detect(pipeline, event): {nodeId, signal} | null` — `event` is `{tool_name, tool_input}`. Walks the pipeline's nodes, returns the LAST node whose `detect` rules match (later stage wins when two match, so `git push` beats `Edit`), with `signal` a short human string like `` `npm run test:server` ``. Returns null when nothing matches, when the pipeline has no rules, or when the event has no `tool_name`.
- `compileRules(pipeline)` — internal, but exported for testing: precompiles each rule's regex ONCE per pipeline and skips (never throws on) an invalid pattern from a user-supplied template.
- [ ] **Step 1: write the failing tests.** Cover: `Bash` + `npm run test:server` → `tests`; `Edit` → `implement`; `Skill` + `brainstorming` → `plan`; `Bash` + `git push` → `ship`; a rule with no `match` fires on tool alone; `Read` (mentioned by no rule) → null; a template whose rule holds an invalid regex is skipped and the rest still work; `flattenInput` survives null/string/array/nested; two matching nodes → the later one wins.
- [ ] **Step 2: run, confirm they fail** (`node --test server/__tests__/stage-detect.test.js`) — module missing.
- [ ] **Step 3: implement.** No DB, no `require` of anything but `node:` builtins.
- [ ] **Step 4: run, confirm they pass.**
- [ ] **Step 5:** header audit, `npm run test:server`, commit — `feat(lanes): rule matcher for inferring a stage from a tool event`.
---
## Task 2 (B2): rules in the template, columns in the database
**Files:** Modify `server/data/pipelines/default.json`; modify `server/db.js`; modify `server/lib/lanes.js`; modify `server/__tests__/lanes-lib.test.js`.
**Produces:**
- `detect` arrays on the default template's nodes. Ship exactly these, and no others — every rule must be defensible:
- `plan`: `Skill` matching `brainstorming|writing-plans`; `Write` matching `docs/.*plan.*\.md`
- `implement`: `Edit`; `Write`
- `tests`: `Bash` matching `\b(npm (run )?test|pytest|vitest|jest|go test|cargo test)\b`
- `review`: `Skill` matching `code-review|requesting-code-review`; `Bash` matching `git diff|gh pr diff`
- `ship`: `Bash` matching `git push|gh pr create`
- `intake`, `gate`, `done`: no rules. A gate is a judgement and `done` is a claim; neither may be inferred.
- Columns `detected_stage`, `detected_signal`, `detected_at` on `lanes`, one probe each.
- `recordDetection(id, {nodeId, signal})` in `server/lib/lanes.js` — applies the guard and returns `{written: boolean, reason?: string}`. It writes only when ALL hold: the detection's node index is strictly greater than the current `detected_stage`'s index; and the lane's DECLARED stage index is strictly less than the detection's. Otherwise it returns `written: false` with a reason (`behind-detected`, `behind-declared`, `unknown-node`) and touches nothing.
- `lanePayload` gains `detected_stage`, `detected_signal`, and `detected: boolean` on each entry of `pipeline_nodes` (true for the detected node and for nodes before it that carry no declaration).
- [ ] **Step 1: write the failing tests** in `lanes-lib.test.js`: a forward detection writes; a backward detection returns `behind-detected` and writes nothing; a detection at or behind the declared stage returns `behind-declared`; an unknown node id returns `unknown-node`; **a lane with detections and no declarations has no `done` node in `pipeline_nodes`**; the migration adds all three columns to a database holding an old-schema `lanes` row, and a simulated mid-migration crash self-heals.
- [ ] **Step 2: run, confirm they fail.**
- [ ] **Step 3: implement** — template rules, the three probes, `recordDetection`, the payload fields.
- [ ] **Step 4: run, confirm they pass;** then `npm run test:server`.
- [ ] **Step 5:** header audit, commit — `feat(lanes): detection rules in the template, detected columns on lanes`.
---
## Task 3 (B3): wire it into the hook stream
**Files:** Modify `server/routes/hooks.js`; modify `server/__tests__/lanes-api.test.js`.
**Produces:** no new exports. Inside the EXISTING `touchLaneFromHook` try/catch, after the lane is resolved: build the event from the hook payload (`data.tool_name`, `data.tool_input`), call `detect(getPipeline(lane.pipeline), event)`, and on a hit call `recordDetection`. Broadcast `lane_update` only when `recordDetection` reports `written: true` — Bash alone produced 29 470 events in a real install, so a broadcast per event is not acceptable.
- [ ] **Step 1: write the failing tests:** a `PostToolUse` hook carrying `Bash: npm run test:server` under a lane's cwd sets `detected_stage` to `tests`; a following `Read` event leaves it unchanged; an event for a path under no lane changes nothing; a lane whose declared stage is already `ship` ignores an `implement` detection; a hook whose `data` is malformed (`tool_input` a string, `tool_name` missing) still returns 200 and leaves the lane untouched.
- [ ] **Step 2: run, confirm they fail.**
- [ ] **Step 3: implement.** Nothing may be added outside the existing try/catch. Add one short comment naming the write-on-change rule and why (the event volume).
- [ ] **Step 4: run, confirm they pass;** then `npm run test:server`.
- [ ] **Step 5:** header audit, commit — `feat(lanes): infer a lane's stage from its hook stream`.
---
## Task 4 (B4): show it, and never as done
**Files:** Modify `client/src/lib/types.ts`, `client/src/components/lanes/PipelineMap.tsx`, `client/src/components/lanes/LaneCard.tsx`, `client/src/i18n/locales/{en,zh,vi,ko}/lanes.json`; modify `client/src/components/lanes/__tests__/PipelineMap.test.tsx`.
**Produces:** `LaneNode` gains `detected?: boolean`; `Lane` gains `detected_stage` and `detected_signal`. A node with `detected: true` renders amber with a **dashed** border, visually distinct from both green `done` and solid-amber `passed-no-evidence`. Its `title` names the signal (`tests ← npm run test:server`). `LaneCard` shows an `auto: <stage>` chip only when the detected stage is ahead of the declared one. All strings via i18n in all four locales.
- [ ] **Step 1: write the failing tests:** a detected node carries `data-detected="true"` and a dashed-border class token; its class differs from both the `done` and the plain `passed-no-evidence` node; the tooltip contains the signal; **no node with `detected: true` ever carries `data-state="done"`**.
- [ ] **Step 2: run, confirm they fail** (`cd client && npx vitest run src/components/lanes/__tests__/PipelineMap.test.tsx`).
- [ ] **Step 3: implement.**
- [ ] **Step 4:** `npm run test:client`, `npm run build`. If the screens snapshot moves, read the diff and accept it only if it is exactly the intended change.
- [ ] **Step 5:** header audit, commit — `feat(lanes): render inferred stages as dashed amber, never as done`.
---
## Task 5 (B5): docs, CLI surface, and the honesty pass
**Files:** Modify `docs/LANES.md`, `docs/API.md`, `server/openapi-extra/lanes.js`, `openapi.yaml` (regenerated), `server/README.md`, `bin/ccam.js`, `server/__tests__/lanes-cli.test.js`.
**Produces:** `ccam lanes` gains a column or suffix showing the inferred stage when it leads the declared one. `docs/LANES.md` gains a Stage detection section stating: what signals are read; that rules live in the template and how to add one via `DASHBOARD_PIPELINES_DIR`; that detection is forward-only; that declared beats detected; and — prominently — **that an inferred stage never counts as evidence and never renders as done**, with the reason. `docs/API.md` and the OpenAPI fragment document the three new payload fields.
- [ ] **Step 1: write the failing CLI test:** `ccam lanes` prints the inferred stage for a lane whose detection leads its declaration, and does not print one when the declaration leads.
- [ ] **Step 2: run, confirm it fails.**
- [ ] **Step 3: implement the CLI change and write the docs.** Every rule you document must match `server/data/pipelines/default.json` exactly — read the file, do not recall it.
- [ ] **Step 4:** `npm run test:server`, `npm run test:client`, `node scripts/generate-openapi-yaml.js` then confirm `git diff openapi.yaml` is empty.
- [ ] **Step 5:** header audit, commit — `docs(lanes): document stage detection and its evidence boundary`.
---
## Out of scope
- Inferring `done` or any gate outcome.
- Back-filling detections for existing lanes.
- Reading `workflows.phases` as a signal — real, but it needs its own reconciliation story with the declared stage.
- Any write to `lanes.stage` from inference.
docs/superpowers/plans/2026-07-28-workspace-page.md
+119
View File
@@ -0,0 +1,119 @@
# Merged Workspace Page Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Merge the Lanes page and the Run page into one Workspace page at `/run` — lane strip, pipeline map, and a full Claude console for the selected lane — without losing any Run capability.
**Architecture:** Design doc: `docs/superpowers/specs/2026-07-28-workspace-page-design.md` — read it once before Task 1. `client/src/pages/Run.tsx` (3658 lines) is extracted into a hook and three components in three separate mechanical commits, each leaving the existing tests green with only import changes. Only then is the new page composed. Four small server pieces support it: runs start through the lane, an `ensure` endpoint, a `lane_id` on run history, and releasing the lane when a run ends.
**Tech Stack:** React 18 + TypeScript + Vite + Tailwind, Vitest + Testing Library (client); Node 18+, Express, better-sqlite3, `node:test` (server).
## Global Constraints
- Branch: `feat/workspace-page`, cut from the head of `feat/stage-detection` (or of `feat/worktree-lanes` if B has not landed). Never work on `master`.
- Every `.js/.ts/.tsx` file created or modified MUST start with a file overview comment plus the exact line `@author Nguyễn Ngọc Trí Vĩ <vinnt@smartgift.vn>`. Verify with `bash .claude/skills/file-headers/scripts/check-headers.sh` (must exit 0).
- **Tasks A1-A3 are pure moves.** No behaviour may change, no logic may be "improved" in passing. The existing Run tests must pass with changes to import paths ONLY. If a test needs a real edit to keep passing, stop and report it — that means the move was not pure.
- **Extraction and composition never share a commit.** A1, A2, A3 are refactors; A5 builds the page.
- **The console never writes a lane's stage.** No code path from the console may call `POST /:id/stage`. Only `ccam stage` declares; only detection infers.
- Schema changes are additive with a per-column probe (`try { SELECT col } catch { ALTER }`).
- Preserve existing behavior: `POST /api/run` and every existing WebSocket message type keep working exactly as they do — the CLI and other callers depend on them. `lane_update` stays the only lane WS type.
- Server CommonJS; client React + TypeScript. No new npm dependencies. Exact-value assertions; bounded polling, no bare sleeps. i18n strings in all four locales (`en`, `zh`, `vi`, `ko`), genuinely translated.
- The screens snapshot (`client/src/pages/__tests__/screens.snapshot.test.tsx`) covers `/run`. Read every snapshot diff before accepting it; never regenerate blindly.
- The pre-commit hook runs Prettier plus both suites and takes minutes. Let it finish. NEVER `--no-verify`.
- Baseline at branch point: 857 server tests, 297 client tests (add B's counts if B landed first). Each task leaves `git status --short` empty.
---
## Task 1 (A1): extract `useRunStream`
**Files:** Create `client/src/hooks/useRunStream.ts`; create `client/src/hooks/__tests__/useRunStream.test.tsx`; modify `client/src/pages/Run.tsx`.
**Produces:** `useRunStream(runId: string | null)` returning `{envelopes, status, lastAck}`. It owns everything `Run.tsx` currently does with `run_stream` / `run_status` / `run_input_ack`: the envelope merge (`mergeEnvelope`, `findLastStreamingAssistant`, `findAssistantByMessageId`, `mutateAssistantAt`), the typewriter (`useTypewriterEnvelopes`), and the `eventBus.subscribe` lifecycle. Move those functions; do not rewrite them.
- [ ] **Step 1: write the hook's tests first** — they are new coverage for code that had none: envelopes for the subscribed run id merge in arrival order; an envelope for a different run id is ignored; a streaming assistant envelope updates in place rather than appending; a terminal `run_status` stops further merging; unmounting disposes the subscription (assert the disposer returned by `eventBus.subscribe` was called).
- [ ] **Step 2: run, confirm they fail** — `cd client && npx vitest run src/hooks/__tests__/useRunStream.test.tsx`.
- [ ] **Step 3: move the code.** Cut the named functions out of `Run.tsx` into the hook, export them if the tests need them, and have `Run.tsx` call the hook. Delete the now-dead copies. Change nothing else.
- [ ] **Step 4: verify the move was pure** — `npm run test:client` (all pre-existing Run tests green, no test bodies edited), `npm run build`, and `git diff client/src/pages/__tests__/` must show no snapshot change.
- [ ] **Step 5:** header audit, commit — `refactor(run): extract useRunStream from the Run page`.
---
## Task 2 (A2): extract `RunConsole`
**Files:** Create `client/src/components/run/RunConsole.tsx`; modify `client/src/pages/Run.tsx`.
**Produces:** `<RunConsole runId prompt onPromptChange onSubmit onStop slashCommands busy />` rendering the envelope stream, the prompt editor with its slash autocomplete (`PromptEditor`, `detectAutocomplete`, `scoreSlashMatch`, `subsequenceMatch`, `commandSourceLabel`, `commandSourceTone`), and the token meter (`TokenMeter`, `computeTokens`, `formatNum`). It consumes `useRunStream` from A1. Props only — no direct API calls, so the same console can serve the Run page and the Workspace page.
- [ ] **Step 1: write the failing test** — `client/src/components/run/__tests__/RunConsole.test.tsx`: given envelopes from a mocked `useRunStream`, the assistant text renders; typing `/co` shows the matching slash command and picking one fills the prompt; the token meter shows the computed totals; `onSubmit` fires with the prompt text; `onStop` fires from the stop control.
- [ ] **Step 2: run, confirm it fails.**
- [ ] **Step 3: move the code.** Pure move plus the props boundary. Do not redesign the editor.
- [ ] **Step 4:** `npm run test:client` (pre-existing Run tests green with only import changes), `npm run build`, snapshot unchanged.
- [ ] **Step 5:** header audit, commit — `refactor(run): extract RunConsole from the Run page`.
---
## Task 3 (A3): extract `RunSetup` and `RunHistory`, leave `Run.tsx` thin
**Files:** Create `client/src/components/run/RunSetup.tsx`, `client/src/components/run/RunHistory.tsx`; modify `client/src/pages/Run.tsx`.
**Produces:** `<RunSetup>` owning mode / model / permission-mode / effort / cwd / resume-session pickers, the binary-status check and `LimitationsBanner`; `<RunHistory>` owning past runs, live runs and attach. After this task `Run.tsx` holds only page-level state and composition — report its final line count in your report.
- [ ] **Step 1: write the failing tests** for both components: `RunSetup` reports each selection through its callbacks and surfaces a missing-binary state; `RunHistory` lists history, marks a live run, and fires attach with the right run id.
- [ ] **Step 2: run, confirm they fail.**
- [ ] **Step 3: move the code.**
- [ ] **Step 4:** `npm run test:client`, `npm run build`, snapshot unchanged.
- [ ] **Step 5:** header audit, commit — `refactor(run): extract RunSetup and RunHistory, thin the Run page`.
---
## Task 4 (A4): the server glue
**Files:** Modify `server/db.js`, `server/lib/lanes.js`, `server/routes/lanes.js`, `server/lib/run-spawner.js`, `server/__tests__/lane-lifecycle.test.js`; docs as listed in the constraints.
**Produces:**
- `POST /api/lanes/ensure` `{cwd, title?}` → `{lane, created: boolean}`. Returns the lane that owns `cwd` (exact match or the longest path-boundary parent, reusing `resolveLaneByCwd`), else creates an `adopted` lane. Behind the same-origin guard. Concurrent calls for the same path must yield ONE lane — rely on the `cwd` UNIQUE constraint and treat the constraint violation as "someone else created it, re-read and return it".
- `mode` accepted by the lane `start` action and passed through to `spawnRun`, so a headless one-shot is reachable through a lane.
- `dashboard_runs.lane_id`, one additive probe, written when a run starts through a lane; `GET /api/run/history` accepts an optional `laneId` filter.
- **A finished run releases its lane.** When the run-spawner observes a child's real exit, clear `run_id` and set `status: "idle"` on the lane holding that `run_id`, and broadcast `lane_update`. Do this without creating a require cycle (`run-spawner` must not import a route module — read how `broadcastLane` is exported and pick the clean direction, or invert it with a callback registered at boot). State in your report which direction you chose and why.
- [ ] **Step 1: write the failing tests:** `ensure` returns the existing lane for an exact path, for a nested path, and creates one otherwise; two concurrent `ensure` calls for the same path create exactly one lane; a lane-started run records `lane_id` in `dashboard_runs` and `GET /api/run/history?laneId=` filters by it; **when a run ends on its own, the lane's `run_id` becomes null and its status returns to `idle`**; a cross-origin `POST /api/lanes/ensure` is refused.
- [ ] **Step 2: run, confirm they fail.**
- [ ] **Step 3: implement.**
- [ ] **Step 4:** `npm run test:server`; regenerate `openapi.yaml` and confirm `git diff openapi.yaml` is empty.
- [ ] **Step 5:** header audit, commit — `feat(lanes): ensure endpoint, run history per lane, release the lane when a run ends`.
---
## Task 5 (A5): compose the Workspace page
**Files:** Create `client/src/pages/Workspace.tsx`; modify `client/src/App.tsx`, `client/src/components/Sidebar.tsx`, `client/src/lib/api.ts`, `client/src/i18n/locales/*/lanes.json`; modify `client/src/pages/__tests__/screens.snapshot.test.tsx`.
**Produces:** the merged page at `/run`; `/lanes` redirects to it (`<Navigate to="/run" replace />`); one sidebar entry. Layout top to bottom: lane strip (horizontal scroll, counters, Add) → `PipelineMap` for the selected lane → `RunSetup` → `RunConsole` → `RunHistory` filtered to the lane. Selecting a lane switches pipeline, console and history together. Starting a run goes through `POST /api/lanes/:id/start`; choosing a cwd that no lane owns calls `POST /api/lanes/ensure` first. `api.lanes` gains `ensure`.
- [ ] **Step 1: write the failing tests** — `client/src/pages/__tests__/Workspace.test.tsx`: the strip lists lanes and the counters match; selecting a lane switches the pipeline and the console's run id; starting a run posts to the LANE start endpoint (assert the URL, not just that something was called); picking an unowned cwd calls `ensure` before `start`; **after a full start-and-message cycle the lane's stage is never posted to** (assert no call to any `/stage` URL).
- [ ] **Step 2: run, confirm they fail.**
- [ ] **Step 3: implement.** Keep `Run.tsx`'s remaining shell only if something still needs it; if the Workspace page fully replaces it, delete it and say so.
- [ ] **Step 4:** `npm run test:client`, `npm run build`. The screens snapshot WILL change here — read the diff, confirm it is only the merged layout, then regenerate.
- [ ] **Step 5:** header audit, commit — `feat(lanes): merge the Lanes and Run pages into one Workspace`.
---
## Task 6 (A6): docs and the seams
**Files:** Modify `docs/LANES.md`, `docs/API.md`, `server/README.md`, `ARCHITECTURE.md`, `README.md`, `server/openapi-extra/lanes.js` (+ regenerate `openapi.yaml`), `CLAUDE.md`.
**Produces:** documentation of the merged page and the new seams: that `/lanes` redirects to `/run`; that the UI starts runs through the lane while `POST /api/run` remains for the CLI; the `ensure` endpoint and when the UI calls it; `dashboard_runs.lane_id`; that a finished run releases its lane. `CLAUDE.md` gains the rule: **the console never writes a lane's stage — declared comes from `ccam stage`, inferred from detection.** Every path and command you print must exist; verify each.
- [ ] **Step 1:** write the docs.
- [ ] **Step 2:** verify every referenced file, route and command exists (`ls`, `grep`, or run it).
- [ ] **Step 3:** `npm run test:server`, `npm run test:client`, `node scripts/generate-openapi-yaml.js` then `git diff openapi.yaml` empty.
- [ ] **Step 4:** header audit, commit — `docs(lanes): document the merged Workspace page and its seams`.
---
## Out of scope
- Any redesign of the prompt editor, the envelope renderer, or the token meter. A1-A3 move them unchanged.
- Multiple concurrent runs per lane. `start` already 409s when one is live.
- Per-lane dependency bootstrap for a fresh worktree (still sub-project D if ever wanted).
- Stage inference — that is sub-project B, and the console must not do it either way.
docs/superpowers/plans/2026-07-28-worktree-lanes.md
+580
View File
@@ -0,0 +1,580 @@
# Worktree-backed Lanes Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Let a lane own a git worktree that CCAM creates, resets, removes and purges, with every destructive action gated on counted facts and on three independent safety checks.
**Architecture:** Design doc: `docs/superpowers/specs/2026-07-28-worktree-lanes-design.md` — read it once before Task 1. All git work goes through one module (`server/lib/worktree.js`) that shells out with `execFile` and an argv array, never a shell string, and re-verifies its own safety preconditions. Lanes gain a `kind` of `adopted` (pointer at a directory the user already had — never destroyable) or `managed` (a worktree CCAM created — destroyable). Destructive actions are serialised per lane and preceded by a preflight endpoint that returns counts, which the confirmation UI renders and the server re-checks before acting.
**Tech Stack:** Node 18+, Express, better-sqlite3, `node:child_process.execFile`, real `git` against temp-directory fixtures, `node:test` (server), React 18 + TypeScript + Vitest (client).
## Global Constraints
- Branch: create `feat/worktree-lanes` off the current head of `feat/lanes-pipeline`. Never work on `master`.
- Every `.js/.ts/.tsx` file created or modified MUST start with a file overview comment plus the exact line `@author Nguyễn Ngọc Trí Vĩ <vinnt@smartgift.vn>`. Verify with `bash .claude/skills/file-headers/scripts/check-headers.sh` (must exit 0).
- Schema changes are additive only and migration-safe on an existing database: `try { SELECT col } catch { ALTER TABLE … ADD COLUMN }`, the pattern at `server/db.js:412-418`. Existing rows must migrate to `kind='adopted'`.
- **Never `rm -rf` a lane directory.** Removal goes through `git worktree remove`; if git refuses, surface git's error unchanged.
- **Never build a shell command string.** `execFile("git", [...args])` only. No `shell: true`, no template-literal commands.
- Destructive routes stay behind the existing same-origin guard exported from `server/routes/run.js`.
- Preserve existing behavior: no existing route, response shape, WebSocket type, or CLI command changes meaning. `lane_update` stays the only lane WS type.
- Server is CommonJS. No new npm dependencies. Server tests use `node:test` + `node:assert/strict`; client tests use Vitest + Testing Library.
- The pre-commit hook runs Prettier and the full server suite; a commit takes minutes. Do not disable it.
- Baseline before this plan: 790 server tests, 279 client tests, all passing.
---
## File Structure
**Create**
- `server/lib/worktree.js` — every git invocation, plus the three-check safety guard. No Express, no DB.
- `server/lib/lane-preflight.js` — counts for `reset` / `remove` / `purge`. Reads git and the DB; mutates nothing.
- `server/lib/lane-lock.js` — per-lane async mutex.
- `server/__tests__/worktree.test.js` — git behaviour against a real temp repo.
- `server/__tests__/lane-lifecycle.test.js` — HTTP: add / preflight / reset / remove / purge.
- `client/src/components/lanes/DestructiveLaneModal.tsx` — preflight table inside the existing `ConfirmModal`.
**Modify**
- `server/db.js` — four additive columns.
- `server/lib/lanes.js` — `kind`/`source_repo`/`base_branch`/`slug` in create/patch/payload; `purgeLaneSessions`.
- `server/routes/lanes.js` — `POST /worktree`, `GET /:id/preflight`, `reset` + `purge` actions, lock usage.
- `bin/ccam.js` — `ccam lanes add --repo`, `ccam lanes reset|remove|purge`.
- `client/src/lib/api.ts`, `client/src/lib/types.ts` — preflight + worktree types and calls.
- `client/src/components/lanes/LaneCard.tsx` — kind badge; destructive buttons only for `managed`.
- `docs/LANES.md`, `CLAUDE.md` — the new lifecycle.
---
## Task 1: `server/lib/worktree.js` — git plumbing and the safety guard
**Files:**
- Create: `server/lib/worktree.js`
- Test: `server/__tests__/worktree.test.js`
**Interfaces:**
- Consumes: nothing from earlier tasks.
- Produces:
- `LANES_ROOT` — `process.env.LANES_ROOT || path.join(os.homedir(), ".claude", "ccam-lanes")`
- `git(cwd, args): Promise<{stdout, stderr}>` — rejects with `err.git = {args, code, stderr}` on non-zero
- `isGitRepo(dir): Promise<boolean>`
- `resolveBase(sourceRepo, wanted): Promise<string>` — `origin/<wanted>` → `<wanted>` → current HEAD
- `slugify(text): string` — lowercase, non-alphanumerics to `-`, collapsed, trimmed, max 40 chars
- `listWorktrees(sourceRepo): Promise<Array<{path, branch, locked}>>` — parses `--porcelain`
- `branchCheckedOutAt(sourceRepo, branch): Promise<string|null>`
- `addWorktree({sourceRepo, dir, branch, base}): Promise<{dir, branch, created: boolean}>`
- `assertDestroyable(lane): Promise<void>` — the three checks; throws `err.code = "ENOTMANAGED" | "EOUTSIDEROOT" | "ENOTWORKTREE"`
- `resetWorktree(lane): Promise<void>`
- `removeWorktree(lane): Promise<void>`
- `statusCounts(dir): Promise<{dirty, untracked, head}>`
- `unpushedCount(dir): Promise<number>`
- [ ] **Step 1: Write the failing test**
Create `server/__tests__/worktree.test.js`. It builds a real repository in a temp directory — mocks would test nothing that matters here.
```js
/**
* @file Tests for server/lib/worktree.js against a REAL git repository created
* in a temp directory. Every behaviour worth testing here is git's own — branch
* collisions, what `clean -fd` spares, what `worktree list` reports — so mocking
* git would only test our idea of git.
* @author Nguyễn Ngọc Trí Vĩ <vinnt@smartgift.vn>
*/
const { describe, it, before, after } = require("node:test");
const assert = require("node:assert/strict");
const fs = require("node:fs");
const os = require("node:os");
const path = require("node:path");
const { execFileSync } = require("node:child_process");
const ROOT = fs.mkdtempSync(path.join(os.tmpdir(), "ccam-wt-"));
process.env.LANES_ROOT = path.join(ROOT, "lanes");
const wt = require("../lib/worktree");
const SRC = path.join(ROOT, "src-repo");
const g = (cwd, ...args) => execFileSync("git", args, { cwd, encoding: "utf8" });
before(() => {
fs.mkdirSync(SRC, { recursive: true });
g(SRC, "init", "-b", "main");
g(SRC, "config", "user.email", "t@example.com");
g(SRC, "config", "user.name", "Test");
fs.writeFileSync(path.join(SRC, "README.md"), "hello\n");
fs.writeFileSync(path.join(SRC, ".gitignore"), "node_modules/\n.env\n");
g(SRC, "add", "-A");
g(SRC, "commit", "-m", "init");
});
after(() => fs.rmSync(ROOT, { recursive: true, force: true }));
function laneFor(dir, branch, over = {}) {
return { id: 1, kind: "managed", cwd: dir, branch, source_repo: SRC, base_branch: "main", ...over };
}
describe("worktree", () => {
it("slugifies a title into a safe single segment", () => {
assert.equal(wt.slugify("Rename Metric → Rule!"), "rename-metric-rule");
assert.equal(wt.slugify(" a//b "), "a-b");
assert.ok(wt.slugify("x".repeat(80)).length <= 40);
});
it("resolves the base branch, falling back when origin has none", async () => {
assert.equal(await wt.resolveBase(SRC, "main"), "main");
assert.equal(await wt.resolveBase(SRC, "does-not-exist"), "main");
});
it("creates a worktree on a new branch and lists it", async () => {
const dir = path.join(process.env.LANES_ROOT, "src-repo__alpha");
const r = await wt.addWorktree({ sourceRepo: SRC, dir, branch: "feat/alpha", base: "main" });
assert.equal(r.created, true);
assert.ok(fs.existsSync(path.join(dir, "README.md")));
const list = await wt.listWorktrees(SRC);
assert.ok(list.some((w) => w.path === dir && w.branch === "feat/alpha"));
});
it("refuses a branch already checked out in another worktree", async () => {
const dir2 = path.join(process.env.LANES_ROOT, "src-repo__alpha2");
await assert.rejects(
() => wt.addWorktree({ sourceRepo: SRC, dir: dir2, branch: "feat/alpha", base: "main" }),
(e) => e.code === "EBRANCHBUSY" && typeof e.checkedOutAt === "string",
);
});
it("counts dirty, untracked and unpushed work", async () => {
const dir = path.join(process.env.LANES_ROOT, "src-repo__alpha");
fs.appendFileSync(path.join(dir, "README.md"), "edit\n");
fs.writeFileSync(path.join(dir, "scratch.txt"), "untracked\n");
fs.mkdirSync(path.join(dir, "node_modules"), { recursive: true });
fs.writeFileSync(path.join(dir, "node_modules", "dep.js"), "x\n");
const s = await wt.statusCounts(dir);
assert.equal(s.dirty, 1);
assert.equal(s.untracked, 1); // node_modules is ignored, so it does not count
assert.match(s.head, /^[0-9a-f]{7,40}$/);
assert.equal(await wt.unpushedCount(dir), 0); // no upstream yet
});
it("reset restores base, drops untracked files, and spares gitignored ones", async () => {
const dir = path.join(process.env.LANES_ROOT, "src-repo__alpha");
await wt.resetWorktree(laneFor(dir, "feat/alpha"));
assert.equal(fs.readFileSync(path.join(dir, "README.md"), "utf8"), "hello\n");
assert.equal(fs.existsSync(path.join(dir, "scratch.txt")), false);
assert.equal(fs.existsSync(path.join(dir, "node_modules", "dep.js")), true);
const s = await wt.statusCounts(dir);
assert.equal(s.dirty, 0);
});
it("refuses to destroy an adopted lane, a path outside LANES_ROOT, or a non-worktree", async () => {
const dir = path.join(process.env.LANES_ROOT, "src-repo__alpha");
await assert.rejects(
() => wt.assertDestroyable(laneFor(dir, "feat/alpha", { kind: "adopted" })),
(e) => e.code === "ENOTMANAGED",
);
await assert.rejects(
() => wt.assertDestroyable(laneFor("/tmp", "feat/alpha")),
(e) => e.code === "EOUTSIDEROOT",
);
const ghost = path.join(process.env.LANES_ROOT, "src-repo__ghost");
fs.mkdirSync(ghost, { recursive: true });
await assert.rejects(
() => wt.assertDestroyable(laneFor(ghost, "feat/ghost")),
(e) => e.code === "ENOTWORKTREE",
);
});
it("removes the worktree and its branch, leaving git's list clean", async () => {
const dir = path.join(process.env.LANES_ROOT, "src-repo__alpha");
await wt.removeWorktree(laneFor(dir, "feat/alpha"));
assert.equal(fs.existsSync(dir), false);
const list = await wt.listWorktrees(SRC);
assert.equal(list.some((w) => w.path === dir), false);
const branches = g(SRC, "branch", "--list", "feat/alpha").trim();
assert.equal(branches, "");
});
it("never deletes the base branch even if a lane claims it", async () => {
const dir = path.join(process.env.LANES_ROOT, "src-repo__beta");
await wt.addWorktree({ sourceRepo: SRC, dir, branch: "feat/beta", base: "main" });
await wt.removeWorktree(laneFor(dir, "main")); // lane lies about its branch
assert.match(g(SRC, "branch", "--list", "main"), /main/);
});
});
```
- [ ] **Step 2: Run test to verify it fails**
Run: `node --test server/__tests__/worktree.test.js`
Expected: FAIL — `Cannot find module '../lib/worktree'`.
- [ ] **Step 3: Implement the module**
Create `server/lib/worktree.js`. Key requirements the tests pin, restated so nothing is inferred:
- `git(cwd, args)` wraps `execFile("git", args, {cwd, maxBuffer: 8 * 1024 * 1024})` promisified. On failure throw an `Error` carrying `err.git = { args, code, stderr }`.
- `slugify` — `text.toLowerCase().normalize("NFKD").replace(/[^a-z0-9]+/g, "-").replace(/^-+|-+$/g, "").slice(0, 40)`, and if the result is empty throw `err.code = "EBADSLUG"`.
- `resolveBase(sourceRepo, wanted)` — try `git rev-parse --verify --quiet origin/<wanted>`, then `<wanted>`, then `git rev-parse --abbrev-ref HEAD`. Return the first that resolves.
- `listWorktrees` — parse `git worktree list --porcelain`: records separated by blank lines, `worktree <path>`, `branch refs/heads/<name>`, bare `locked` line. Return `{path, branch, locked}` with `branch` null for a detached worktree.
- `branchCheckedOutAt(sourceRepo, branch)` — the `path` from `listWorktrees` whose branch matches, else null.
- `addWorktree({sourceRepo, dir, branch, base})`:
- if `branchCheckedOutAt` returns a path, throw `err.code = "EBRANCHBUSY"`, `err.checkedOutAt = thatPath`
- `fs.mkdirSync(path.dirname(dir), {recursive: true})`
- if `git rev-parse --verify --quiet <branch>` succeeds, run `worktree add <dir> <branch>` and return `{created: false}`; otherwise `worktree add -b <branch> <dir> <base>` and return `{created: true}`
- `assertDestroyable(lane)` — in order: `kind !== "managed"` → `ENOTMANAGED`; `fs.realpathSync(lane.cwd)` not inside `fs.realpathSync(LANES_ROOT)` on a path boundary → `EOUTSIDEROOT` (a non-existent path fails this check too, which is correct — it cannot be a live worktree); not present in `listWorktrees(lane.source_repo)` → `ENOTWORKTREE`.
- `PROTECTED_BRANCHES = new Set(["main", "master"])`, plus the lane's own `base_branch`: `deleteBranchSafely(sourceRepo, branch, baseBranch)` returns without acting when the branch is protected or falsy.
- `resetWorktree(lane)` — `assertDestroyable` first, then, all in `lane.cwd`: `fetch origin --prune` (tolerate failure when there is no remote), `checkout <base>` (creating it from `origin/<base>` if absent), `reset --hard <base>`, `clean -fd` (**never** `-x`), then in `source_repo` `deleteBranchSafely(lane.branch)`, then back in the worktree `checkout -b <lane.branch> <base>`.
- `removeWorktree(lane)` — `assertDestroyable`, then in `source_repo`: `worktree unlock <dir>` (ignore failure), `worktree remove --force <dir>`, `worktree prune`, `deleteBranchSafely(lane.branch, lane.base_branch)`. If `worktree remove` fails, rethrow git's error untouched — do not fall back to filesystem deletion.
- `statusCounts(dir)` — parse `git status --porcelain=v1 --untracked-files=normal`: lines starting `??` are untracked, others dirty. `head` from `git rev-parse --short HEAD`.
- `unpushedCount(dir)` — `git rev-list --count @{u}..HEAD`; when there is no upstream, git exits non-zero — return 0.
- [ ] **Step 4: Run test to verify it passes**
Run: `node --test server/__tests__/worktree.test.js`
Expected: PASS, 8 tests.
- [ ] **Step 5: Header audit and commit**
Run: `bash .claude/skills/file-headers/scripts/check-headers.sh`
```bash
git add server/lib/worktree.js server/__tests__/worktree.test.js
git commit -m "feat(lanes): git worktree plumbing with a three-check destroy guard"
```
---
## Task 2: Schema, lane fields, and per-lane locking
**Files:**
- Modify: `server/db.js` (the `lanes` block)
- Modify: `server/lib/lanes.js`
- Create: `server/lib/lane-lock.js`
- Test: `server/__tests__/lanes-lib.test.js` (append a `describe`)
**Interfaces:**
- Consumes: nothing from Task 1 (kept independent so both can be reviewed alone).
- Produces:
- four columns on `lanes`: `kind` (`NOT NULL DEFAULT 'adopted'`), `source_repo`, `base_branch`, `slug`
- `createLane` accepts and stores `kind`, `source_repo`, `base_branch`, `slug`; unknown values of `kind` are rejected with `err.code = "EBADKIND"`
- `PATCHABLE` gains `kind`, `source_repo`, `base_branch`, `slug`
- `purgeLaneSessions(id): {sessions, events, tokenRows}` — deletes the lane's sessions (never the one in `lanes.session_id`), their events, and `token_usage` rows left orphaned
- `server/lib/lane-lock.js`: `withLaneLock(id, fn): Promise<any>` — serialises per lane id, releases on throw
- [ ] **Step 1: Write the failing test** (append to `server/__tests__/lanes-lib.test.js`)
```js
const { withLaneLock } = require("../lib/lane-lock");
describe("lane kind, worktree fields and purge", () => {
it("defaults to adopted and stores worktree fields when given", () => {
const a = lanes.createLane({ cwd: "/tmp/wt-kind-a" });
assert.equal(a.kind, "adopted");
const m = lanes.createLane({
cwd: "/tmp/wt-kind-b", kind: "managed",
source_repo: "/tmp/src", base_branch: "main", slug: "b",
});
assert.equal(m.kind, "managed");
assert.equal(m.source_repo, "/tmp/src");
assert.equal(m.base_branch, "main");
assert.equal(m.slug, "b");
lanes.deleteLane(a.id);
lanes.deleteLane(m.id);
});
it("rejects an unknown kind", () => {
assert.throws(() => lanes.createLane({ cwd: "/tmp/wt-kind-c", kind: "gremlin" }),
(e) => e.code === "EBADKIND");
});
it("purges a lane's sessions, their events and orphaned token rows, sparing the live one", () => {
const l = lanes.createLane({ cwd: "/tmp/wt-purge" });
const { db } = require("../db");
db.prepare("INSERT INTO sessions (id, cwd, status) VALUES (?, ?, 'completed')").run("purge-1", "/tmp/wt-purge/sub");
db.prepare("INSERT INTO sessions (id, cwd, status) VALUES (?, ?, 'active')").run("purge-live", "/tmp/wt-purge");
db.prepare("INSERT INTO events (session_id, event_type) VALUES (?, 'PostToolUse')").run("purge-1");
db.prepare("INSERT INTO token_usage (session_id, model, input_tokens) VALUES (?, 'm', 5)").run("purge-1");
lanes.updateLane(l.id, { session_id: "purge-live" });
const counts = lanes.purgeLaneSessions(l.id);
assert.equal(counts.sessions, 1);
assert.equal(counts.events, 1);
assert.equal(counts.tokenRows, 1);
assert.equal(db.prepare("SELECT COUNT(*) c FROM sessions WHERE id='purge-live'").get().c, 1);
assert.equal(db.prepare("SELECT COUNT(*) c FROM events WHERE session_id='purge-1'").get().c, 0);
assert.equal(db.prepare("SELECT COUNT(*) c FROM token_usage WHERE session_id='purge-1'").get().c, 0);
lanes.deleteLane(l.id);
});
it("serialises work per lane and releases the lock when the body throws", async () => {
const order = [];
const slow = withLaneLock(7, async () => { order.push("a-start"); await new Promise((r) => setTimeout(r, 50)); order.push("a-end"); });
const fast = withLaneLock(7, async () => { order.push("b"); });
await Promise.all([slow, fast]);
assert.deepEqual(order, ["a-start", "a-end", "b"]);
await assert.rejects(() => withLaneLock(7, async () => { throw new Error("boom"); }));
await withLaneLock(7, async () => order.push("c"));
assert.equal(order[order.length - 1], "c");
});
});
```
- [ ] **Step 2: Run test to verify it fails**
Run: `node --test server/__tests__/lanes-lib.test.js`
Expected: FAIL — `Cannot find module '../lib/lane-lock'`.
- [ ] **Step 3: Add the columns**
In `server/db.js`, after the `lanes` table and its index, following the probe pattern at `server/db.js:412-418`:
```js
// Managed lanes own a git worktree CCAM created and may be destroyed; adopted
// lanes merely point at a directory the user already had and never may be.
// Existing rows default to 'adopted', so no lane gains a destructive path by
// upgrading.
try {
db.prepare("SELECT kind FROM lanes LIMIT 1").get();
} catch {
db.prepare("ALTER TABLE lanes ADD COLUMN kind TEXT NOT NULL DEFAULT 'adopted'").run();
db.prepare("ALTER TABLE lanes ADD COLUMN source_repo TEXT").run();
db.prepare("ALTER TABLE lanes ADD COLUMN base_branch TEXT").run();
db.prepare("ALTER TABLE lanes ADD COLUMN slug TEXT").run();
}
```
- [ ] **Step 4: Extend `server/lib/lanes.js` and write the lock**
`createLane` gains the four fields (validating `kind` against `new Set(["adopted", "managed"])`), `PATCHABLE` gains them, and `purgeLaneSessions(id)` runs inside one `db.transaction`:
- select the lane's sessions: `WHERE (cwd = ? OR cwd LIKE ? || '/%')` against `lane.cwd`, excluding `lanes.session_id` and any session whose `status = 'active'`
- count and delete their `events`, then their `token_usage`, then the sessions themselves
- return `{sessions, events, tokenRows}`
- run `db.pragma("optimize")` after the transaction commits — never `VACUUM`, which locks the whole database
Create `server/lib/lane-lock.js` — a `Map<laneId, Promise>` chain:
```js
const chains = new Map();
function withLaneLock(id, fn) {
const key = String(id);
const prev = chains.get(key) || Promise.resolve();
const run = prev.then(fn, fn); // run regardless of how the previous holder settled
// Keep the chain alive but never let a rejection poison the next waiter.
chains.set(key, run.then(() => {}, () => {}));
return run;
}
```
- [ ] **Step 5: Run tests to verify they pass**
Run: `node --test server/__tests__/lanes-lib.test.js`
Expected: PASS — the four new tests plus every earlier one.
- [ ] **Step 6: Full suite and commit**
Run: `npm run test:server`
```bash
git add server/db.js server/lib/lanes.js server/lib/lane-lock.js server/__tests__/lanes-lib.test.js
git commit -m "feat(lanes): managed/adopted kinds, worktree fields, session purge, per-lane lock"
```
---
## Task 3: Preflight — counted facts before anything destructive
**Files:**
- Create: `server/lib/lane-preflight.js`
- Test: `server/__tests__/lane-lifecycle.test.js` (new file; later tasks append to it)
**Interfaces:**
- Consumes: `statusCounts`, `unpushedCount`, `listWorktrees` (Task 1); `getLane` (Task 2).
- Produces: `preflight(lane, action): Promise<object>` where `action ∈ "reset" | "remove" | "purge"`.
- `reset` / `remove` → `{action, lane, kind, branch, dirty, untracked, unpushed, head, blocked: string[], warnings: string[]}`
- `purge` → `{action, lane, sessions, events, tokenRows, bytesEstimate, activeSessionSkipped: boolean}`
- `blocked` contains `"adopted"` when the lane is not managed, `"missing"` when the directory is gone, and `"unpushed-commits"` when `unpushed > 0`. It is advisory data, not an exception — the route decides.
- [ ] **Step 1: Write the failing test**
Create `server/__tests__/lane-lifecycle.test.js` with the standard harness (temp `DASHBOARD_DB_PATH`, `DASHBOARD_REMOTE_SYNC_MS=0`, `DASHBOARD_LIVENESS_PROBE=0`, `LANES_ROOT` pointed at a temp dir, `startServer(createApp(), 0)`; copy the request helper from `server/__tests__/lanes-api.test.js`), plus a real git fixture repo as in Task 1. Tests:
```js
it("preflight on an adopted lane blocks and counts nothing", async () => { /* create adopted lane, GET preflight?action=reset, expect blocked includes "adopted" */ });
it("preflight counts dirty, untracked and unpushed for a managed lane", async () => { /* dirty the worktree, expect dirty:1 untracked:1 and a head sha */ });
it("preflight for purge counts only this lane's non-live sessions", async () => { /* two sessions, one bound live, expect sessions:1 and activeSessionSkipped:true */ });
it("preflight 404s for an unknown lane and 400s for an unknown action", async () => {});
```
- [ ] **Step 2: Run to verify it fails**
Run: `node --test server/__tests__/lane-lifecycle.test.js`
Expected: FAIL — the route does not exist yet (404 with an HTML body).
- [ ] **Step 3: Implement `server/lib/lane-preflight.js` and the route**
The module is read-only. `bytesEstimate` is `(events + tokenRows) * 512` — label it in `docs/LANES.md` as a rough estimate, because a real per-row size needs `dbstat`, which is not compiled in by default.
In `server/routes/lanes.js`, add **before** the `/:id/:action` route so it is not swallowed:
```js
router.get("/:id/preflight", async (req, res) => {
const lane = lanesLib.getLane(req.params.id);
if (!lane) return res.status(404).json({ error: { code: "ENOLANE", message: "lane not found" } });
const action = String(req.query.action || "");
if (!["reset", "remove", "purge"].includes(action)) {
return res.status(400).json({ error: { code: "EBADACTION", message: `unknown action ${action}` } });
}
try {
res.json(await preflight(lane, action));
} catch (err) {
res.status(500).json({ error: { code: err.code, message: err.message } });
}
});
```
- [ ] **Step 4: Run to verify it passes**
Run: `node --test server/__tests__/lane-lifecycle.test.js`
Expected: PASS, 4 tests.
- [ ] **Step 5: Commit**
```bash
git add server/lib/lane-preflight.js server/routes/lanes.js server/__tests__/lane-lifecycle.test.js
git commit -m "feat(lanes): preflight counts for reset, remove and purge"
```
---
## Task 4: `add` — provision a worktree in the background
**Files:**
- Modify: `server/routes/lanes.js`
- Test: `server/__tests__/lane-lifecycle.test.js` (append)
**Interfaces:**
- Consumes: `addWorktree`, `resolveBase`, `slugify`, `LANES_ROOT` (Task 1); `createLane`, `updateLane` (Task 2); `broadcastLane`, `sameOriginGuard` (existing).
- Produces: `POST /api/lanes/worktree` with body `{sourceRepo, title, base?, slug?}` → `202 {lane}` with `status: "provisioning"`, then a background `lane_update` when the worktree is ready or `status: "failed"` with the git error in `notes`.
- [ ] **Step 1: Write the failing tests** (append)
```js
it("creates a managed lane, returns 202 provisioning, then flips to idle when the worktree lands", async () => {});
it("rejects a sourceRepo that is not an absolute path or not a git repo", async () => {});
it("suffixes the slug when the directory already exists", async () => {});
it("marks the lane failed with git's message when provisioning fails", async () => {});
```
Poll `GET /api/lanes/:id` until `status !== "provisioning"` with a bounded deadline (2 s, 50 ms interval) — never a bare sleep.
- [ ] **Step 2: Run to verify they fail**
Run: `node --test server/__tests__/lane-lifecycle.test.js`
Expected: FAIL — `POST /api/lanes/worktree` 404s.
- [ ] **Step 3: Implement**
Registered before `/:id/:action`, behind `sameOriginGuard`. Validate: `sourceRepo` absolute, exists, `isGitRepo`. Compute `slug = slugify(req.body.slug || req.body.title)`, `dir = path.join(LANES_ROOT, `${path.basename(sourceRepo)}__${slug}`)`, suffixing `-2`, `-3`… while the directory exists. Create the lane row `kind: "managed", status: "provisioning"`, respond `202`, then in the background — wrapped in `withLaneLock(lane.id, …)` — resolve the base, `addWorktree`, and `updateLane` to `status: "idle"` (or `"failed"` with `notes` set to `err.git?.stderr || err.message`), broadcasting either way.
Provisioning must never leave a half-state: if `addWorktree` throws, the lane row stays with `kind: "managed"` and `status: "failed"` so the user can `remove` it, and no directory is left behind that git does not know about.
- [ ] **Step 4: Run to verify they pass** — Expected: PASS, 8 tests total in the file.
- [ ] **Step 5: Commit**
```bash
git add server/routes/lanes.js server/__tests__/lane-lifecycle.test.js
git commit -m "feat(lanes): provision a git worktree for a managed lane"
```
---
## Task 5: `reset`, `remove`, `purge` actions
**Files:**
- Modify: `server/routes/lanes.js`
- Test: `server/__tests__/lane-lifecycle.test.js` (append)
**Interfaces:**
- Consumes: everything from Tasks 1-4.
- Produces: `reset` and `purge` join the `ACTIONS` set; `remove` gains worktree teardown. All three require `{confirm: true}`; `reset` and `remove` additionally require `{force: true}` when preflight reports `unpushed > 0`, and accept `{expect: {head, dirty, untracked, unpushed}}` — a mismatch returns `409 ESTALE`.
- [ ] **Step 1: Write the failing tests** (append)
```js
it("reset requires confirm, restores the branch from base and clears lane state", async () => {});
it("reset refuses with 409 when the worktree has unpushed commits, and proceeds with force", async () => {});
it("reset returns 409 ESTALE when the head moved since preflight", async () => {});
it("remove tears down the worktree and the branch, and deletes the lane row", async () => {});
it("reset and remove refuse an adopted lane with 400 ENOTMANAGED", async () => {});
it("purge deletes the lane's sessions and reports the counts", async () => {});
```
The adopted-lane refusal is the single most important test in this plan: it is what stands between a mis-click and a user's real project directory.
- [ ] **Step 2: Run to verify they fail** — Expected: FAIL, the actions are unknown or non-destructive.
- [ ] **Step 3: Implement**
Inside the existing `/:id/:action` handler, all three branches run within `withLaneLock(lane.id, async () => …)`, and each begins by killing the lane's run and awaiting its exit (poll `runs.getRun(lane.run_id)` until it is no longer `running`/`spawning`, bounded, then clear `run_id`).
Map the guard errors to HTTP: `ENOTMANAGED` / `EOUTSIDEROOT` / `ENOTWORKTREE` → `400` with the code intact; `ESTALE` → `409`; `EUNPUSHED` → `409`; git failures → `500` carrying `err.git.stderr`.
- [ ] **Step 4: Run to verify they pass** — Expected: PASS, 14 tests in the file.
- [ ] **Step 5: Full suite and commit**
Run: `npm run test:server`
```bash
git add server/routes/lanes.js server/__tests__/lane-lifecycle.test.js
git commit -m "feat(lanes): reset, remove and purge with preflight and stale-state guards"
```
---
## Task 6: CLI
**Files:**
- Modify: `bin/ccam.js`
- Test: `server/__tests__/lanes-cli.test.js` (append)
**Interfaces:**
- Consumes: the routes from Tasks 3-5, via the existing `get` / `post` helpers (`bin/ccam.js:191-192`).
- Produces: `ccam lanes add --repo <path> [--title <t>] [--base <branch>]` (worktree mode; the existing `--cwd` form still adopts); `ccam lanes reset|remove|purge <id> [--force]`, each printing the preflight table and refusing without `--yes`.
- [ ] **Step 1: Write the failing tests** (append) — worktree add via CLI lands a managed lane; `reset` without `--yes` exits non-zero and changes nothing; `--yes` performs it.
- [ ] **Step 2: Run to verify they fail.**
- [ ] **Step 3: Implement**, reusing the async `cli()` harness and the existing flag reader. Print the preflight counts as a small aligned table before asking for `--yes`, so the terminal path has the same "confirm against numbers" property as the UI.
- [ ] **Step 4: Run to verify they pass.**
- [ ] **Step 5: Commit** — `feat(lanes): ccam lanes add --repo, reset, remove, purge`
---
## Task 7: UI and docs
**Files:**
- Create: `client/src/components/lanes/DestructiveLaneModal.tsx`
- Modify: `client/src/components/lanes/LaneCard.tsx`, `client/src/lib/api.ts`, `client/src/lib/types.ts`, `client/src/i18n/locales/*/lanes.json`
- Modify: `docs/LANES.md`, `CLAUDE.md`
- Test: `client/src/components/lanes/__tests__/DestructiveLaneModal.test.tsx`
**Interfaces:**
- Consumes: `api.lanes.preflight(id, action)` and `api.lanes.action(id, action, body)`.
- Produces: `<DestructiveLaneModal lane action onClose onConfirm>` — fetches preflight on open, renders the counts, disables the confirm button while loading or when `blocked` contains anything other than `unpushed-commits`, and exposes a "Force" checkbox only for `unpushed-commits`.
- [ ] **Step 1: Write the failing test** — the modal renders the counts it was given; the confirm button is disabled for an `adopted` lane; ticking Force enables confirm when the only blocker is unpushed commits; confirming passes back the `expect` block it displayed.
- [ ] **Step 2: Run to verify it fails.**
- [ ] **Step 3: Implement**, wrapping the repo's existing `ConfirmModal`. `LaneCard` shows a `managed`/`adopted` badge and renders reset/remove/purge only for `managed` lanes. Every string goes through i18n in all four locales.
- [ ] **Step 4: Run `npm run test:client` and `npm run build`.** Review the screens snapshot diff before accepting it.
- [ ] **Step 5: Docs** — `docs/LANES.md` gains a Lifecycle section covering the two kinds, the three safety checks, each verb with what it destroys and what it spares (`clean -fd` keeps gitignored files), the preflight contract, the env vars, and the fresh-worktree-has-no-dependencies limitation. `CLAUDE.md`'s Lanes section gains the rule: **never `rm -rf` a lane; never build a git command as a shell string; adopted lanes are not destroyable.**
- [ ] **Step 6: Header audit and commit** — `feat(lanes): destructive-action modal with preflight counts, lifecycle docs`
---
## Out of scope
- Dependency bootstrap for a fresh worktree (`node_modules`, `.env`) — Shipyard's profile-hook subsystem. A separate sub-project if wanted.
- `VACUUM` as part of `purge` — it locks the whole database; if disk reclamation is wanted it becomes its own maintenance action.
- Per-lane ports, databases, Docker services.
- Stage auto-detection (sub-project B) and the merged Workspace page (sub-project A) — separate specs.
docs/superpowers/plans/2026-07-29-workspace-ui.md
+273
View File
@@ -0,0 +1,273 @@
# Workspace UI Rebuild Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Rebuild the Workspace page to the reference screen's legibility — card grid, large pipeline, collapsible console — and fill the two data gaps that make lanes look emptier than they are (git facts, expiring detection).
**Architecture:** Design doc: `docs/superpowers/specs/2026-07-29-workspace-ui-design.md` — read it once before Task 1. Two server tasks land first because the client renders what they produce: detection expiry in `recordDetection`, and a read-only `GET /api/lanes/:id/git` reusing `worktree.js`'s existing `git()` and `statusCounts()`. Then the card is rebuilt, then the page shell around it.
**Tech Stack:** Node 18+, Express, better-sqlite3, `node:test` (server); React 18 + TypeScript + Vite + Tailwind, Vitest + Testing Library (client).
## Global Constraints
- Branch: `feat/workspace-ui`, cut from the head of `feat/workspace-page`. Never work on `master`.
- Every `.js/.ts/.tsx` created or modified MUST start with a file overview comment plus the exact line `@author Nguyễn Ngọc Trí Vĩ <vinnt@smartgift.vn>`. Verify with `bash .claude/skills/file-headers/scripts/check-headers.sh` (must exit 0).
- **Detection never writes `lanes.stage`**, and **an inferred node never renders `done`.** Both are load-bearing invariants from sub-project B; a change that lets either slip is a failed task regardless of what else it achieves.
- **No git command may be built as a shell string.** `execFile` with an argv array only, through the existing `git()` wrapper in `server/lib/worktree.js` — it scrubs the inherited `GIT_*` environment, and that scrub exists because a real bug was traced to it.
- The destroy guard (`assertDestroyable`) and the preflight/`expect` echo are not touched by this plan.
- `GET /api/lanes` stays free of git subprocesses. Git facts are their own endpoint.
- Schema changes are additive with a per-column probe (`try { SELECT col } catch { ALTER }`).
- Server CommonJS. No new npm dependencies. Server tests `node:test` + `node:assert/strict`; client tests Vitest + Testing Library. Exact-value assertions, no bare sleeps.
- i18n strings in all four locales (`en`, `zh`, `vi`, `ko`), genuinely translated — no English copied into the other three.
- **Node 24 is required to run the suites.** Node 25 breaks 20 client tests (global `localStorage`) and 6 server tests (better-sqlite3 ABI). Run with `PATH="$HOME/.nvm/versions/node/v24.14.1/bin:$PATH"`.
- The pre-commit hook runs Prettier plus both suites and takes minutes. Let it finish. NEVER `--no-verify`.
- Baseline at branch point: 906 server tests, 347 client tests, all passing. Each task leaves `git status --short` empty.
---
## Task 1 (D1): detection expires
**Files:** Modify `server/lib/lanes.js`, `server/__tests__/lanes-lib.test.js`.
**Produces:** `recordDetection` gains a staleness window. When the lane's
`detected_at` is older than `DETECTION_TTL_MS` (read from `process.env`, default
`1_800_000`), the forward-only comparison against `detected_stage` is skipped
entirely and a fresh detection is accepted even if it sits behind. Inside the
window, behaviour is byte-for-byte what it is today.
The declared-wins rule is NOT affected by the window: a lane whose declared
stage leads still refuses the detection, stale or not. Only the
detected-vs-detected comparison expires.
A lane with `detected_stage` set but `detected_at` NULL (rows written before
this column was populated) is treated as stale — an unknown age cannot be
proven fresh.
- [ ] **Step 1: write the failing tests** in `lanes-lib.test.js`: a backward detection inside the window still returns `behind-detected` and writes nothing; the same backward detection with `detected_at` set beyond the TTL is written and returns `{written: true}`; a stale detection that is behind the DECLARED stage still returns `behind-declared`; a lane with `detected_stage` set and `detected_at` NULL accepts a backward detection; the TTL reads from `DETECTION_TTL_MS`. Set `detected_at` by writing the column directly in the fixture — do not sleep.
- [ ] **Step 2: run, confirm they fail** — `node --test server/__tests__/lanes-lib.test.js`.
- [ ] **Step 3: implement.** One added branch in `recordDetection`. Do not touch `withDetected`, `clearLane`, or the payload shape.
- [ ] **Step 4: run, confirm they pass;** then the full server suite.
- [ ] **Step 5:** header audit, commit — `feat(lanes): expire a stale detection so a lane can move backwards between sessions`.
---
## Task 2 (D2): the detected signal says what matched
**Files:** Modify `server/lib/stage-detect.js`, `server/__tests__/stage-detect.test.js`.
**Produces:** `detect()` returns a `signal` built from the span the rule's regex
actually matched plus surrounding context, instead of the whole flattened input.
A rule with no `match` (it fired on the tool name alone) keeps today's behaviour:
the flattened input, capped. The existing `capSignal` cap (120 chars, whitespace
collapsed, ellipsis) still applies last.
Concretely: `Bash` with
`cd /very/long/path && npm run test:server 2>&1 | tail -5` currently yields the
whole string; it must yield a signal containing `npm run test:server` and not the
`cd` prefix.
`detect()` must remain total — it never throws on any input.
- [ ] **Step 1: write the failing tests:** the Bash example above yields a signal containing `npm run test:server` and not `/very/long/path`; a rule with no `match` still yields the flattened input; a signal longer than the cap is still capped with the ellipsis; a matched span at the very start and at the very end of the input both survive; `detect` still returns null for an unmentioned tool.
- [ ] **Step 2: run, confirm they fail.**
- [ ] **Step 3: implement.**
- [ ] **Step 4: run, confirm they pass;** then the full server suite.
- [ ] **Step 5:** header audit, commit — `feat(lanes): report the matched span as the detection signal`.
---
## Task 3 (D3): `gitFacts()` in the worktree library
**Files:** Modify `server/lib/worktree.js`; modify `server/__tests__/worktree.test.js`.
**Produces:** `gitFacts(dir)` returning `{branch, head, subject, dirty, untracked}`.
It reuses the EXISTING `git()` wrapper and `statusCounts(dir)` in the same file —
do NOT add a second subprocess helper and do NOT build any command as a shell
string. `branch` comes from `rev-parse --abbrev-ref HEAD`, `subject` from
`log -1 --format=%s`, and `head`/`dirty`/`untracked` come from `statusCounts`.
It throws nothing the caller must catch beyond what `git()` already throws; the
route in D4 decides what a failure means.
No route, no HTTP, no OpenAPI in this task.
- [ ] **Step 1: write the failing tests** against a real temporary git repo fixture (`git init`, one commit, then one modified tracked file and one untracked file): the branch name, the short head matching `rev-parse --short HEAD`, the exact commit subject, `dirty: 1`, `untracked: 1`. Also: a detached HEAD yields a `branch` of `HEAD` (assert the exact value the command returns, do not invent one); a repo whose only commit has a subject containing spaces returns it whole.
- [ ] **Step 2: run, confirm they fail.**
- [ ] **Step 3: implement.**
- [ ] **Step 4:** full server suite.
- [ ] **Step 5:** header audit, commit — `feat(lanes): read branch, head, subject and working-tree counts from a worktree`.
---
## Task 4 (D4): the `GET /api/lanes/:id/git` route
**Files:** Modify `server/routes/lanes.js`, `server/openapi-extra/lanes.js` (+ regenerate `openapi.yaml`); modify `server/__tests__/lanes-api.test.js`.
**Produces:** `GET /api/lanes/:id/git` → `200 {available: true, ...facts}` for a git
worktree; `200 {available: false}` when the lane's `cwd` is missing, is not a git
repo, or git fails for any reason. A missing lane is `404`. It is a READ endpoint:
no same-origin guard (that guard is for the destructive actions), and it must
never mutate a lane.
Register the route **before** the `/:id/:action` catch-all, the same way
`/ensure` had to be — otherwise `git` is swallowed as an action name. State in
your report that you checked the ordering and how.
- [ ] **Step 1: write the failing tests:** a lane pointing at a real temporary git repo returns the branch, short head, subject, `dirty` and `untracked`; a lane whose `cwd` is a plain directory returns `{available: false}` with HTTP 200; a lane whose `cwd` does not exist returns `{available: false}`; an unknown lane id returns 404; the route is NOT shadowed by `/:id/:action` — assert the response body shape, not merely the status.
- [ ] **Step 2: run, confirm they fail.**
- [ ] **Step 3: implement.**
- [ ] **Step 4:** full server suite; `node scripts/generate-openapi-yaml.js` then confirm `git diff openapi.yaml` shows only the added path.
- [ ] **Step 5:** header audit, commit — `feat(lanes): expose a lane's git facts over the API`.
---
## Task 5 (D5): client API + types for git facts
**Files:** Modify `client/src/lib/api.ts`, `client/src/lib/types.ts`; modify the matching api test if one exists, else add the assertion to `client/src/lib/__tests__/`.
**Produces:** `api.lanes.git(id)` calling `GET /api/lanes/:id/git`, and a
`LaneGitFacts` type (`{available: true, branch, head, subject, dirty, untracked} | {available: false}`)
exported from `client/src/lib/types.ts`. Nothing renders it yet.
Keep the discriminated union — a caller must be forced to check `available`
before reading `branch`. Do not make the fields optional on one flat type.
- [ ] **Step 1: write the failing test:** `api.lanes.git(3)` requests exactly `/api/lanes/3/git` with method GET, and returns the parsed body.
- [ ] **Step 2: run, confirm it fails.**
- [ ] **Step 3: implement.**
- [ ] **Step 4:** `npm run test:client`, `npm run build`.
- [ ] **Step 5:** header audit, commit — `feat(lanes): client binding for a lane's git facts`.
---
## Task 6 (D6): rebuild the lane card's own fields
**Files:** Modify `client/src/components/lanes/LaneCard.tsx`, `client/src/i18n/locales/{en,zh,vi,ko}/lanes.json`; create `client/src/components/lanes/__tests__/LaneCard.test.tsx`.
**Produces:** the card laid out per the design doc's table, using only fields the
lane payload ALREADY carries: header row (`LANE <id>`, liveness dot, status),
title, declared-stage chip with progress bar / `%` / time-on-stage, the
dashed-amber `auto: <stage>` chip carrying `detected_signal` as its tooltip, the
kind and CI tags, the needs-you banner, and the action row.
**No git block in this task** — that is D7. Do not call `api.lanes.git` here.
The existing action wiring and `DestructiveLaneModal` usage are preserved
exactly: `reset` and `remove` keep their preflight and `expect` echo. Every
string goes through i18n in all four locales, genuinely translated.
The card shows a chip, never a node state — it must not render a detected stage
as done.
- [ ] **Step 1: write the failing tests** in `LaneCard.test.tsx`: every field of a fully-populated fixture lane renders with its exact value; the `auto` chip appears only when the detected stage leads the declared one, and its `title` contains the signal; a lane whose detected stage equals or trails the declared one shows NO auto chip; clicking `reset` opens the destructive modal rather than firing the action directly; the plain action callbacks fire with the right action name.
- [ ] **Step 2: run, confirm they fail** — `cd client && npx vitest run src/components/lanes/__tests__/LaneCard.test.tsx`.
- [ ] **Step 3: implement.**
- [ ] **Step 4:** `npm run test:client`, `npm run build`.
- [ ] **Step 5:** header audit, commit — `feat(lanes): rebuild the lane card for legibility`.
---
## Task 7 (D7): the card's git block
**Files:** Modify `client/src/components/lanes/LaneCard.tsx`, `client/src/i18n/locales/{en,zh,vi,ko}/lanes.json`; modify `client/src/components/lanes/__tests__/LaneCard.test.tsx`.
**Produces:** the card fetches its own facts through `api.lanes.git(lane.id)` on
mount and every 30s, and renders a git row — branch, short head, commit subject,
and the dirty/untracked counts. It renders the rest of the card unchanged while
the facts are still loading and whenever `available` is false. A failed request
is silent: no error banner, no retry storm.
The interval must be cleared on unmount.
- [ ] **Step 1: write the failing tests:** the git row renders each fact from a mocked `available: true` response; an `available: false` response renders the card with NO git row and no error; a rejected request renders the card with no git row and no error; unmounting clears the interval (assert the timer count, or that no further request is made after unmount).
- [ ] **Step 2: run, confirm they fail.**
- [ ] **Step 3: implement.**
- [ ] **Step 4:** `npm run test:client`, `npm run build`.
- [ ] **Step 5:** header audit, commit — `feat(lanes): show a lane's branch and working-tree state on its card`.
---
## Task 8 (D8): the page header and the card grid
**Files:** Modify `client/src/pages/Workspace.tsx`, `client/src/i18n/locales/{en,zh,vi,ko}/lanes.json`; modify `client/src/pages/__tests__/Workspace.test.tsx`.
**Produces:** the header bar — page title, the four counters (`lanes`, `running`,
`needs you`, `dead`) from the API's `counts`, and the Add-lane control — and the
responsive card grid (1 column, 2 at `md`, 3 at `xl`) replacing today's
horizontal lane strip. Selecting a card still drives the same `selectedLaneId`
state it does now.
Do NOT touch the console or the pipeline panel in this task; leave them exactly
where they are, below the grid, however they currently render.
- [ ] **Step 1: write the failing tests:** the four counters render the exact values from the API's `counts`; every lane in the response gets a card; clicking a card sets it selected (assert an observable consequence, e.g. the pipeline panel's lane, not an internal state variable).
- [ ] **Step 2: run, confirm they fail.**
- [ ] **Step 3: implement.**
- [ ] **Step 4:** `npm run test:client`, `npm run build`. The screens snapshot WILL change — read the diff, confirm it is only the header and grid, then regenerate.
- [ ] **Step 5:** header audit, commit — `feat(lanes): lane grid and counters in the Workspace header`.
---
## Task 9 (D9): the selected-lane detail panel
**Files:** Modify `client/src/pages/Workspace.tsx`, `client/src/i18n/locales/{en,zh,vi,ko}/lanes.json`; modify `client/src/pages/__tests__/Workspace.test.tsx`.
**Produces:** the detail panel between the header and the grid: the selected
lane's title, its declared stage and — when detection leads — the inferred one,
a large `PipelineMap`, and the legend naming the five node states plus the
dashed-amber inferred treatment.
**Do not change `PipelineMap` itself** — not its node-state logic, not its props.
This task places and sizes it.
- [ ] **Step 1: write the failing tests:** the panel shows the selected lane's title and declared stage; selecting a different card switches the panel's pipeline; **no node rendered in the panel carries both `data-detected="true"` and `data-state="done"`** (the sub-project B premise guard, re-asserted at the new layout); the legend names each of the five states.
- [ ] **Step 2: run, confirm they fail.**
- [ ] **Step 3: implement.**
- [ ] **Step 4:** `npm run test:client`, `npm run build`, snapshot diff read.
- [ ] **Step 5:** header audit, commit — `feat(lanes): a full-width pipeline panel for the selected lane`.
---
## Task 10 (D10): collapse the console
**Files:** Modify `client/src/pages/Workspace.tsx`, `client/src/i18n/locales/{en,zh,vi,ko}/run.json`; modify `client/src/pages/__tests__/Workspace.test.tsx`.
**Produces:** `RunSetup` + `RunConsole` + `RunHistory` wrapped in a disclosure
that starts collapsed and expands on click, with the console section moved above
the card grid so an expanded console sits beside the lane it belongs to.
**The subscription must stay mounted while collapsed.** Collapse the visual
container with CSS; do NOT conditionally unmount `RunConsole` — unmounting
disposes `useRunStream`'s subscription and a live run's envelopes are lost.
State in your report which mechanism you used and how you proved the
subscription survived.
No prop of `RunSetup`, `RunConsole` or `RunHistory` changes. The console still
never posts a stage.
- [ ] **Step 1: write the failing tests:** the console is collapsed on first render and expands on click; **an envelope delivered through the mocked event bus while the console is collapsed is present in the DOM once it is expanded**; after a full start-and-message cycle no request is made to any `/stage` URL.
- [ ] **Step 2: run, confirm they fail.**
- [ ] **Step 3: implement.**
- [ ] **Step 4:** `npm run test:client`, `npm run build`, snapshot diff read then regenerated.
- [ ] **Step 5:** header audit, commit — `feat(lanes): collapse the console without dropping its stream`.
---
## Task 11 (D11): docs
**Files:** Modify `docs/LANES.md`, `docs/API.md`, `README.md`, `ARCHITECTURE.md`, `CLAUDE.md`.
**Produces:** the new layout described where the old one was; `GET /api/lanes/:id/git` documented with its `available: false` contract and the reason it is not folded into `GET /api/lanes`; the detection TTL documented with `DETECTION_TTL_MS`, its default, and the explicit note that expiry does NOT weaken declared-wins or let inference render `done`. Every path, route and command printed must exist — verify each.
- [ ] **Step 1:** write the docs.
- [ ] **Step 2:** verify every referenced file, route and command exists (`ls`, `grep`, or run it).
- [ ] **Step 3:** full server suite, full client suite, `node scripts/generate-openapi-yaml.js` then `git diff openapi.yaml` empty.
- [ ] **Step 4:** header audit, commit — `docs(lanes): document the rebuilt Workspace, git facts and detection expiry`.
---
## Out of scope
- Tickets, preview-port links, per-lane credentials, and the `agents`/`creds` buttons from the reference screen — CCAM has no data behind any of them.
- Any change to `PipelineMap`'s node-state logic, the destroy guard, or the preflight contract.
- Inferring `done` or a gate outcome. Still forbidden, TTL or not.
- Re-styling any page other than `/run`.
docs/superpowers/specs/2026-04-24-agent-conversation-viewer-design.md
+419
View File
@@ -0,0 +1,419 @@
# Agent Conversation Viewer Design
## Overview
Add a conversation viewer to the SessionDetail page, enabling visual inspection of Main Agent and sub-agent interactions (message content and tool call details), with data sourced from real-time JSONL transcript files.
## Problem
The current dashboard tracks agent sessions, events, and tool usage at a summary level, but does not expose the actual conversation content — user messages, assistant replies, tool call parameters, and tool results. Users cannot see what each agent actually did or said, limiting debugging and audit capabilities.
### v2 Additional Problems: Poor Pagination UX + No Real-time Updates
After v1 implementation, two core UX issues emerged:
1. **Pagination doesn't match conversation intuition** — v1 uses offset-based pagination starting from the beginning, so users see the oldest messages first and must page through to reach recent interactions, which doesn't align with chat product conventions.
2. **No real-time updates** — v1 doesn't subscribe to WebSocket events, so users must manually refresh to see new messages, making it impossible to follow active sessions in real time.
3. **Sub-agent selection uses database IDs** — v1's `agent_id` parameter relies on database agent IDs, but JSONL files are named with short IDs (e.g. `ad18a79192af10ed1`), causing a mismatch that prevents sub-agent transcripts from loading.
## Design Decisions
| Decision | Choice | Rationale |
|----------|--------|-----------|
| Data source | Real-time JSONL reads | Data is always current, no extra storage needed |
| UI location | Conversation tab within SessionDetail | User-requested; keeps agent tree in the same context |
| Claude home path | Configurable via `CLAUDE_HOME` env var | Supports non-default paths like `~/.codefuse/engine/cc/` |
| Message rendering | Collapsible tool calls and thinking blocks | Keeps the view scannable; expand for details |
| Load strategy (v2) | Chat-flow: load latest N by default, scroll up for history | Matches chat product intuition; users care most about recent interactions |
| Real-time updates (v2) | WebSocket `new_event` triggers incremental load | Active sessions don't need manual refresh |
| Agent selection (v2) | Filesystem scan + dropdown | Bypasses database ID mismatch by using file short IDs directly |
## Architecture
### Data Flow
**v1 (deprecated):**
```
User clicks "Conversation" tab
→ Frontend calls GET /api/sessions/:id/transcript[?agent_id=xxx&limit=50&offset=0]
→ Server resolves JSONL path via claude-home.js
→ Server reads and parses JSONL file
→ Server returns structured message list
→ Frontend renders MessageList (with collapsible blocks)
```
**v2 Chat-flow (current implementation):**
```
Initial load:
User opens Conversation tab
→ GET /api/sessions/:id/transcripts ← fetch available transcript list
→ GET /api/sessions/:id/transcript?limit=50 ← default returns latest 50 messages
→ Frontend renders message list + auto-scrolls to bottom
Real-time updates:
CLI Hook → POST /api/hooks/event → processEvent()
→ broadcast("new_event", {session_id, ...})
→ WebSocket → ConversationView
→ GET /api/sessions/:id/transcript?after=N ← incremental load
→ Append to bottom + auto-scroll (if user is at bottom)
History load:
User scrolls to top
→ GET /api/sessions/:id/transcript?before=M&limit=50 ← load older messages
→ Prepend to top + preserve scroll position (no jump)
```
### Configurable Claude Home Directory
New module `server/lib/claude-home.js` centralizes all Claude directory path logic:
```
CLAUDE_HOME env var (default: ~/.claude)
├── projects/<encoded-cwd>/<session-id>.jsonl ← main session transcript
│ (encoding rule: all non-alphanumeric chars → "-", e.g. "/Users/txj/.codefuse" → "-Users-txj--codefuse")
├── projects/<encoded-cwd>/<session-id>/subagents/agent-<id>.jsonl ← sub-agent transcript
│ (sub-agent ID format: ad18a79192af10ed1, acompact-f8427be966459435)
└── settings.json ← hooks configuration
```
Existing hardcoded paths in `import-history.js`, `install-hooks.js`, and `settings.js` are migrated to use this module.
---
## API
### GET /api/sessions/:id/transcripts (v2 new)
List available transcript files for a session (main + sub-agents), scanned directly from the filesystem.
**Response (200):**
```json
{
"transcripts": [
{ "id": "main", "name": "Main Agent", "type": "main", "has_transcript": true },
{ "id": "ad18a79192af10ed1", "name": "code-reviewer", "type": "subagent", "subagent_type": "code-reviewer", "has_transcript": true },
{ "id": "acompact-f8427be966459435", "name": "Context Compaction", "type": "compaction", "has_transcript": true }
]
}
```
**Design notes:**
- Bypasses database agent IDs; scans the filesystem directly for JSONL file short IDs
- `id` field maps directly to the filename: `agent-<id>.jsonl`, used as the `agent_id` parameter for the `transcript` API
- Compaction file name format: `agent-acompact-<hex>.jsonl`, id is `acompact-<hex>`
- Attempts to read `.meta.json` in the same directory for agent type description
- Falls back to scanning all `projects/` subdirectories when the exact encoded path doesn't exist
### GET /api/sessions/:id/transcript
Read a session's JSONL transcript file and return a structured message list.
**Query Parameters:**
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `agent_id` | string | null | Transcript short ID (from `transcripts` endpoint); omit for main session |
| `limit` | number | 50 | Max messages to return (max 200) |
| `after` | number | null | Incremental mode: only return messages with JSONL line > after (v2 new) |
| `before` | number | null | History mode: only return the latest N messages with JSONL line < before (v2 new) |
| `offset` | number | 0 | Legacy pagination offset (compatible, mutually exclusive with after/before) |
**Response (200):**
```json
{
"messages": [
{
"type": "user",
"timestamp": "2026-04-24T10:23:45Z",
"content": [
{ "type": "text", "text": "Please implement the login feature" }
]
},
{
"type": "assistant",
"timestamp": "2026-04-24T10:23:52Z",
"model": "claude-sonnet-4-6",
"usage": { "input_tokens": 1500, "output_tokens": 800 },
"content": [
{ "type": "text", "text": "I'll help you implement the login feature." },
{ "type": "thinking", "text": "Let me analyze the codebase..." },
{
"type": "tool_use",
"name": "Read",
"id": "toolu_abc123",
"input": { "file_path": "/src/auth.ts" }
}
]
}
],
"total": 120,
"has_more": true,
"last_line": 523,
"first_line": 474
}
```
**v2 New Response Fields:**
| Field | Type | Description |
|-------|------|-------------|
| `last_line` | number | JSONL line number of the last message in the current response; used as the `after` parameter for incremental requests |
| `first_line` | number | JSONL line number of the first message in the current response; used as the `before` parameter for history loading |
**Loading Modes:**
| Mode | Parameters | Behavior | Use Case |
|------|-----------|----------|----------|
| Default | No after/before/offset | Return the latest N messages | Initial load |
| Incremental | `after=N` | Return messages with line > N (up to limit) | WebSocket-triggered new message loading |
| History | `before=M` | Return the latest N messages with line < M | Scroll-up to load older messages |
| Compatible | `offset=K` | Skip first K, return next N | Legacy pagination (kept for compatibility) |
**Error Responses:**
| Status | Condition |
|--------|-----------|
| 200 | When JSONL file doesn't exist, returns empty `{ messages: [], total: 0, has_more: false, last_line: 0, first_line: 0 }` |
| 404 | Session ID not found in database |
**Implementation Rules:**
- Only extract entries with `type: "user"` or `type: "assistant"`; skip system/progress entries
- Match `tool_use` and `tool_result` via `id` field; unpaired tool_use shows no result section
- Truncate individual content exceeding 10KB, appending `[truncated]`
- Re-read the file on every request (no server-side caching) to ensure real-time freshness
- When `cwd` is null, scan all `projects/` subdirectories to find the JSONL for the sessionId
- Internally use JSONL line numbers as cursors; remove the `line` field from responses, expose `first_line` / `last_line` to the client
---
## Frontend
### SessionDetail Page Changes
Replace the current flat layout with a **tabbed interface**:
```
[Agents] [Conversation] [Timeline]
```
- **Agents tab** — existing agent hierarchy tree (active by default)
- **Conversation tab** — new conversation viewer
- **Timeline tab** — existing event timeline
### Conversation Tab Components
**v2 Chat-flow architecture:**
```
ConversationView.tsx
├── TranscriptSelector — dropdown selector (v2 replaces AgentFilter)
├── ScrollContainer — scrollable message container
│ ├── HistoryLoader — scroll-up history loading indicator
│ └── MessageList.tsx
│ ├── UserMessage — user message
│ └── AssistantMessage
│ ├── TextBlock — plain text content
│ ├── ThinkingBlock — collapsible thinking content
│ └── ToolCallBlock — collapsible tool call + result
│ ├── ToolUse — tool name + parameters
│ └── ToolResult — execution result / error
└── NewMsgButton — "New messages" floating button (v2 new)
```
### TranscriptSelector (v2 replaces AgentFilter)
- Top dropdown selector: `[Main Agent ▾]` or `[Context Compaction ▾]`
- Data source: `GET /api/sessions/:id/transcripts` (filesystem scan, not database)
- Reloads the corresponding transcript on switch
- Only shown when transcripts > 1
- Message count displayed alongside: `518 messages`
### Chat-flow Behavior (v2 new)
**Initial load:**
- Call `transcript?limit=50` to get the latest 50 messages
- Auto-scroll to bottom after rendering
- Track `last_line` and `first_line` for subsequent requests
**Real-time updates (WebSocket-driven):**
- Subscribe to `eventBus` `new_event` events
- Only process events where `session_id` matches the current session
- On event, call `transcript?after=last_line&limit=50` for incremental loading
- If user is at bottom (< 100px from bottom), auto-scroll to latest message
- If user has scrolled up, show "New messages" floating button; click to scroll to bottom
**Scroll-up history loading:**
- Listen for scroll events; trigger when `scrollTop < 50` and `has_more` is true
- Call `transcript?before=first_line&limit=50` to fetch older messages
- Prepend to top of list; preserve scroll position via `scrollHeight` delta
- Show spinner while loading; show "↑ Scroll up for older messages" hint at top
**Key Refs:**
- `lastLineRef` — tracks the JSONL line number of the newest message, used for incremental requests
- `firstLineRef` — tracks the JSONL line number of the oldest loaded message, used for history loading
- `scrollContainerRef` — scroll container DOM reference
- `isAtBottomRef` — boolean flag tracking whether user is at the bottom
### Message Rendering
- **User messages**: right-aligned, blue background, display text content
- **Assistant messages**: left-aligned, default background, including:
- Model name and token usage as faded metadata
- Text blocks rendered inline
- Thinking blocks: collapsed by default, click to expand (dimmed style)
- Tool calls: collapsed by default showing only tool name, click to expand:
- Tool name as header with icon
- Input parameters formatted as JSON (collapsible)
- Tool result with success/error indicator
### Interaction Details
- **Long text truncation**: content over 500 characters is truncated by default, with an "expand" link
- **Lazy loading (v2)**: initial load of latest 50 messages; scroll-up auto-loads older 50; WebSocket-driven incremental append
- **Real-time updates (v2)**: on WebSocket `new_event` with matching `session_id`, incrementally load new messages
- **Auto-scroll (v2)**: auto-scroll to latest when user is at bottom; show floating "New messages" button when user has scrolled up
- **Empty state**: when JSONL is missing or empty, show "No conversation records found."
---
## Server Module: claude-home.js
```js
// Centralized Claude home directory path management
function getClaudeHome() {
return process.env.CLAUDE_HOME || path.join(os.homedir(), ".claude");
}
function getProjectsDir() {
return path.join(getClaudeHome(), "projects");
}
function getSettingsPath() {
return path.join(getClaudeHome(), "settings.json");
}
// Encoding rule: all non-alphanumeric characters replaced with "-"
// Example: "/Users/txj/.codefuse" → "-Users-txj--codefuse"
function encodeCwd(cwd) {
return cwd.replace(/[^a-zA-Z0-9]/g, "-");
}
function getTranscriptPath(sessionId, cwd) {
if (!cwd) return null;
const encoded = encodeCwd(cwd);
const candidate = path.join(getProjectsDir(), encoded, `${sessionId}.jsonl`);
if (fs.existsSync(candidate)) return candidate;
// Fallback: scan projects/ subdirectories
return findTranscriptPath(sessionId);
}
function getSubagentTranscriptPath(sessionId, cwd, agentId) {
if (!cwd) return null;
const encoded = encodeCwd(cwd);
const candidate = path.join(getProjectsDir(), encoded, sessionId, "subagents", `agent-${agentId}.jsonl`);
if (fs.existsSync(candidate)) return candidate;
// Fallback: scan all project directories
return findSubagentTranscriptPath(sessionId, agentId);
}
function findTranscriptPath(sessionId) {
// Fallback: when cwd is unknown, scan projects/ subdirectories
const projectsDir = getProjectsDir();
if (!fs.existsSync(projectsDir)) return null;
const dirs = fs.readdirSync(projectsDir, { withFileTypes: true });
for (const d of dirs) {
if (!d.isDirectory()) continue;
const candidate = path.join(projectsDir, d.name, `${sessionId}.jsonl`);
if (fs.existsSync(candidate)) return candidate;
}
return null;
}
// v2 new: support prefix fuzzy matching for compaction type
function findSubagentTranscriptPath(sessionId, agentId) {
const projectsDir = getProjectsDir();
if (!fs.existsSync(projectsDir)) return null;
const dirs = fs.readdirSync(projectsDir, { withFileTypes: true });
for (const d of dirs) {
if (!d.isDirectory()) continue;
const subagentsDir = path.join(projectsDir, d.name, sessionId, "subagents");
if (!fs.existsSync(subagentsDir)) continue;
// Exact match
const exact = path.join(subagentsDir, `agent-${agentId}.jsonl`);
if (fs.existsSync(exact)) return exact;
// Prefix fuzzy match (compaction type: agentId starts with "acompact-")
if (agentId.startsWith("acompact-")) {
const files = fs.readdirSync(subagentsDir);
const match = files.find(f => f.startsWith("agent-acompact-") && f.endsWith(".jsonl"));
if (match) return path.join(subagentsDir, match);
}
}
return null;
}
```
---
## File Changes
| File | Action | Description |
|------|--------|-------------|
| `server/lib/claude-home.js` | **New** | Claude home directory path management; v2 adds `findSubagentTranscriptPath` prefix fuzzy matching |
| `server/routes/sessions.js` | Modified | v1: add `GET /sessions/:id/transcript`; v2: add `GET /sessions/:id/transcripts`, transcript endpoint gains `after`/`before` params and `first_line`/`last_line` response |
| `scripts/import-history.js` | Modified | Use `getClaudeHome()` instead of hardcoded path |
| `scripts/install-hooks.js` | Modified | Use `getSettingsPath()` instead of hardcoded path |
| `server/routes/settings.js` | Modified | Use `getClaudeHome()` for hooks detection |
| `client/src/lib/types.ts` | Modified | v1: add `TranscriptMessage`, `TranscriptContent`; v2: add `TranscriptInfo`, `TranscriptListResult`, `TranscriptResult` gains `last_line`/`first_line` |
| `client/src/lib/api.ts` | Modified | v1: add `sessions.transcript()`; v2: add `sessions.transcripts()`, `transcript()` gains `after`/`before` params |
| `client/src/pages/SessionDetail.tsx` | Modified | Add tab switching and Conversation tab; v2: remove `agents` prop from ConversationView |
| `client/src/components/conversation/ConversationView.tsx` | **New** → v2 rewrite | v1: basic pagination; v2: chat-flow mode (WebSocket incremental + scroll-up history + auto-scroll) |
| `client/src/components/conversation/MessageList.tsx` | **New** | Message list (with collapsible blocks, command formatting, skill content folding, task notification folding) |
| `client/src/components/conversation/ToolCallBlock.tsx` | **New** | Collapsible tool call display |
---
## Error Handling
| Scenario | Handling |
|----------|----------|
| JSONL file doesn't exist | Return `{ messages: [], total: 0, has_more: false, last_line: 0, first_line: 0 }`; UI shows "No conversation records found." |
| JSONL line parse failure | Skip the line, continue processing remaining lines |
| Single content exceeds 10KB | Truncate and append `[truncated]` marker |
| Sub-agent JSONL doesn't exist | Same as main file — return empty list |
| Session cwd is null | Use `findTranscriptPath()` to scan project directories |
| CLAUDE_HOME path invalid | Log warning, return empty list |
| Incremental load returns no new messages (v2) | `after` request returns empty array, frontend silently ignores |
| History load failure (v2) | Silent failure, doesn't interrupt user experience |
| WebSocket disconnection (v2) | Doesn't affect loaded messages; next event after reconnect triggers incremental load |
## Edge Cases
- **Compaction**: After `/compact`, older messages are lost from the JSONL. The viewer only shows what's currently in the file — this is expected behavior. Compact transcripts appear as separate entries in the transcript selector.
- **Active sessions**: JSONL may be actively written to. Every request re-reads the file for real-time freshness. WebSocket events trigger incremental loading — no polling needed.
- **Unpaired tool_use/tool_result**: Display the tool call without the result section; no error.
- **Message order**: JSONL is ordered chronologically; responses preserve the same order (oldest first).
- **Database ID vs file ID mismatch (v2)**: Database agent IDs use format `<sessionId>-jsonl-<shortId>`, but JSONL filenames use `agent-<shortId>.jsonl`. v2 bypasses database IDs entirely via the `transcripts` endpoint, which scans the filesystem and uses file short IDs.
- **Compaction filename format (v2)**: In the database, compaction agent IDs use format `<sessionId>-compact-<uuid>`, but filenames use `agent-acompact-<hex>.jsonl`. `findSubagentTranscriptPath` supports prefix fuzzy matching for `agent-acompact-*.jsonl`.
- **Scroll position preservation (v2)**: When loading history, the scroll position is preserved by computing the `scrollHeight` delta, ensuring the viewport content doesn't jump.
- **Duplicate events (v2)**: WebSocket may send multiple `new_event` messages; incremental loading uses `after` line number for deduplication, preventing duplicate appends.
## Testing Strategy
| Layer | Test Content |
|-------|-------------|
| API unit tests | `GET /sessions/:id/transcript` — normal response, file not found, invalid session, pagination params, agent_id filtering |
| API unit tests | `GET /sessions/:id/transcript` — v2: `after` incremental loading, `before` history loading, `first_line`/`last_line` response |
| API unit tests | `GET /sessions/:id/transcripts` — v2: file scanning, compaction type, meta.json reading |
| API unit tests | `claude-home.js` — path inference logic, env var override, fallback scanning, compaction prefix fuzzy matching |
| Frontend component tests | `MessageList` rendering, `ToolCallBlock` collapse/expand, command formatting, skill content folding |
| Frontend component tests | `ConversationView` — v2: initial load, incremental append, history load, scroll detection, new messages indicator |
## Environment Variables
| Variable | Default | Description |
|----------|---------|-------------|
| `CLAUDE_HOME` | `~/.claude` | Claude Code home directory (e.g. `~/.codefuse/engine/cc/`) |
docs/superpowers/specs/2026-05-22-fix-transcript-cache-leak-design.md
+220
View File
@@ -0,0 +1,220 @@
# Design: Fix Agent-Monitor server memory leak
- **Date**: 2026-05-22
- **Author**: zhihua + Claude (brainstorming collaboration)
- **Status**: Design Approved, pending implementation plan
## Background
After running `npm start` locally, the server process memory grows continuously over time and eventually exhausts host memory when combined with Claude / IDE / browser. The initial proposal was to deploy Agent-Monitor on a remote server and access it via the local browser, but investigation showed this only relocates the problem — the root cause is in the server itself, and a long-running remote instance will also OOM.
This design focuses on **root-cause remediation**, not remote deployment. Once memory is stable post-fix, we can revisit whether remote deployment is still desirable.
## Current diagnosis (with code evidence)
Measured locally:
| Metric | Value |
|---|---|
| `data/dashboard.db` | 192 MB |
| `events` row count | 251,244 |
| `sessions` count | 1130 (completed 1015 + abandoned 110 + active 5) |
| Largest single event size | 369 KB |
| `~/.claude/projects` | 58 MB |
Three leak / performance sources were identified:
### Leak #1: TranscriptCache entry has no per-entry size cap
In `server/lib/transcript-cache.js`, every cache entry holds three push-only arrays:
- `state.turnDurations.push(...)` (l.327)
- `state.errors.push(...)` (l.332, 343)
- `state.compaction.entries.push(...)` (l.315)
`_merge()` incremental merging (l.456, 462, 468) likewise only pushes and never trims.
`MAX_CACHE_ENTRIES = 200` bounds the number of entries, but **each entry is unbounded in size**. A long session emits one turnDuration per turn (~50 bytes), so a few thousand turns = MB-scale per entry; 200 entries × tens of MB = **multiple GB**.
### Leak #2: `_set()` stores everything twice (per-entry memory doubled)
`server/lib/transcript-cache.js:51-58`:
```js
this._set(key, {
errors: result?.errors ? [...result.errors] : null, // top-level shallow copy
turnDurations: result?.turnDurations ? [...result.turnDurations] : null,
compaction: this._cloneCompaction(result.compaction),
...
result, // contains references to the same fields
});
```
The top-level fields are shallow-copied (`[...result.errors]`) new array objects that do not share references with `result.errors`. **Each array exists twice on the heap per cache entry.**
### Performance issue: the sweep does a full scan over events
`server/index.js:329` runs every 60-300s:
```sql
SELECT DISTINCT e.session_id, json_extract(e.data,'$.transcript_path') AS tp
FROM events e JOIN sessions s ON s.id=e.session_id
WHERE s.status='active' AND json_extract(e.data,'$.transcript_path') IS NOT NULL
GROUP BY e.session_id ORDER BY MAX(e.id) DESC
```
Doing `json_extract` + DISTINCT + ORDER BY across 250k events rows produces large temporary SQLite memory spikes and is slow.
## Goals and constraints
**Goals**:
1. Server process RSS stays stable over long runs (< 300 MB)
2. No Agent log loss (events table remains complete; no retention)
3. Reversible changes confined to `server/`; no changes to hook-handler / UI / WebSocket protocol
**Non-goals** (explicitly out of scope):
- Remote deployment
- Events table retention / archival
- DB engine swap / compression / sharding
- UI / frontend / CLI changes
## Design
### Change A: TranscriptCache per-entry sliding window
Add a configurable cap:
```js
const MAX_ARRAY_LEN = parseInt(process.env.TRANSCRIPT_CACHE_MAX_ARRAY_LEN, 10) || 1000;
```
After each push in `_streamRange` parsing (l.315/327/332/343 etc.) and in `_merge` incremental merging (l.456/462/468), trim immediately:
```js
if (arr.length > MAX_ARRAY_LEN) arr.splice(0, arr.length - MAX_ARRAY_LEN);
```
Applies to `turnDurations`, `errors`, `compaction.entries`, and `usageExtras.{service_tiers, speeds, inference_geos}` (these Set→Array conversions can also accumulate).
**Why no data loss**:
`routes/hooks.js:583, 633` already inserts `result.errors` / `result.turnDurations` into the events table on every hook trigger, with dedup (`SELECT 1 ... WHERE summary=?` / `WHERE created_at=?`). After cache truncation, the next hook re-reads the transcript file → dedup skips existing rows → only new rows are inserted. The events table stays complete.
**Capacity estimate**:
- 1 turn ≈ 50 bytes
- 1000 turns = 50 KB / cache entry
- 200 entries full ≈ 10 MB
### Change B: Eliminate `_set()` double storage
Simplify the cache entry shape:
```js
this._cache.set(key, { mtimeMs, size, bytesRead, result });
```
Drop all top-level `errors` / `turnDurations` / `compaction` / `usageExtras` / `tokensByModel` / `thinkingBlockCount` / `latestModel` fields. `_merge` computes via local variables and writes back only into `result`.
**Expected effect**: ~50% memory reduction per entry.
### Change C: Stop sweeping events for transcript_path
**Schema migration** (`server/db.js`):
```sql
-- Add column (idempotent)
ALTER TABLE sessions ADD COLUMN transcript_path TEXT;
-- One-time backfill (runs once at startup, gated by a .migrations marker file to prevent reruns)
UPDATE sessions SET transcript_path = (
SELECT json_extract(data,'$.transcript_path') FROM events
WHERE events.session_id=sessions.id
AND json_extract(data,'$.transcript_path') IS NOT NULL
LIMIT 1
) WHERE transcript_path IS NULL;
```
Follow the idempotent migration pattern at `server/db.js:284` (the `agents_new` rebuild).
**Write path** (`server/routes/hooks.js` `ensureSession`):
When `transcript_path` is first seen, run `UPDATE sessions SET transcript_path=? WHERE id=? AND transcript_path IS NULL`.
**Sweep query rewrite** (`server/index.js:329`):
```sql
SELECT id, transcript_path FROM sessions
WHERE status='active' AND transcript_path IS NOT NULL
```
The query at `server/index.js:309` that fetches `transcript_path` on abandonment is also rewritten to read from the sessions table.
**Complexity**: drops from O(total events rows) to O(active sessions ≈ single digits). **Not a single events row is removed.**
## Verification strategy
### Unit tests (new `server/__tests__/transcript-cache-bounded.test.js`)
1. With `MAX_ARRAY_LEN=100`, feed 500 turns → `result.turnDurations.length === 100`, tail retained
2. After cache truncation, re-extracting → events table dedup skips existing rows, insert count == 0
3. Coarse memory assertion: 200 entries × 1000 turns, `process.memoryUsage().heapUsed` delta < 30 MB
### Integration tests
- `npm run test:server` green
- `npm run test:client` green
- `npm run mcp:typecheck` passes
### Measurement script (one-off)
New `scripts/memory-soak-test.js`:
- Generate fake transcript jsonl with 10000 turns
- Start the server, simulate 10 concurrent active sessions, fire a hook every 1s
- Run for 30 minutes, log `process.memoryUsage().rss` per minute
- Assert: RSS growth at minute 30 < 50 MB
### Verification checklist (pre-merge)
- [ ] Unit + integration tests green
- [ ] `npm run mcp:typecheck` passes
- [ ] Local `npm start` for 1h, `ps -o rss=` monitoring shows a flat curve
- [ ] DB migration idempotent: two consecutive `npm start` runs without errors
- [ ] Old DB (no `transcript_path` column) → migrate + backfill → sweep works
- [ ] After cache truncation, the UI events list still shows all old turns/errors
## Risks and rollback
### Risk matrix
| Risk | Likelihood | Impact | Mitigation |
|---|---|---|---|
| `MAX_ARRAY_LEN=1000` too small for ultra-long sessions | Low | Medium | Env var tunable to 5000-10000; events table is always complete, UI can still query |
| Extra dedup SELECTs after cache truncation | Medium | Low | Sweep runs every 60-300s; an extra 100-1000 primary-key lookups per run is acceptable |
| ALTER TABLE fails on old DB | Very low | High | Use the migration pattern at `db.js:284` — try-catch + column-existence check |
| transcript_path backfill is slow due to events scan | Low | Low | One-time migration takes ~1s; use EXISTS subquery instead of join |
| `_set()` shape change breaks other readers | Low | Medium | Grep the repo to confirm all external consumers of `extract()` only read `result.*` |
### Rollback
- All changes are confined to `server/`; **hook-handler / UI / WebSocket protocol untouched**
- Rollback at any phase = `git revert` of the matching commit
- DB schema: `ALTER TABLE ... ADD COLUMN` is not reversible, but an unread/unwritten new column is harmless; once code is reverted, sessions just has an extra empty column
## Optional follow-ups (out of scope here)
- Add a `(session_id, event_type, created_at)` composite index on events (UI query performance)
- Add a `lastProcessedTurnTimestamp` cursor to the cache so `extract` only returns new turns (eliminates dedup SELECTs entirely)
- `/api/internal/memory` diagnostic endpoint returning `cache.stats()` + `process.memoryUsage()`
## Decision record
| Option | Choice | Rationale |
|---|---|---|
| Remote deployment vs fix leak | Fix leak | Remote deployment relocates the problem; the leak hits remote too |
| Permanent events retention vs retention policy | Permanent | Hard user constraint: guarantee Agent log integrity |
| Truncate cache vs not truncate | Truncate to MAX_ARRAY_LEN | Events table already persists raw data; the cache is a derived view |
| Delete events vs rewrite the sweep query | Rewrite the query | Satisfies the "no log loss" constraint |
| Introduce LRU byte-budget instead of entry count | No | Entry-count cap + per-entry cap is already enough; byte accounting adds complexity |
docs/superpowers/specs/2026-05-28-tabby-companion-design.md
+256
View File
@@ -0,0 +1,256 @@
# Tabby — Floating Companion (Design Spec)
**Date:** 2026-05-28
**Status:** Approved (design) — pending spec review before planning
**Owner:** Nguyễn Ngọc Trí Vĩ (David)
**Topic:** A cute-but-functional cat companion that lives in the dashboard's bottom corner, reacts to live session events, and expands into a panel for status, quick actions, and asking questions.
---
## 1. Summary
**Tabby** is a floating cat avatar pinned to the bottom-right corner of the Agent Dashboard on every route. It is two things at once:
1. **A reactive mascot** — an SVG cat whose face, ears, eyes, and posture react in real time to what the monitored Claude Code sessions are doing (a session finishes → tail-up, eyes `^^`; an error/hook fails → arch + ears-back; idle → curls up asleep). Eyes track the cursor when alert.
2. **An assistant** — click the avatar (or press `⌘B` / `Ctrl+B`) to expand a panel with a live status line, quick navigation actions, and an **Ask** box that answers simple questions from cached dashboard data, with a handoff to the existing **Run** page to ask Claude for real.
The "do the job" path reuses what already exists: `POST /api/run` spawns a real `claude` subprocess and streams over WebSocket. Tabby does **not** introduce any new LLM backend, API key, or server route in P1/P2. P3 adds a single client-only deep-link prefill.
Name **Tabby** matches the app's identity: this is a **Monitor** ("watching your agents"), and Tabby is the alert watcher curled in the corner.
---
## 2. Goals / Non-Goals
### Goals
- Delightful, on-theme personality layer over live session data — "cute but does the job."
- Always-present, low-footprint corner avatar that auto-surfaces notable events as transient speech bubbles, then settles.
- One-keystroke (`⌘B`) expand to a functional panel: status, quick actions, local Ask.
- Reuse the existing event stream (`eventBus`) and Run flow — no new backend in P1/P2.
- Fully consistent with the existing dark Tailwind theme (`surface-*`, `accent`, `border`).
- Accessible: keyboard-operable, `aria-live` bubbles, honors `prefers-reduced-motion`.
- Degrades safe: if WebSocket is down/delayed, Tabby shows a calm/disconnected state — never errors, never blocks the page.
### Non-Goals (YAGNI)
- No drag-to-reposition (fixed bottom-right).
- No sound effects.
- No new LLM/chat backend or API key (Ask is rule-based locally; real Claude = handoff to Run).
- No server-side persistence (preferences in `localStorage` only).
- No multi-avatar / skins / customization.
- No changes to existing pages beyond the minimal mount + the P3 Run prefill.
---
## 3. Where it lives (architecture)
```
App.tsx
└─ useWebSocket(onMessage = eventBus.publish) // single shared socket, already exists
└─ Layout.tsx
├─ UpdateNotifier (existing global floater)
├─ Tabby ◀── NEW: mounted here, sibling of UpdateNotifier
└─ <Outlet/> (page routes)
```
- **Mount point:** `client/src/components/Layout.tsx`, right next to `<UpdateNotifier/>`. This guarantees Tabby persists across every route and shares the one WebSocket connection.
- **Data source:** the existing `eventBus` (`client/src/lib/eventBus.ts`).
- `eventBus.subscribe(handler)` → every `WSMessage`.
- `eventBus.onConnection(handler)` + `eventBus.connected` → WS up/down.
- No prop drilling, no new context provider. The brain hook subscribes directly.
- **Navigation:** quick actions use `react-router` (`useNavigate`) to jump to existing routes (`/sessions`, `/sessions/:id`, `/activity`, `/run`).
### Component layout (new, isolated directory)
```
client/src/components/Tabby/
Tabby.tsx # Container. Owns open/collapsed/muted state, ⌘B + Esc handlers,
# localStorage persistence. Composes the three presentational parts.
CatAvatar.tsx # Pure presentational SVG cat. Props: { mood, eyeTarget, reducedMotion }.
# No data access — fully testable in isolation.
SpeechBubble.tsx # Transient bubble. Props: { text, onDismiss }. aria-live="polite",
# auto-dismiss ~4.5s. No data access.
TabbyPanel.tsx # Expanded panel: status header + quick actions + Ask box.
# Receives status summary + handlers as props.
useTabbyBrain.ts # The brain. Subscribes eventBus → derives { mood, statusSummary,
# bubbleQueue }. Owns all timers (idle/sleep/stuck). The only unit
# that touches eventBus.
intents.ts # Local Ask: maps a free-text question → templated answer from cached
# status, or a { runHandoff: prompt } signal. Pure function.
quips.ts # mood/event → randomized phrase pool. The personality. Pure data + picker.
tabby.css # Keyframes (breathe/blink/ear-twitch/arch/tail-flick), translucency,
# prefers-reduced-motion overrides.
```
**Boundaries / contracts:**
- `useTabbyBrain` is the *only* unit that subscribes to `eventBus`. Everything else receives plain props. This keeps the live-data surface in one place and the rest trivially testable.
- `CatAvatar`, `SpeechBubble`, `TabbyPanel` are pure presentational components — given props, render UI. No side effects.
- `intents.ts` and `quips.ts` are pure functions over inputs — unit-testable with no DOM.
---
## 4. Data flow
```
server broadcast ──► useWebSocket ──► eventBus.publish ──► useTabbyBrain subscriber
│
(reduce WSMessage + timers into state)│
▼
{ mood, statusSummary, bubbleQueue }
│
┌──────────────────────────────┬──────────────┴───────────────┐
▼ ▼ ▼
CatAvatar(mood) SpeechBubble(next bubble) TabbyPanel(statusSummary)
│
quick action │ Ask
▼
useNavigate(route) | intents() → answer
| or → /run?prompt=
```
`useTabbyBrain` maintains a small in-memory model derived from the stream (it does not refetch):
- `liveCount` — active sessions/agents currently working.
- `errorCount` — sessions/agents in error since last clear.
- `lastEventAt` — timestamp of most recent `new_event`/update (drives `stuck`/`sleeping`).
- `connected` — from `eventBus.onConnection`.
- `recentDone` — transient flag set on a `session_updated` → status `completed`, cleared after the happy animation.
The exact `WSMessage.type` union the brain switches on (from `client/src/lib/types.ts`):
`session_created`, `session_updated`, `agent_created`, `agent_updated`, `new_event`,
`import.progress`, `update_status`, `run_stream`, `run_status`, `run_input_ack`, `cc_config_changed`.
Tabby only cares about: `session_created`/`session_updated`/`agent_created`/`agent_updated` (mood + counts),
`new_event` (activity heartbeat → `lastEventAt`, and hook-failure detection via the event payload),
`run_status` (run finished → `happy`). The rest are ignored.
These feed both the avatar mood and the panel's status line. Counts are best-effort from the stream; the panel may also read a one-shot from existing stats endpoints if needed for an accurate initial number (open item — see §10).
---
## 5. Mood state machine (rule-based brain)
Mood is a pure function of `(streamModel, timers)`, evaluated on every relevant event and on timer ticks. **Highest-priority matching state wins:**
| Priority | Mood | Trigger | Cat expression |
|---------:|------|---------|----------------|
| 1 | `disconnected` | WS down (`eventBus.connected === false`) | faded/desaturated, flat ears, still |
| 2 | `worried` | `session_updated`/`agent_updated` with status `error`, or a hook-failure `new_event` | arch + puff, ears back, brow down, brief shake |
| 3 | `stuck` | ≥1 live session AND `now - lastEventAt > STUCK_MS` | ears-up alert stare, `!` |
| 4 | `happy` | `session_updated` → `completed`, or `run_status` finished (transient, ~4s) | tail-up, eyes `^^`, head-bob |
| 5 | `thinking` | Ask in flight (panel) | head-tilt, `…` |
| 6 | `watching` | ≥1 live session, recent activity | eyes track cursor, ears up, tail flick |
| 7 | `sleeping` | no activity AND idle `> SLEEP_MS` | curled, eyes shut, `zzz` |
| 8 | `idle` | default / fallback | slow blink, gentle breathe |
Constants (tunable, defined in `useTabbyBrain`): `STUCK_MS` (~10 min), `SLEEP_MS` (~3 min). All timers cleared on unmount.
**Event → mood mapping (concrete):**
- `onConnection(true)` → recompute (leaves `disconnected`).
- `onConnection(false)` → `disconnected`.
- `session_updated` data.status `error` → `worried` (+ increment `errorCount`).
- `session_updated` data.status `completed` → `happy` (transient) + decrement `liveCount`.
- `session_created` / `session_updated` data.status `active` → `watching`, recompute `liveCount`.
- `agent_updated` status `error` → `worried`.
- `new_event` → refresh `lastEventAt`; hook-failure event types (confirm in build, see §10) → `worried`.
- `run_status` finished → `happy` (transient).
- (timers) inactivity → `stuck` (if live) or `sleeping` (if not).
---
## 6. Eyes & motion
- **Eye tracking (`watching`/`idle`):** pupils follow the mouse, clamped inside the eye socket via a small vector-normalize + clamp. Throttled (rAF or ~30ms) to stay cheap.
- **On event:** eyes glance toward the bubble, then relax back to tracking.
- **Ears/tail/body:** CSS keyframe animations in `tabby.css`, swapped by a `data-mood` attribute on the avatar root.
- **`prefers-reduced-motion`:** static eyes (centered), no breathe/shake/arch — mood still conveyed via static pose + face. Detected via `matchMedia`, passed as `reducedMotion` prop.
---
## 7. Auto-surface (speech bubbles)
- Pipeline: event → `quips.pick(mood/event)` → enqueue bubble → show ~4.5s → dismiss → settle.
- **Rate limit:** at most one bubble every few seconds; coalesce bursts ("3 sessions finished" instead of three bubbles).
- **Mute toggle:** persisted in `localStorage`. Muted = no bubbles, but faces/animations still react. Toggle lives in the panel.
- **Accessibility:** bubble container is `aria-live="polite"` so screen readers announce notable events without stealing focus.
Example quips (from `quips.ts`, randomized):
- happy: "session wrapped 🐾", "nice, that one's done", "4m12s — clean run"
- worried: "ow, an error", "a hook tripped — peek?"
- stuck: "this one's been quiet a while…", "still chewing on something?"
- sleeping: "zzz", "wake me if something happens"
---
## 8. Panel (click / ⌘B)
Opens as a small card anchored above the avatar. Themed with `surface-3`/`border`/`accent`.
**Status header:** `🐾 N live · M errored · ●connected` (from brain's `statusSummary`; `●` reflects WS state, colored by health).
**Quick actions** (each = `useNavigate` to an existing route, or a local toggle):
- Jump to errored session → `/sessions/:id` (most recent error) or `/sessions?status=error`.
- Active sessions → `/sessions` (or `/activity`).
- **Run Claude** → `/run`.
- Activity feed → `/activity`.
- Mute / unmute bubbles (local toggle, persisted).
- Clear alerts (reset `errorCount`).
**Ask box:**
- P1/P2: `intents()` matches the query against a small set of local intents over cached status — e.g. *what's running*, *any errors*, *how many today*, *slowest* — and returns a templated answer rendered in the panel.
- Unmatched query → offer: "Ask Claude directly?" → opens `/run?prompt=<query>` (P3).
**Dismiss:** `Esc`, click-outside, or re-press `⌘B`.
---
## 9. Phasing
### P1 — Mascot (delight, zero backend)
- `CatAvatar.tsx` (full SVG + all moods + eye tracking + reduced-motion).
- `useTabbyBrain.ts` (eventBus subscription, mood machine, timers, bubble queue).
- `SpeechBubble.tsx`, `quips.ts`, `tabby.css`.
- `Tabby.tsx` container mounting avatar + bubble; `⌘B` reserved but panel stubbed.
- Mounted in `Layout.tsx`.
- **Outcome:** living, reacting cat in the corner with auto-bubbles. No panel yet.
### P2 — Panel (functional)
- `TabbyPanel.tsx`: status header + quick actions (router nav) + local Ask.
- `intents.ts` local intent matching.
- `localStorage` for `collapsed` + `muted`; mute/clear in panel.
- `Settings.tsx`: a single on/off toggle for Tabby (persisted), read by `Tabby.tsx`.
- **Outcome:** click/⌘B opens a useful panel; Ask answers from local data.
### P3 — "Do the job" handoff
- `Run.tsx`: read `?prompt=` search param → `setPrompt(prefill)` on mount (mirrors the existing `?session=` pattern). Client-only, no server change.
- Wire Ask's unmatched-query path → `/run?prompt=<query>`.
- **Outcome:** Tabby can hand a real question to a real `claude` subprocess via the existing Run flow.
---
## 10. Open items (resolve during planning/build)
1. **Accurate initial counts:** the stream gives deltas; on first mount counts are unknown until events arrive. Decide: (a) start at 0 and let the stream fill in (simplest), or (b) one-shot read from the existing stats endpoint for an accurate seed. Leaning (a) for P1, optional (b) in P2 panel.
2. **Hook-failure detection:** confirm which `event` `eventType` values represent hook failures vs. normal lifecycle, so `worried` only fires on real problems. Verify against `server/routes/hooks.js` + DB event types during build.
3. **Errored-session deep link:** confirm `/sessions` supports a `status=error` query or whether to navigate to the specific `/sessions/:id`.
---
## 11. Theme & accessibility notes
- Colors strictly from existing tokens: `surface-0..5`, `border`/`border-light`, `accent`/`accent-hover`. Cat palette: warm accent-tinted body that reads on the dark `surface-0` background; soft glow via `accent-muted`.
- Fonts inherit (`Inter` / `JetBrains Mono`) — bubble/status text uses existing classes.
- Keyboard: `⌘B`/`Ctrl+B` toggle, `Esc` close, panel actions tab-focusable.
- `prefers-reduced-motion`: disables continuous animation.
- z-index above content, below modals; never traps focus when collapsed.
---
## 12. Verification (per CLAUDE.md)
- **Frontend:** `npm run test:client`.
- Unit tests for `useTabbyBrain` mood transitions (each event → expected mood, priority ordering, timer-driven `stuck`/`sleeping`).
- Unit tests for `intents()` (known queries → templated answers; unknown → runHandoff).
- Unit test for `quips.pick` (returns a string for every mood).
- **No server change in P1/P2** → `npm run test:server` not required for those phases. P3 touches only `Run.tsx` (client) → still client-only; run `test:client`.
- Manual: load dashboard, trigger a run, observe mood/bubble transitions; toggle reduced-motion; toggle mute; ⌘B/Esc.
---
## 13. File change summary
**New:** `client/src/components/Tabby/{Tabby,CatAvatar,SpeechBubble,TabbyPanel}.tsx`, `client/src/components/Tabby/{useTabbyBrain.ts,intents.ts,quips.ts,tabby.css}`, plus `__tests__` for brain/intents/quips.
**Edited:** `client/src/components/Layout.tsx` (mount, P1) · `client/src/pages/Settings.tsx` (on/off toggle, P2) · `client/src/pages/Run.tsx` (`?prompt=` prefill, P3) · i18n files (`tabby:*` keys, as strings are added).
docs/superpowers/specs/2026-07-28-stage-detection-design.md
+74
View File
@@ -0,0 +1,74 @@
# Stage auto-detection — design
**Status:** approved 2026-07-28. Sub-project B of three (C = worktree lanes, shipped on `feat/worktree-lanes`; A = merged Workspace page, next). Built after C because C settled the lane data model.
## Problem
A lane's stage only moves when the driving agent calls `ccam stage <name>`. Every un-instrumented session — which is most of them — sits at `idle` forever while its agent works, so the pipeline map shows nothing. The dashboard already receives every tool call the agent makes; it just never reads them.
## Goal
Infer a lane's stage from the hook stream it already ingests, and show it **without ever claiming it as evidence**.
## The rule that shapes everything
**Inference never renders green.** A node the dashboard inferred reaches `passed-no-evidence` (amber) at most; `done` requires a declared stage carrying evidence via `ccam stage --evidence`. If inference could paint a node green, the amber/green distinction — the reason this feature exists — would be worthless.
Declared always outranks detected. A lane that has declared `review` ignores a detection for `implement`.
## Signals that actually exist here
Verified against a real 121 MB install before designing:
| Source | Rows | Notes |
|---|---|---|
| `events.tool_name` | Bash 29 470, Read 20 236, Edit 4 678, Write 1 330, Agent 1 188, TaskUpdate 908, Skill 127 | the bulk of the signal |
| `events.data.tool_input` | present on every `PostToolUse` | full Bash command strings, Edit/Write paths, Skill names |
| `TodoWrite` | **0** | this Claude Code build uses `TaskCreate`/`TaskUpdate` instead — do not design around TodoWrite |
So the signal is `tool_name` plus a regex over `tool_input`. No model call, no extra query.
## Where the rules live
In the pipeline template, not in code. `server/data/pipelines/default.json` gains an optional `detect` array per node:
```json
{ "id": "tests", "detect": [{ "tool": "Bash", "match": "\\b(npm (run )?test|pytest|vitest|jest|go test|cargo test)\\b" }] }
```
A rule is `{tool, match?}`: `tool` matches `events.tool_name` exactly; `match` is a regex tested against a flattened string of the tool's input. A rule with no `match` fires on the tool alone. A custom template supplied through `DASHBOARD_PIPELINES_DIR` may define its own rules, so a team can teach the dashboard their own conventions without touching the code.
## Where it runs
`server/lib/stage-detect.js` exposes one pure function, `detect(pipeline, event) → {nodeId, signal} | null`. It is called from the existing fail-safe block in `touchLaneFromHook` (`server/routes/hooks.js`) that already resolves the lane — no new pass over the hook path, no new query, and the same swallow-everything guarantee, because a hook must never fail on account of bookkeeping.
## Anti-flapping
- **Forward only.** A detection whose node index is not greater than the current detected index is dropped. Reading a file after editing it must not pull a lane back to `plan`.
- **Write only on change.** Bash alone accounts for 29 470 rows in a real install; the lane row is written only when the detected node actually advances.
- **Declared wins.** If the lane's declared stage sits at or beyond the detection, nothing is written.
## Data model
Three additive columns on `lanes`, each behind its own `try { SELECT col } catch { ALTER }` probe so a partial migration self-heals: `detected_stage`, `detected_signal`, `detected_at`. `stage` keeps its exact current meaning — the declared stage.
`lanePayload` gains `detected_stage`, `detected_signal`, and a per-node `detected: boolean` inside `pipeline_nodes`.
## What the user sees
A detected node renders amber with a **dashed** border, distinguishing it from an amber solid node (declared without evidence). Its tooltip names the signal: `tests ← npm run test:server`. The lane card shows `auto: tests` when the detection is ahead of the declaration. The header line still shows the declared stage, because that is what the agent asserted.
## Deliberately not in scope
- No inference of `done`, and no inference of gate results. A gate is a judgement; only an agent may claim one.
- No back-filling of history. Detection starts when this ships; existing lanes gain nothing retroactively.
- No inference from `workflows.phases`. It exists and would work, but it covers only Workflow-tool runs and would need its own reconciliation with the declared stage — a separate feature if wanted.
- No writing to `stage`. Ever. Detection lives in its own columns so that turning the feature off loses nothing.
## Testing
- Rule matcher: one test per shipped rule, plus a rule with no `match`, an invalid regex in a template (must be skipped, not crash the hook), and a tool the rules do not mention.
- Monotonic guard: an out-of-order detection is dropped; a same-node detection writes nothing.
- Declared precedence: a lane declared at `review` ignores an `implement` detection.
- Fail-safety: a malformed event cannot throw out of the hook path.
- **Inference never renders green:** given a lane with only detections and no declarations, no node in `pipeline_nodes` may have state `done`. This is the test that guards the feature's whole premise.
docs/superpowers/specs/2026-07-28-workspace-page-design.md
+59
View File
@@ -0,0 +1,59 @@
# Merged Workspace page — design
**Status:** approved 2026-07-28. Sub-project A of three (C = worktree lanes, shipped; B = stage detection, planned). Built last because it consumes both.
## Problem
Lanes and Run are two pages that describe the same activity. `Run` spawns a `claude` process in a directory and streams its output; `Lanes` shows what a lane is doing and where it is in its pipeline. A user watching an agent work has to hold both in their head, and the lane card cannot even send a prompt — `start` opens a promptless conversation run and `message` has no input field.
## Goal
One page. A lane strip across the top, the selected lane's pipeline beneath it, and that lane's Claude console below — with every capability the Run page has today.
## Decisions already taken
- **The merged page lives at `/run`.** `/lanes` redirects there. One sidebar entry.
- **Every run belongs to a lane.** Choosing a working directory that no lane owns creates one (`kind='adopted'`) rather than running loose. A lane is, after all, just a working directory the dashboard is watching.
- **Everything from Run survives:** slash-command autocomplete in the prompt editor, model / permission-mode / effort selectors, the token meter with cost, run history, and attach-to-a-live-run.
- **Layout:** lane strip (horizontal, scrollable, with the counters and Add) → pipeline map of the selected lane → console. Selecting a lane switches both the pipeline and the console.
## Architecture
`client/src/pages/Run.tsx` is 3658 lines holding an envelope model, a merge/typewriter engine, a slash-autocomplete prompt editor, a token meter, cwd suggestions, run history and the page shell. It is extracted into pieces that the new page composes:
| Unit | Responsibility |
|---|---|
| `client/src/hooks/useRunStream.ts` | envelope state for one run id: subscribe `run_stream` / `run_status` / `run_input_ack`, merge envelopes, typewriter |
| `client/src/components/run/RunConsole.tsx` | render the envelope stream, the prompt editor with slash autocomplete, the token meter, stop/clear |
| `client/src/components/run/RunSetup.tsx` | mode / model / permission / effort / cwd / resume pickers, binary status, the limitations banner |
| `client/src/components/run/RunHistory.tsx` | past runs, live runs, attach |
| `client/src/pages/Workspace.tsx` | lane strip + `PipelineMap` + the three above |
**The extraction is mechanical and must not change behaviour.** Each unit moves in its own commit with the existing Run tests passing untouched except for import paths. Only once `Run.tsx` is a thin composition does the new page get built. Extraction and composition never share a commit — that is the difference between a reviewable refactor and an unreviewable rewrite.
## Server glue
Four small pieces, each independently useful:
1. **Runs start through the lane.** The UI always calls `POST /api/lanes/:id/start`, which already exists, sits behind the same-origin guard, and records `run_id` on the lane. `POST /api/run` stays for the CLI and other callers; the UI simply stops using it. Lane `start` gains `mode` so a headless one-shot is still possible.
2. **`POST /api/lanes/ensure`** — `{cwd, title?}` returns the lane owning that path or creates an `adopted` one. Avoids the UI having to catch a 409 and re-read, and keeps the create-then-start pair from racing.
3. **`dashboard_runs.lane_id`** — one additive column, set when a run is started through a lane, so history can be filtered per lane instead of guessed at by `cwd`.
4. **A finished run releases its lane.** Today nothing clears `lanes.run_id` when a run ends on its own: the lane reads `running` forever and `message` keeps targeting a dead run. The run-spawner already knows the moment of exit (`actualExitedAt`, added on the worktree branch); on that event, clear the owning lane's `run_id` and set its status back to `idle`. This is a bug the merge exposes rather than causes.
## What the console must not do
**It never touches the lane's stage.** Typing `/code-review` in the UI does not move the lane to `review`; only `ccam stage` declares, and only detection (sub-project B) infers. The console is a window onto a process, not a driver of the pipeline. Keeping that boundary is what stops the pipeline from becoming a lie.
## Risks and how they are contained
- **The extraction is the whole risk.** 3658 lines, one of them the typewriter engine, with a screens snapshot over the page. Containment: one unit per commit, tests untouched but for imports, snapshot diffs read rather than regenerated, and the composition deferred until the last extraction is green.
- **Two consoles for one lane.** Only one run is live per lane (`start` 409s when one exists), so the console shows exactly one stream.
- **A lane created just to try a command** leaves an `adopted` lane behind. Acceptable: `adopted` lanes are never destroyable, forgetting one is a click, and the alternative — runs that belong to nothing — is what this design set out to remove.
## Testing
- Each extraction: the existing Run tests pass with only import changes, and the screens snapshot for `/run` is unchanged until the page itself changes.
- `useRunStream`: envelopes merge in order; a `run_status` terminal event stops the stream; the subscription is disposed on unmount.
- `POST /api/lanes/ensure`: returns the existing lane for a path already owned, for a path nested inside one, and creates exactly one lane under concurrent calls.
- Run-exit releases the lane: after a run ends by itself, the lane's `run_id` is null and its status is `idle`.
- The console does not move the stage: after a full run through the console, the lane's `stage` is what it was.
docs/superpowers/specs/2026-07-28-worktree-lanes-design.md
+133
View File
@@ -0,0 +1,133 @@
# Worktree-backed lanes + Shipyard-style lifecycle — design
**Status:** approved 2026-07-28. Sub-project C of three (B = stage auto-detection, A = merged Workspace page) — each gets its own spec, plan and execution cycle. C is being built first because it fixes the lane data model that the other two build on.
## Problem
A lane today is a pointer at a directory that already exists. Two agents working in parallel therefore work in the *same* checkout and collide — the exact failure Shipyard solves by giving every lane its own clone. CCAM has no provisioning at all: no way to create a lane's working copy, no way to reset it between features, no way to remove it, and no way to reclaim the database a finished lane leaves behind (a real install reached 121 MB).
## Goals
- A lane can own a **git worktree** that CCAM creates, resets and removes.
- The lifecycle verbs mirror Shipyard's, because that vocabulary is proven: `add`, `clear`, `reset`, `remove`, plus `purge` as CCAM's analogue of Shipyard's per-lane `dropdb`.
- Every destructive action is confirmed **against counted facts**, not adjectives.
- Directories the user already had must be impossible for CCAM to destroy.
## Non-goals
- No dependency bootstrap. A fresh worktree has no `node_modules`, no `.env`, no `.claude/settings.local.json` — all gitignored. Shipyard solves this with per-project `bootstrap`/`migrate`/`seed` hooks, which is a whole subsystem. Out of scope; documented as a limitation.
- No per-lane ports, databases, or Docker services.
- No orchestration. Unchanged from the existing feature: the driving Claude session declares its own stage.
- `VACUUM` is not part of `purge` (see Database reclamation).
## Data model
Additive columns on `lanes`, each guarded by the repo's `try { SELECT col } catch { ALTER }` probe:
| column | meaning |
|---|---|
| `kind` | `adopted` \| `managed`. Default `adopted`, so every pre-existing row migrates into the safe class. |
| `source_repo` | absolute path of the checkout a managed worktree was created from |
| `base_branch` | the branch the worktree was cut from, e.g. `development` |
| `slug` | sanitised from the title; used for both the directory and the branch name |
`branch`, `cwd`, `stage`, `stages` and the rest keep their current meaning. `cwd` stays `UNIQUE`.
Environment: `LANES_ROOT` (default `~/.claude/ccam-lanes`), `LANE_BASE_BRANCH`, `LANE_BRANCH_PREFIX` (default `feat/`).
Layout: worktree at `$LANES_ROOT/<repo-basename>__<slug>`, branch `<prefix><slug>`. Numbered `lane1..lane9` slots were considered and rejected — CCAM is multi-repo, and slot numbers carry no meaning without Shipyard's per-lane ports.
## Safety model
`adopted` lanes expose no destructive verb. No reset, no remove-with-worktree, no branch deletion. The UI hides those controls; the API refuses them.
A `managed` lane may be destroyed only when **all three** independent checks pass:
1. `kind === 'managed'`
2. the lane's `cwd`, fully resolved (symlinks included), lies inside `LANES_ROOT`
3. `git worktree list --porcelain` run in `source_repo` actually lists that path
Shipyard gets away with one check (`case "$DIR" in */lane$N`) because its directory names are fixed. Dropping numbered slots costs that guarantee, so three cheaper checks replace it. Every destructive function in `server/lib/worktree.js` re-runs the three checks itself rather than trusting its caller.
CCAM never runs `rm -rf` on a lane. Removal goes through `git worktree remove`; if git refuses, the error surfaces unchanged.
## Verbs
| verb | steps | destructive |
|---|---|---|
| `add` | resolve base → `git worktree add -b <prefix><slug> <dir> <base>` → insert lane row `kind=managed` | no |
| `adopt` | today's `POST /api/lanes` — point a lane at an existing directory, `kind=adopted` | no |
| `clear` | reset stage/status fields only (already implemented) | no |
| `reset` | kill and await the run → `git fetch origin --prune` → checkout base → `reset --hard <base>` → `clean -fd` → delete the feature branch → recreate it from base → clear lane state | **yes** |
| `remove` | kill and await the run → unlock if locked → `git worktree remove --force` → `git worktree prune` → delete the branch → delete the lane row | **yes** |
| `purge` | delete the lane's sessions and their events plus the orphan `token_usage` rows | **yes** |
`clean -fd` deliberately omits `-x`, exactly as Shipyard does: gitignored files (`node_modules`, `.env`) survive a reset, untracked-but-not-ignored files do not.
Branch deletion never touches `main`, `master`, or the lane's `base_branch`, and only runs after the worktree holding that branch is gone.
## Preflight
`GET /api/lanes/:id/preflight?action=reset|remove|purge` returns counted facts, never prose:
```json
{
"action": "reset",
"lane": 3, "branch": "feat/criteria-form", "kind": "managed",
"dirty": 4, "untracked": 11, "unpushed": 2,
"head": "9b3e74a",
"blocked": ["unpushed-commits"],
"warnings": ["no-remote"]
}
```
For `purge`: `sessions`, `events`, `tokenRows`, `bytesEstimate`, and `activeSessionSkipped`.
The confirmation modal renders those numbers. The action then re-verifies: the client echoes back the `head` and counts it was shown, and the server returns `409` if they moved. `unpushed > 0` blocks `reset`, and `remove` when a managed worktree is actually at risk, unless the request carries `{force: true}`. `unpushed` counts what the action would really discard — with no remote that is `<base_branch>..HEAD`, this lane's own work, not the repository's whole history.
## Concurrency
One mutex per lane serialises destructive actions, mirroring `repo_lock()` in the AutomaticWorkflow bot — where concurrent git operations on a shared checkout produced a real `git checkout` exit 128, not a theoretical one. A destructive action first kills the lane's run and **awaits its exit** before touching git.
`add` can take seconds on a large repo, so it returns `202` with `status=provisioning` and finishes in the background, broadcasting `lane_update` on completion — the same pattern Shipyard uses to drive its spinner.
## Edge cases and their resolutions
- **Branch already exists:** if unused, `git worktree add` without `-b`; if checked out elsewhere, refuse and name the other path. Slug collisions get a `-2` suffix.
- **Base branch missing on origin:** resolve `origin/<base>` → local `<base>` → the source repo's current HEAD.
- **Source repo is itself a worktree, or bare:** works; git resolves through `--git-common-dir`.
- **Repo with no commits:** `worktree add` fails and the lane lands `failed` with git's stderr in `notes`. Preflight does not pre-empt it — the lane is created first, then provisioning reports the git failure, and the row is forgotten with `DELETE /api/lanes/:id`.
- **Worktree directory deleted by hand:** the lane reports `missing` and only `remove` is offered, taking the prune path.
- **`cwd` uniqueness:** `remove` deletes the row, so re-adding the same slug is clean; `reset` keeps the path.
## Database reclamation
`events` cascades from `sessions`, but `token_usage` has no foreign key — `purge` must delete those rows explicitly or leave orphans. The currently-live session is never purged.
SQLite does not shrink on `DELETE`. `purge` runs `DELETE` plus `PRAGMA optimize` and reports the reclaimable size; `VACUUM` is a separate, explicitly-labelled maintenance action because it locks the whole database for seconds. Hiding a database-wide lock inside a button labelled "clean up" would be a trap.
## Surfaces
**API:** `POST /api/lanes/worktree` (add), `GET /api/lanes/:id/preflight`, and `reset` / `purge` joining the existing `POST /api/lanes/:id/:action` set, all behind the existing same-origin guard.
**CLI:** `ccam lanes add --repo <path> [--title <t>] [--base <branch>]`, `ccam lanes reset|remove|purge <id> [--force]`.
**UI:** a `managed` / `adopted` badge on the lane card; destructive buttons rendered only for `managed`; the existing `ConfirmModal` showing the preflight table.
## Testing
Against a real git repository fixture created in a temp directory — no mocks, because every bug worth catching here lives in git's actual behaviour:
- worktree created, listed, removed, pruned clean
- each of the three safety refusals: an `adopted` lane, a path outside `LANES_ROOT`, a path git does not list as a worktree
- `reset` keeps gitignored files and removes untracked ones
- the unpushed-commit guard blocks, and `force` overrides it
- preflight's counts equal what the action actually changes
- `purge` removes sessions, events and token rows, and skips the live session
- `add` returns 202 and broadcasts `lane_update` when provisioning finishes
## Known limitations
- A fresh worktree has no installed dependencies or local env files (see Non-goals).
- Nine worktrees of a large repository cost nine working trees of disk; the `add` preflight estimates the size first.
- `--repo` may point anywhere the user can read. That is their own machine; validation is limited to "absolute, exists, is a git repo", and every route stays behind the loopback guard.
docs/superpowers/specs/2026-07-29-workspace-ui-design.md
+140
View File
@@ -0,0 +1,140 @@
# Workspace UI rebuild — design
**Status:** approved 2026-07-29. Sub-project D, built on top of A (merged Workspace page).
Reference: the Shipyard "Feature Harness" screen the user supplied.
## Problem
The merged Workspace page shipped with the right information and the wrong shape.
Everything a lane knows — declared stage, inferred stage, progress, liveness,
needs-you — is already on the card (`client/src/components/lanes/LaneCard.tsx`)
and already correct on the wire. None of it is legible: the lane strip is a
horizontal scroller of cramped cards, the pipeline sits above a console that
dominates the viewport, and the `auto: <stage>` chip that proves detection works
is 10px of amber text nobody sees.
Measured on the live install while writing this: lane 5 carried
`detected_stage: "tests"` with a real signal, and the user's report was
"the lane does not auto-detect". Detection was never broken. The display was.
Two facts also make lanes look emptier than they are:
- `branch` and `ci_status` are columns nobody writes, so those rows are always
blank even for a managed worktree sitting on a real branch.
- Detection is forward-only with no expiry, so a lane parks at the highest stage
it ever touched. Lane 5 reached `tests` and can never show `implement` again,
even while the agent is editing code.
## Goal
The reference screen's legibility, on CCAM's real data: a lane's state readable
from across the room, the pipeline large enough to trace, and the console present
but out of the way until wanted.
## Decisions taken
- **Card grid, not a strip.** Responsive 1 / 2 / 3 columns.
- **The console collapses.** It keeps every capability from A; it starts
collapsed and opens for the selected lane. Watching lanes is the default
posture, driving one is the exception.
- **Only real data.** No placeholder tiles for facts CCAM does not have
(tickets, preview ports, per-lane credentials). Branch/commit/CI are added
because they can be read for real — see below.
- **Detection expires.** A detection older than a TTL stops holding the floor.
## Layout
Top to bottom, one column:
```
header: title · [N lanes][N running][N need you][N dead] · [+ Add lane]
detail: selected lane · declared + inferred headline · large PipelineMap · legend
console: collapsed by default; expands to RunSetup + RunConsole + RunHistory
grid: lane cards, 1/2/3 columns
```
Selecting a card switches the detail panel and the console together, exactly as
A wired it. The console is unchanged behind its new disclosure — no prop of
`RunConsole`, `RunSetup` or `RunHistory` moves.
## The card
Reference layout, CCAM's fields, nothing invented:
| Row | Content | Source |
|---|---|---|
| header | `LANE <id>` · liveness dot · status | `id`, `liveness`, `status` |
| title | title, falling back to `cwd` | existing |
| progress | declared stage chip · bar · `%` · time on stage | `stage`, `progress`, `stage_seconds` |
| inferred | dashed amber `auto: <stage>` with the signal as tooltip | `detected_stage`, `detected_signal` |
| tags | `kind` (adopted/managed), CI when known | `kind`, `ci_status` |
| git | branch · short head · last commit subject · dirty/untracked counts | new, see below |
| alert | needs-you banner | `needs_action` |
| actions | start · stop · clear · reset · remove | existing lane actions |
`reset` and `remove` keep their preflight + `expect` echo through
`DestructiveLaneModal`. This redesign does not touch the destroy guard.
## Git facts
A new read-only endpoint, `GET /api/lanes/:id/git`, returning
`{branch, head, subject, dirty, untracked}` or `{available: false}` when the
lane's `cwd` is not a git repo or is unreadable.
Deliberately **not** folded into `GET /api/lanes`: that payload is polled and
broadcast, and shelling out to git once per lane on the hot path would put a
subprocess burst behind every hook-driven `lane_update`. The card fetches its
own facts when it mounts and on a slow interval, and renders without them until
they arrive.
`server/lib/worktree.js` already has `statusCounts(dir)` returning
`{dirty, untracked, head}` and a `git()` wrapper that scrubs the inherited
`GIT_*` environment. Both are reused as-is; the endpoint adds only the branch
name and the commit subject. No second git helper, no shell strings.
## Detection expiry
`recordDetection` gains one rule: a `detected_stage` whose `detected_at` is
older than `DETECTION_TTL_MS` (default 30 minutes) no longer blocks a new
detection — the forward-only comparison is skipped and the fresh signal wins.
Within the window nothing changes: forward-only and declared-wins hold exactly
as they do today.
This keeps the anti-flapping property that motivated forward-only (a `Read`
right after an `Edit` must not drag the lane backwards) while admitting the
thing it got wrong: a work session ends, and the next one starts somewhere else
in the pipeline.
**Unchanged, and not negotiable:** detection still never writes `lanes.stage`,
and an inferred node still never renders `done`.
## Signal legibility
`detected_signal` currently captures the whole flattened tool input, so the chip's
tooltip reads `cd /very/long/path && npm run test:server 2>&1 | grep …`. The
matcher already knows which regex fired; the signal becomes the matched span plus
a little context rather than the entire command. Cosmetic, but it is the text the
tooltip exists to show.
## Risks
- **The console's disclosure is the only structural risk.** Mounting it inside a
collapsed container must not unmount `useRunStream` and lose a live stream.
The subscription stays mounted; only the visual container collapses.
- **Git calls per card.** Bounded by the number of lanes on screen and a slow
refresh; failure is silent and the card renders without those rows.
- **The screens snapshot over `/run` will change.** It is read, not regenerated
blindly.
## Testing
- The card renders every field from a fixture lane, and renders without the git
block when the endpoint reports `available: false`.
- A detected node still never carries `data-state="done"` — the premise guard
from sub-project B is re-asserted at the new layout.
- Collapsing and expanding the console does not tear down the run subscription:
a stream envelope delivered while collapsed is present when it re-expands.
- `GET /api/lanes/:id/git` returns the facts for a real repo fixture and
`available: false` for a plain directory, and never shells out through a shell.
- A detection older than the TTL is accepted even when it is behind the current
`detected_stage`; one inside the window is still refused.