feat: Claude Code Monitor — lanes, pipelines and a merged workspace
Internal SmartGift build of a Claude Code monitoring dashboard. Lanes: a durable unit of parallel agent work, one per working directory, tracked across session restarts. Managed lanes are git worktrees the dashboard provisions and can reset or remove behind a three-check destroy guard and a counted preflight; adopted lanes are directories you already own and are never destroyable. Pipelines: a lane moves through pipeline stages. A stage the agent declares with evidence renders green; a stage inferred from the tool-event stream renders dashed amber and never counts as done. Detection is forward-only within a 30-minute window, and never writes the declared stage. Workspace: one page at /run with a lane grid, the selected lane's pipeline, and a full Claude console behind a disclosure.
This commit is contained in:
@@ -0,0 +1,82 @@
|
||||
---
|
||||
description: >
|
||||
Detect quality and efficiency regressions over time using Agent Monitor data —
|
||||
rising error rate (APIError events), falling cache hit rate, growing compaction
|
||||
frequency, and climbing cost-per-session. Splits history into an earlier
|
||||
baseline window and a recent window and reports which metrics are getting
|
||||
worse, by how much, and where. Use when checking whether things are degrading
|
||||
or trending in the wrong direction.
|
||||
---
|
||||
|
||||
# Regression Watch
|
||||
|
||||
Detect whether Claude Code sessions are getting worse over time across quality and
|
||||
efficiency metrics, using Agent Monitor data.
|
||||
|
||||
## Input
|
||||
|
||||
The user provides: **$ARGUMENTS**
|
||||
|
||||
This may be:
|
||||
- empty or "all" — check every regression metric (default)
|
||||
- "errors" — error-rate regression only
|
||||
- "cache" — cache hit-rate regression only
|
||||
- "compaction" — compaction-frequency regression only
|
||||
- "cost" — cost-per-session regression only
|
||||
- A window like "last 30d" or "30 vs 90" — set the recent vs baseline window sizes
|
||||
|
||||
## Data Sources
|
||||
|
||||
| Endpoint | Returns |
|
||||
|----------|---------|
|
||||
| `GET /api/analytics` | `daily_events` (365d), `daily_sessions` (365d), `event_types`, `tokens` (total_input, total_output, total_cache_read, total_cache_write — baselines pre-summed), `avg_events_per_session` |
|
||||
| `GET /api/events?session_id=X` | Event stream incl. `APIError`, `Compaction`, `PreToolUse`/`PostToolUse` — used to localize regressions to specific sessions |
|
||||
| `GET /api/pricing/cost` | `{ total_cost, breakdown[...] }` — total cost to derive cost-per-session |
|
||||
| `GET /api/pricing/cost/{sessionId}` | Per-session cost — used to compare recent vs baseline session cost |
|
||||
| `GET /api/workflows/{sessionId}` | `compaction` (impact), `errorPropagation` (by depth), `effectiveness` — per-session quality signals |
|
||||
| `GET /api/sessions?limit=N` | Sessions with `started_at`, `cost`, `metadata` — to bucket sessions into time windows |
|
||||
|
||||
## Report Sections
|
||||
|
||||
### 1. Windowing
|
||||
Split history into a **baseline window** (older) and a **recent window** (newer).
|
||||
Default: recent = last 30 days, baseline = the 30–90 day range before it. Use
|
||||
`daily_events`/`daily_sessions` for series metrics and `GET /api/sessions?limit=N`
|
||||
to assign sessions to each window by `started_at`.
|
||||
|
||||
### 2. Error Rate Regression
|
||||
- Recent error rate = `APIError count / total events` in the recent window
|
||||
(from `event_types` and `daily_events`, or per-session `GET /api/events`).
|
||||
- Compare to the baseline rate. Flag if recent is higher.
|
||||
- Report the absolute and relative change and which sessions contributed most
|
||||
`APIError` events.
|
||||
|
||||
### 3. Cache Hit Rate Regression
|
||||
- Cache hit rate = `total_cache_read / (total_cache_read + total_input)`.
|
||||
- Compute for each window (per-window input/cache_read from session metadata or
|
||||
the pricing breakdown). Flag a **falling** hit rate — that means more
|
||||
uncached input tokens and higher cost.
|
||||
|
||||
### 4. Compaction Frequency Regression
|
||||
- Compaction frequency = `Compaction events / session` per window (from
|
||||
`event_types` / `daily_events`, confirmed via per-session
|
||||
`GET /api/workflows/{id}` `compaction`). Flag a **rising** rate — context is
|
||||
overflowing more often.
|
||||
|
||||
### 5. Cost-per-Session Regression
|
||||
- Cost-per-session = window total cost / window session count, using
|
||||
`GET /api/pricing/cost` overall and `GET /api/pricing/cost/{id}` for the
|
||||
sessions in each window. Flag a **climbing** value.
|
||||
|
||||
### 6. Verdict
|
||||
Roll up which metrics regressed, rank by relative worsening, and name the most
|
||||
likely driver (e.g., cache hit rate fell → cost per session climbed).
|
||||
|
||||
## Output
|
||||
|
||||
- A Markdown table: metric | baseline | recent | Δ | direction (▲ worse / ▼ better) | verdict.
|
||||
- Tag each regressed metric 🔴 (clear regression), 🟡 (mild/within noise), or 🟢 (improved).
|
||||
- Currency in USD to 4 decimals; rates as percentages to 2 decimals.
|
||||
- List the specific session IDs that contributed most to any regression.
|
||||
- End with the single highest-priority regression to address and a concrete next step.
|
||||
- Read-only: only report what the API returns; never fabricate baselines.
|
||||
Reference in New Issue
Block a user