feat: Claude Code Monitor — lanes, pipelines and a merged workspace
Internal SmartGift build of a Claude Code monitoring dashboard. Lanes: a durable unit of parallel agent work, one per working directory, tracked across session restarts. Managed lanes are git worktrees the dashboard provisions and can reset or remove behind a three-check destroy guard and a counted preflight; adopted lanes are directories you already own and are never destroyable. Pipelines: a lane moves through pipeline stages. A stage the agent declares with evidence renders green; a stage inferred from the tool-event stream renders dashed amber and never counts as done. Detection is forward-only within a 30-minute window, and never writes the declared stage. Workspace: one page at /run with a lane grid, the selected lane's pipeline, and a full Claude console behind a disclosure.
This commit is contained in:
@@ -0,0 +1,57 @@
|
||||
---
|
||||
description: >
|
||||
Produce a detailed report on APIError events from Agent Monitor data — counts
|
||||
over time, which sessions and models are affected, and the likely root cause
|
||||
(rate limits, overload/529, or context-window pressure) inferred from each
|
||||
event's summary and data payload. Use when API errors spike or when you need to
|
||||
explain why requests are failing.
|
||||
---
|
||||
|
||||
# API Error Report
|
||||
|
||||
Drill into `APIError` events: how many, when, where, and most likely why.
|
||||
|
||||
## Input
|
||||
|
||||
The user provides: **$ARGUMENTS**
|
||||
|
||||
This may be:
|
||||
- empty or "all" — report on every APIError in the recent window (default)
|
||||
- a session ID — report APIErrors for that one session only
|
||||
- a window like "today" or "last 7d" — restrict the time range
|
||||
- a cause filter: "rate-limit", "overload", or "context"
|
||||
|
||||
## Data Sources
|
||||
|
||||
| Endpoint | Returns |
|
||||
|----------|---------|
|
||||
| `GET /api/analytics` | `event_types` (total `APIError` count), `daily_events` (365d) — APIError volume and trend over time |
|
||||
| `GET /api/events?session_id=X` | Per-session event stream — each `APIError` carries `summary`, `data`, and `timestamp` used to classify the cause |
|
||||
| `GET /api/sessions?limit=N` | Sessions with `id`, `model`, `started_at` — attribute each error to a model and place it on the timeline |
|
||||
|
||||
## Report Sections
|
||||
|
||||
### 1. Volume & Trend
|
||||
From `GET /api/analytics`: total `APIError` count and its share of `total_events`. Use `daily_events` to chart APIErrors over the requested window and flag any day that spikes above the window mean.
|
||||
|
||||
### 2. Affected Sessions & Models
|
||||
For each session in scope, pull `GET /api/events?session_id=X` and collect `APIError` events. Group by `session_id` and, via `GET /api/sessions`, by `model`. Report the top affected sessions and which model accounts for the most errors.
|
||||
|
||||
### 3. Likely Cause Classification
|
||||
Inspect each error's `summary`/`data` and bucket it:
|
||||
- **Rate limit** — mentions 429, "rate limit", "quota", or retry-after.
|
||||
- **Overload** — mentions 529, "overloaded", or capacity.
|
||||
- **Context** — mentions context length, token limit, or "too long" (correlate with nearby `Compaction` events).
|
||||
- **Other** — anything else; quote the `summary`.
|
||||
Report the count and percentage in each bucket.
|
||||
|
||||
### 4. Timeline
|
||||
List the most recent APIErrors with `timestamp`, `session_id`, `model`, classified cause, and a one-line `summary` excerpt.
|
||||
|
||||
## Output
|
||||
|
||||
- A Markdown table per section (volume, by model, by cause).
|
||||
- Rates as percentages to 2 decimals; any currency in USD to 4 decimals.
|
||||
- Cite exact `session_id`, `model`, `timestamp`, and `summary` values — never invent a cause not supported by the payload; bucket as "Other" when unclear.
|
||||
- End with the dominant cause and a concrete mitigation (e.g., back off and retry on 529, reduce context to cut context errors, slow request rate on 429).
|
||||
- Read-only: only report what the API returns. If `curl` cannot reach `http://localhost:4820`, tell the user to start the dashboard with `npm start` from the repo root.
|
||||
@@ -0,0 +1,57 @@
|
||||
---
|
||||
description: >
|
||||
Scan recent Claude Code activity for errors and failure signals across all
|
||||
sessions using Agent Monitor data — APIError events and PreToolUse→PostToolUse
|
||||
gaps (tools that started but never completed) — then group failures by tool and
|
||||
model and rank them by frequency. Use when checking for errors or asking
|
||||
"what's failing right now".
|
||||
---
|
||||
|
||||
# Error Scan
|
||||
|
||||
Sweep recent events across sessions for error and failure signals, then rank them
|
||||
by how often they occur and which tool or model produced them.
|
||||
|
||||
## Input
|
||||
|
||||
The user provides: **$ARGUMENTS**
|
||||
|
||||
This may be:
|
||||
- empty or "all" — scan every failure signal (default)
|
||||
- "api" — APIError events only
|
||||
- "tools" — tool-failure gaps only
|
||||
- a number N — limit the scan to the most recent N sessions
|
||||
- a session ID — scan a single session
|
||||
|
||||
## Data Sources
|
||||
|
||||
| Endpoint | Returns |
|
||||
|----------|---------|
|
||||
| `GET /api/analytics` | `event_types` (counts per type incl. PreToolUse, PostToolUse, APIError), `tool_usage` (top 20), `daily_events` (365d) — fleet-wide failure baseline |
|
||||
| `GET /api/events?session_id=X` | Per-session event stream: `event_type`, `tool_name`, `summary`, `data`, `timestamp` — locate `APIError` and unmatched `PreToolUse` |
|
||||
| `GET /api/sessions?limit=N` | Sessions with `id`, `status`, `model`, `started_at` — pick the recent window and attribute failures to a model |
|
||||
|
||||
## Report Sections
|
||||
|
||||
### 1. Scope
|
||||
Resolve `$ARGUMENTS` to a session set: pull `GET /api/sessions?limit=N` (default 50, ordered by `started_at`). Report how many sessions and what time span are covered.
|
||||
|
||||
### 2. Fleet Failure Counts
|
||||
From `GET /api/analytics` `event_types`, report total `APIError` count and the PreToolUse→PostToolUse gap: `gap = PreToolUse − PostToolUse` (unmatched tool starts = likely failures). State both as raw counts and as a share of `total_events`.
|
||||
|
||||
### 3. Group by Tool
|
||||
For each session in scope, pull `GET /api/events?session_id=X`. Match each `PreToolUse` to its following `PostToolUse` by `tool_name`; unmatched starts are failures. Aggregate failures and `APIError` events per `tool_name`. Rank tools by failure frequency (descending).
|
||||
|
||||
### 4. Group by Model
|
||||
Join failures to the owning session's `model` (from `GET /api/sessions`). Rank models by APIError count and tool-failure count.
|
||||
|
||||
### 5. Top Offenders
|
||||
List the single most failure-prone tool, the most error-prone model, and the session with the most failures, each with its exact count and one-line `summary` excerpt from a representative event.
|
||||
|
||||
## Output
|
||||
|
||||
- A ranked Markdown table: tool/model | APIError count | tool-failure (gap) count | total failures | share of events.
|
||||
- Rates as percentages to 2 decimals.
|
||||
- Cite exact `event_type`, `tool_name`, and `session_id` values — never fabricate counts.
|
||||
- End with the one failure pattern most worth investigating and a concrete next step.
|
||||
- Read-only: only report what the API returns. If `curl` cannot reach `http://localhost:4820`, tell the user to start the dashboard with `npm start` from the repo root.
|
||||
@@ -0,0 +1,54 @@
|
||||
---
|
||||
description: >
|
||||
Audit hook delivery health from Agent Monitor data — balance PreToolUse vs
|
||||
PostToolUse (a gap means tools that started but never reported back), detect
|
||||
missing Stop/SubagentStop terminators (sessions/subagents that never closed),
|
||||
and check for stale ingestion (no recent events). Use when hooks look
|
||||
unreliable or events seem to be dropping.
|
||||
---
|
||||
|
||||
# Hook Failure Audit
|
||||
|
||||
Assess whether the hook pipeline is delivering events reliably, using the event
|
||||
counts and stream the dashboard already has. This is about *delivery* health
|
||||
(missing/dropped events), not about why a model errored.
|
||||
|
||||
## Input
|
||||
|
||||
The user provides: **$ARGUMENTS**
|
||||
|
||||
This may be:
|
||||
- empty or "all" — run every check (default)
|
||||
- "balance" — PreToolUse/PostToolUse balance only
|
||||
- "terminators" — missing Stop/SubagentStop only
|
||||
- "freshness" — stale-ingestion check only
|
||||
|
||||
## Data Sources
|
||||
|
||||
| Endpoint | Returns |
|
||||
|----------|---------|
|
||||
| `GET /api/analytics` | `event_types` (counts per type: PreToolUse, PostToolUse, Stop, SubagentStop, SessionStart, SessionEnd), `daily_events` (365d), `total_subagents`, `sessions_by_status` — fleet-wide delivery balance |
|
||||
| `GET /api/stats` | `total_sessions`, `total_agents`, `total_events`, `events_today` — expected terminator counts and recency |
|
||||
| `GET /api/events?session_id=X` | Per-session stream — confirm which sessions are missing a `PostToolUse`, `Stop`, or `SubagentStop` |
|
||||
|
||||
## Report Sections
|
||||
|
||||
### 1. PreToolUse / PostToolUse Balance
|
||||
From `GET /api/analytics` `event_types`: `gap = PreToolUse − PostToolUse`. A positive gap means tools whose completion hook never arrived. Report the gap as a count and as a percentage of `PreToolUse`. A healthy pipeline keeps this near 0%.
|
||||
|
||||
### 2. Missing Terminators
|
||||
Compare `Stop` count against completed sessions (`sessions_by_status`) and `SubagentStop` against `total_subagents`/`total_agents` (from `/api/stats`). A shortfall means sessions or subagents that ran but never emitted a closing hook — likely dropped delivery or a crashed handler. Report expected vs observed for each.
|
||||
|
||||
### 3. Stale Ingestion
|
||||
Check `events_today` from `/api/stats` and the tail of `daily_events` from analytics. If recent days are empty while sessions exist, ingestion has stalled. Report the most recent day with events and how long ago that was.
|
||||
|
||||
### 4. Localize
|
||||
For the sessions with the largest gaps or missing terminators, pull `GET /api/events?session_id=X` and confirm which specific hook types are absent. List the offending session IDs.
|
||||
|
||||
## Output
|
||||
|
||||
- A check-by-check report with a PASS / WARN / FAIL marker each (✅ / ⚠️ / ❌) and the expected-vs-observed numbers.
|
||||
- Rates as percentages to 2 decimals.
|
||||
- Cite exact `event_type` counts and `session_id` values — never fabricate.
|
||||
- End with an overall verdict (e.g., "4/4 checks passed" or "hook delivery DEGRADED") and the single highest-impact remediation (e.g., reinstall hooks via the dashboard Settings, or restart the server with `npm start`).
|
||||
- Read-only: only report what the API returns. If `curl` cannot reach `http://localhost:4820`, tell the user to start the dashboard with `npm start` from the repo root.
|
||||
@@ -0,0 +1,63 @@
|
||||
---
|
||||
description: >
|
||||
Compare this period's reliability against the prior period using Agent Monitor
|
||||
data — error rate (APIError/total) and tool-failure rate (PreToolUse→PostToolUse
|
||||
gap) — flag any regression where reliability got worse, and optionally wire a
|
||||
persistent alert rule so the dashboard catches the next regression
|
||||
automatically. Use when checking whether reliability degraded.
|
||||
---
|
||||
|
||||
# Regression Alert
|
||||
|
||||
Detect whether Claude Code reliability is getting worse period-over-period, and
|
||||
optionally arm an alert so it never has to be checked by hand again. Scope is
|
||||
reliability/failures only — for cache/cost/compaction drift, use ccam-insights'
|
||||
`regression-watch` instead.
|
||||
|
||||
## Input
|
||||
|
||||
The user provides: **$ARGUMENTS**
|
||||
|
||||
This may be:
|
||||
- empty or "all" — check error rate and tool-failure rate (default)
|
||||
- "errors" — APIError-rate regression only
|
||||
- "tools" — tool-failure-rate regression only
|
||||
- a window like "7 vs 7" or "30 vs 30" — recent vs baseline window sizes (default: last 7 days vs the prior 7)
|
||||
- "arm" — after reporting, also create an alert rule via `POST /api/alerts/rules` (only on explicit request)
|
||||
|
||||
## Data Sources
|
||||
|
||||
| Endpoint | Returns |
|
||||
|----------|---------|
|
||||
| `GET /api/analytics` | `daily_events` (365d), `daily_sessions` (365d), `event_types` — split into recent vs baseline windows to compute per-window failure rates |
|
||||
| `GET /api/events?session_id=X` | Per-session stream — localize a regression to the sessions driving it |
|
||||
| `GET /api/alerts/rules` | Existing alert rules — check whether a matching reliability rule already exists before arming a new one |
|
||||
| `POST /api/alerts/rules` | Create a new alert rule (only when the user says "arm") |
|
||||
|
||||
## Report Sections
|
||||
|
||||
### 1. Windowing
|
||||
Split history into a **recent window** (newer) and a **baseline window** (the equal-length period just before it). Default: recent = last 7 days, baseline = the prior 7. Use `daily_events`/`daily_sessions` to bucket counts by day.
|
||||
|
||||
### 2. Error-Rate Regression
|
||||
- Per window: `error rate = APIError count / total events`.
|
||||
- Compare recent vs baseline. Flag if recent is higher. Report absolute change (pp) and relative change (%), plus the recent sessions contributing the most `APIError` events.
|
||||
|
||||
### 3. Tool-Failure-Rate Regression
|
||||
- Per window: `tool-failure rate = (PreToolUse − PostToolUse) / PreToolUse`.
|
||||
- Compare recent vs baseline. Flag a rising rate as a reliability regression. Name the tools whose gap grew most.
|
||||
|
||||
### 4. Verdict
|
||||
Roll up which rates regressed, rank by relative worsening, and name the most likely driver.
|
||||
|
||||
### 5. Optional — Arm an Alert
|
||||
**Only if the user passed "arm".** First `GET /api/alerts/rules` to avoid duplicates. Then `POST /api/alerts/rules` with a rule that fires when the regressed metric crosses a threshold near the recent value (e.g., error rate > recent rate). Echo the created rule back; do not create webhooks or fire alerts.
|
||||
|
||||
## Output
|
||||
|
||||
- A Markdown table: metric | baseline | recent | Δ (pp) | Δ (%) | direction (▲ worse / ▼ better) | verdict.
|
||||
- Tag each metric 🔴 (clear regression), 🟡 (within noise), or 🟢 (improved).
|
||||
- Rates as percentages to 2 decimals; any currency in USD to 4 decimals.
|
||||
- List the specific session IDs that contributed most to any regression.
|
||||
- End with the single highest-priority regression and a concrete next step (and, if armed, the new rule's id/threshold).
|
||||
- Read-only **except** the explicit "arm" path, which is the only write. Never mutate alert rules otherwise. If `curl` cannot reach `http://localhost:4820`, tell the user to start the dashboard with `npm start` from the repo root.
|
||||
@@ -0,0 +1,61 @@
|
||||
---
|
||||
description: >
|
||||
Define and check simple service-level objectives for Claude Code from Agent
|
||||
Monitor data — session completion rate, tool success rate
|
||||
(PostToolUse/PreToolUse), and error rate (APIError/total) — then compare each
|
||||
to its target and report the error budget remaining. Use when reporting
|
||||
reliability or when someone asks "are we meeting our SLOs?".
|
||||
---
|
||||
|
||||
# SLO Check
|
||||
|
||||
Turn raw event counts into a clear SLO scorecard with error-budget accounting.
|
||||
|
||||
## Input
|
||||
|
||||
The user provides: **$ARGUMENTS**
|
||||
|
||||
This may be:
|
||||
- empty — use the default SLO targets below over all available data
|
||||
- targets like "completion=95 success=99 error=1" — override the defaults (percentages)
|
||||
- a window like "last 7d" or "today" — restrict the measurement period
|
||||
|
||||
Default SLO targets: completion rate ≥ 95%, tool success rate ≥ 99%, error rate ≤ 1%.
|
||||
|
||||
## Data Sources
|
||||
|
||||
| Endpoint | Returns |
|
||||
|----------|---------|
|
||||
| `GET /api/analytics` | `event_types` (PreToolUse, PostToolUse, APIError counts), `sessions_by_status`, `daily_events` (365d), `daily_sessions` (365d) — the raw numerators/denominators for every SLI |
|
||||
| `GET /api/stats` | `total_sessions`, `total_events`, `events_today`, `sessions_by_status` — fleet totals and recency |
|
||||
| `GET /api/events?session_id=X` | Per-session stream — drill into the sessions that breach an SLO |
|
||||
|
||||
## Report Sections
|
||||
|
||||
### 1. Service Level Indicators (SLIs)
|
||||
Compute each SLI from `GET /api/analytics` / `GET /api/stats`:
|
||||
- **Completion rate** = completed sessions / total sessions (from `sessions_by_status`; count `active`/`running` as in-flight, exclude them from the denominator if still open).
|
||||
- **Tool success rate** = `PostToolUse / PreToolUse` (capped at 100%).
|
||||
- **Error rate** = `APIError / total_events`.
|
||||
Within a window, derive the numerators/denominators from `daily_events` / `daily_sessions`.
|
||||
|
||||
### 2. SLO Scorecard
|
||||
For each SLI, compare to its target and mark MET ✅ or BREACHED ❌.
|
||||
|
||||
### 3. Error Budget
|
||||
For each objective, report the **error budget** and how much remains:
|
||||
- Budget = `1 − target` (e.g., 1% for a 99% target).
|
||||
- For "higher-is-better" SLOs (completion, success): remaining = `(observed − target) / (1 − target)`.
|
||||
- For "lower-is-better" SLOs (error rate): remaining = `(target − observed) / target`.
|
||||
- A negative result means the budget is exhausted — report how far over (e.g., "2.7× over budget").
|
||||
|
||||
### 4. Breach Drill-Down
|
||||
For any breached SLO, list the sessions contributing most to the breach (most failed tools or most APIErrors) via `GET /api/events?session_id=X`.
|
||||
|
||||
## Output
|
||||
|
||||
- A Markdown scorecard table: SLI | observed | target | status | error budget remaining.
|
||||
- Rates as percentages to 2 decimals; any currency in USD to 4 decimals.
|
||||
- Cite exact counts and `session_id` values — never fabricate numerators or denominators.
|
||||
- End with the SLO most at risk and the single action that would recover the most budget.
|
||||
- Read-only: only report what the API returns. If `curl` cannot reach `http://localhost:4820`, tell the user to start the dashboard with `npm start` from the repo root.
|
||||
Reference in New Issue
Block a user