refactor!: rebrand CCAM to AgentHub
Clean break, no compatibility alias. Renames the product name, the CLI binary (`ccam` -> `agenthub`), the env var prefix (`CCAM_` -> `AGENTHUB_`), the Prometheus metric namespace (`ccam_*` -> `agenthub_*`), the tmux session prefix (`ccam-lane-<id>` -> `agenthub-lane-<id>`), the per-repo runtime profile directory (`.ccam/profile` -> `.agenthub/profile`), the lanes state root (`~/.claude/ccam-lanes` -> `~/.claude/agenthub-lanes`), the bundled plugins (`plugins/ccam*` -> `plugins/agenthub*`) and their slash commands (`/ccam-doctor` -> `/agenthub-doctor`, `/ccam:*` -> `/agenthub:*`). Because there is no alias, an existing install has to be updated by hand: rename its `.ccam/profile` directory and any `CCAM_*` variables, and reinstall the plugin under its new name. Two other name families are deliberately left alone here, since they are not just strings: the `Claude-Code-Monitor` repository/marketplace name, and the `agent-dashboard` npm package plus its `~/.claude/agent-dashboard` runtime directory, which holds the live database and needs a migration rather than a rename. Verified: server 1406/1406, client 459/459, `grep -i ccam` clean across tracked files, `openapi.yaml` regenerated (95 paths, unchanged count).
This commit is contained in:
@@ -0,0 +1,82 @@
|
||||
---
|
||||
description: >
|
||||
Detect quality and efficiency regressions over time using Agent Monitor data —
|
||||
rising error rate (APIError events), falling cache hit rate, growing compaction
|
||||
frequency, and climbing cost-per-session. Splits history into an earlier
|
||||
baseline window and a recent window and reports which metrics are getting
|
||||
worse, by how much, and where. Use when checking whether things are degrading
|
||||
or trending in the wrong direction.
|
||||
---
|
||||
|
||||
# Regression Watch
|
||||
|
||||
Detect whether Claude Code sessions are getting worse over time across quality and
|
||||
efficiency metrics, using Agent Monitor data.
|
||||
|
||||
## Input
|
||||
|
||||
The user provides: **$ARGUMENTS**
|
||||
|
||||
This may be:
|
||||
- empty or "all" — check every regression metric (default)
|
||||
- "errors" — error-rate regression only
|
||||
- "cache" — cache hit-rate regression only
|
||||
- "compaction" — compaction-frequency regression only
|
||||
- "cost" — cost-per-session regression only
|
||||
- A window like "last 30d" or "30 vs 90" — set the recent vs baseline window sizes
|
||||
|
||||
## Data Sources
|
||||
|
||||
| Endpoint | Returns |
|
||||
|----------|---------|
|
||||
| `GET /api/analytics` | `daily_events` (365d), `daily_sessions` (365d), `event_types`, `tokens` (total_input, total_output, total_cache_read, total_cache_write — baselines pre-summed), `avg_events_per_session` |
|
||||
| `GET /api/events?session_id=X` | Event stream incl. `APIError`, `Compaction`, `PreToolUse`/`PostToolUse` — used to localize regressions to specific sessions |
|
||||
| `GET /api/pricing/cost` | `{ total_cost, breakdown[...] }` — total cost to derive cost-per-session |
|
||||
| `GET /api/pricing/cost/{sessionId}` | Per-session cost — used to compare recent vs baseline session cost |
|
||||
| `GET /api/workflows/{sessionId}` | `compaction` (impact), `errorPropagation` (by depth), `effectiveness` — per-session quality signals |
|
||||
| `GET /api/sessions?limit=N` | Sessions with `started_at`, `cost`, `metadata` — to bucket sessions into time windows |
|
||||
|
||||
## Report Sections
|
||||
|
||||
### 1. Windowing
|
||||
Split history into a **baseline window** (older) and a **recent window** (newer).
|
||||
Default: recent = last 30 days, baseline = the 30–90 day range before it. Use
|
||||
`daily_events`/`daily_sessions` for series metrics and `GET /api/sessions?limit=N`
|
||||
to assign sessions to each window by `started_at`.
|
||||
|
||||
### 2. Error Rate Regression
|
||||
- Recent error rate = `APIError count / total events` in the recent window
|
||||
(from `event_types` and `daily_events`, or per-session `GET /api/events`).
|
||||
- Compare to the baseline rate. Flag if recent is higher.
|
||||
- Report the absolute and relative change and which sessions contributed most
|
||||
`APIError` events.
|
||||
|
||||
### 3. Cache Hit Rate Regression
|
||||
- Cache hit rate = `total_cache_read / (total_cache_read + total_input)`.
|
||||
- Compute for each window (per-window input/cache_read from session metadata or
|
||||
the pricing breakdown). Flag a **falling** hit rate — that means more
|
||||
uncached input tokens and higher cost.
|
||||
|
||||
### 4. Compaction Frequency Regression
|
||||
- Compaction frequency = `Compaction events / session` per window (from
|
||||
`event_types` / `daily_events`, confirmed via per-session
|
||||
`GET /api/workflows/{id}` `compaction`). Flag a **rising** rate — context is
|
||||
overflowing more often.
|
||||
|
||||
### 5. Cost-per-Session Regression
|
||||
- Cost-per-session = window total cost / window session count, using
|
||||
`GET /api/pricing/cost` overall and `GET /api/pricing/cost/{id}` for the
|
||||
sessions in each window. Flag a **climbing** value.
|
||||
|
||||
### 6. Verdict
|
||||
Roll up which metrics regressed, rank by relative worsening, and name the most
|
||||
likely driver (e.g., cache hit rate fell → cost per session climbed).
|
||||
|
||||
## Output
|
||||
|
||||
- A Markdown table: metric | baseline | recent | Δ | direction (▲ worse / ▼ better) | verdict.
|
||||
- Tag each regressed metric 🔴 (clear regression), 🟡 (mild/within noise), or 🟢 (improved).
|
||||
- Currency in USD to 4 decimals; rates as percentages to 2 decimals.
|
||||
- List the specific session IDs that contributed most to any regression.
|
||||
- End with the single highest-priority regression to address and a concrete next step.
|
||||
- Read-only: only report what the API returns; never fabricate baselines.
|
||||
Reference in New Issue
Block a user