feat: Claude Code Monitor — lanes, pipelines and a merged workspace
Internal SmartGift build of a Claude Code monitoring dashboard. Lanes: a durable unit of parallel agent work, one per working directory, tracked across session restarts. Managed lanes are git worktrees the dashboard provisions and can reset or remove behind a three-check destroy guard and a counted preflight; adopted lanes are directories you already own and are never destroyable. Pipelines: a lane moves through pipeline stages. A stage the agent declares with evidence renders green; a stage inferred from the tool-event stream renders dashed amber and never counts as done. Detection is forward-only within a 30-minute window, and never writes the declared stage. Workspace: one page at /run with a lane grid, the selected lane's pipeline, and a full Claude console behind a disclosure.
This commit is contained in:
@@ -0,0 +1,24 @@
|
||||
{
|
||||
"name": "ccam-insights",
|
||||
"description": "AI-powered insights for Claude Code — pattern detection, anomaly alerting, optimization recommendations, and session comparison using Agent Monitor analytics.",
|
||||
"version": "1.0.0",
|
||||
"author": {
|
||||
"name": "Nguyễn Ngọc Trí Vĩ",
|
||||
"url": "https://git.smartgift.io.vn/Smartgift-AI"
|
||||
},
|
||||
"homepage": "https://git.smartgift.io.vn/Smartgift-AI/Claude-Code-Monitor",
|
||||
"repository": {
|
||||
"type": "git",
|
||||
"url": "https://git.smartgift.io.vn/Smartgift-AI/Claude-Code-Monitor.git"
|
||||
},
|
||||
"license": "MIT",
|
||||
"keywords": [
|
||||
"insights",
|
||||
"patterns",
|
||||
"anomaly-detection",
|
||||
"optimization",
|
||||
"ai-analytics",
|
||||
"claude-code"
|
||||
],
|
||||
"categories": ["insights", "analytics", "ai"]
|
||||
}
|
||||
@@ -0,0 +1,62 @@
|
||||
---
|
||||
name: insights-advisor
|
||||
description: >
|
||||
Deep analysis agent that uses the full Agent Monitor data model — workflow
|
||||
intelligence (11 datasets per session), token tracking (baselines pre-summed
|
||||
into totals), pricing engine with pattern-matched model rules, session metadata
|
||||
(thinking_blocks, turn_count, turn_duration_ms, usage_extras including
|
||||
service_tier/speed/inference_geo), and the complete event taxonomy. Connects
|
||||
patterns across sessions to provide strategic, causation-based insights.
|
||||
model: sonnet
|
||||
tools:
|
||||
- Bash
|
||||
- Read
|
||||
- Grep
|
||||
---
|
||||
|
||||
# Insights Advisor
|
||||
|
||||
You are a strategic insights advisor. You analyze data from the Agent Monitor
|
||||
at `http://localhost:4820` to find deep patterns, predict trends, and provide
|
||||
high-impact recommendations.
|
||||
|
||||
## Available Data
|
||||
|
||||
| Endpoint | Returns |
|
||||
|----------|---------|
|
||||
| `/api/stats` | total_sessions, active_sessions, active_agents, total_agents, total_events, events_today |
|
||||
| `/api/analytics` | tokens (total_input, total_output, total_cache_read, total_cache_write — baselines pre-summed), tool_usage (top 20), daily_events (365d), daily_sessions (365d), event_types, agent_types, avg_events_per_session, total_subagents, sessions_by_status, agents_by_status |
|
||||
| `/api/sessions?limit=N` | Sessions with metadata JSON: thinking_blocks, turn_count, total_turn_duration_ms, usage_extras ({service_tiers[], speeds[], inference_geos[]}) |
|
||||
| `/api/sessions/:id` | Full session with nested agents[] and events[] |
|
||||
| `/api/pricing/cost` | `{ total_cost, breakdown: [{ model, input_tokens, output_tokens, cache_read_tokens, cache_write_tokens, cost, matched_rule }] }` |
|
||||
| `/api/pricing` | Model pricing rules: pattern, display_name, rates per Mtok for 4 token types |
|
||||
| `/api/workflows/:id` | **11 datasets**: stats, orchestration (DAG), toolFlow (transitions), effectiveness (subagent success), patterns (sequences), modelDelegation, errorPropagation (by depth), concurrency (lanes), complexity (score), compaction (impact), cooccurrence (agent pairs) |
|
||||
| `/api/events?session_id=X` | Full event stream: event_type ∈ {PreToolUse, PostToolUse, Stop, SubagentStop, SessionStart, SessionEnd, Notification, Compaction, APIError, TurnDuration} |
|
||||
|
||||
## Key Derived Metrics
|
||||
|
||||
- **Token totals**: Analytics API returns `total_input`, `total_output`, `total_cache_read`, `total_cache_write` (baselines pre-summed at DB level)
|
||||
- **Cache efficiency**: `total_cache_read / (total_cache_read + total_input)` — trend over time
|
||||
- **Tool success**: `PostToolUse / PreToolUse` — should be ~1.0
|
||||
- **Turn velocity**: `turn_count / (total_turn_duration_ms / 1000)`
|
||||
- **Cost per turn**: `session_cost / turn_count`
|
||||
|
||||
## Analysis Framework
|
||||
|
||||
1. **Descriptive** — What happened? Aggregate metrics, distributions, trends
|
||||
2. **Diagnostic** — Why? Correlations, root causes, comparative analysis
|
||||
3. **Predictive** — What will happen? Trend extrapolation with confidence
|
||||
4. **Prescriptive** — What should change? Behavioral changes with quantified impact
|
||||
|
||||
## Output Standards
|
||||
|
||||
- Most important insight first
|
||||
- Support every claim with specific data from the API
|
||||
- Confidence levels: High (>80% data support), Medium (50-80%), Low (<50%)
|
||||
- End with a prioritized action plan (max 5 items)
|
||||
|
||||
## Constraints
|
||||
|
||||
- Read-only — never modify data
|
||||
- Only use API data — never fabricate
|
||||
- Acknowledge uncertainty explicitly
|
||||
@@ -0,0 +1,69 @@
|
||||
---
|
||||
name: trend-forecaster
|
||||
description: >
|
||||
Forecasting agent that projects near-future Claude Code cost and usage from
|
||||
the Agent Monitor's 365-day daily series (daily_sessions, daily_events). Fits
|
||||
a simple moving average plus linear slope, extrapolates the next 7/14/30 days,
|
||||
and flags inflection points where the trend changes direction or
|
||||
accelerates. Anchors projected cost to the live pricing engine totals.
|
||||
model: sonnet
|
||||
tools:
|
||||
- Bash
|
||||
- Read
|
||||
- Grep
|
||||
---
|
||||
|
||||
# Trend Forecaster
|
||||
|
||||
You are a usage and cost forecaster. You query the Agent Monitor dashboard API at
|
||||
`http://localhost:4820` using `curl -s http://localhost:4820/api/...` to project
|
||||
near-future activity from historical daily trends and to flag inflection points.
|
||||
|
||||
## Available Data Sources
|
||||
|
||||
| Endpoint | Returns |
|
||||
|----------|---------|
|
||||
| `GET /api/analytics` | `daily_sessions` (365d), `daily_events` (365d), `tokens` (total_input, total_output, total_cache_read, total_cache_write — baselines pre-summed), `event_types`, `tool_usage`, `avg_events_per_session` |
|
||||
| `GET /api/pricing/cost` | `{ total_cost, breakdown:[{ model, input_tokens, output_tokens, cache_read_tokens, cache_write_tokens, cost, matched_rule }] }` — anchors cost-per-event/session |
|
||||
| `GET /api/sessions?limit=N` | Recent sessions with `cost`, `started_at`, `ended_at`, `model`, `metadata` — used to validate the daily series against per-session cost |
|
||||
| `GET /api/stats` | `total_sessions`, `events_today` — current-day sanity check against the series |
|
||||
|
||||
## Analysis Framework
|
||||
|
||||
1. **Pull the series** — `GET /api/analytics`; read `daily_sessions` and
|
||||
`daily_events` (each a 365-day `{ date, count }` array). Sort by date and fill
|
||||
missing days with zero so the windows are evenly spaced.
|
||||
2. **Smooth** — compute a trailing simple moving average (SMA) at windows 7 and 30
|
||||
for both series. The 7-day SMA is the short-term signal; the 30-day SMA is the
|
||||
baseline.
|
||||
3. **Slope** — fit a least-squares line over the last 30 days: `slope = Σ((i-ī)(y-ȳ)) / Σ((i-ī)²)`
|
||||
in units per day. Report slope for sessions/day and events/day.
|
||||
4. **Project** — extrapolate the last SMA value forward by the slope for horizons
|
||||
of 7, 14, and 30 days: `projected(t) = last_SMA + slope × t`. Floor projections
|
||||
at zero.
|
||||
5. **Cost-anchor** — from `GET /api/pricing/cost`, derive cost-per-event =
|
||||
`total_cost / total_events` (use `/api/analytics` total_events) and
|
||||
cost-per-session = `total_cost / total_sessions`. Multiply the projected
|
||||
event/session counts to get projected USD spend per horizon.
|
||||
6. **Inflection points** — flag dates where the 7-day SMA crosses the 30-day SMA
|
||||
(regime change), or where the rolling slope flips sign, or where week-over-week
|
||||
change exceeds ±50% (acceleration/collapse). Report the date and magnitude.
|
||||
|
||||
## Output Standards
|
||||
|
||||
- Lead with the headline projection: "Next 30 days ≈ N sessions / N events / $X.XXXX".
|
||||
- Cite real numbers pulled from the API — never fabricate counts or rates.
|
||||
- Currency in USD to 4 decimals; counts as integers; slope to 2 decimals/day.
|
||||
- Use ▲ for rising trends and ▼ for falling trends next to each metric.
|
||||
- Give a confidence label: High (steady slope, low variance), Medium, or Low
|
||||
(sparse/volatile series) — state the reason.
|
||||
- Present projections as a Markdown table: horizon | sessions | events | est. cost.
|
||||
- List inflection points with date, type (crossover/sign-flip/spike), and size.
|
||||
|
||||
## Constraints
|
||||
|
||||
- Read-only advisory role — never modify data.
|
||||
- Only use data returned by the API — never fabricate metrics.
|
||||
- A linear/SMA model is intentionally simple; call out that it assumes the recent
|
||||
regime persists and does not capture seasonality beyond the chosen windows.
|
||||
- If the dashboard is unreachable, tell the user to start it with `npm start` from the repo root.
|
||||
@@ -0,0 +1,23 @@
|
||||
---
|
||||
description: List current cost and token outlier sessions via z-score
|
||||
argument-hint: "[threshold]"
|
||||
---
|
||||
|
||||
List the current cost/token **outlier** sessions from the Agent Monitor dashboard using a z-score test. **$ARGUMENTS** optionally sets the z-score threshold (default `2.0`; lower = stricter).
|
||||
|
||||
1. Fetch the population:
|
||||
- `curl -s "http://localhost:4820/api/sessions?limit=200"` → a session list; each item has `id`, `status`, `model`, `cwd`, `started_at`, `cost`, and `metadata`.
|
||||
|
||||
2. Compute the baseline over all returned sessions:
|
||||
- Mean and standard deviation of `cost`.
|
||||
- For sessions where you need token totals, pull `curl -s http://localhost:4820/api/pricing/cost/<id>` and sum `input_tokens + output_tokens + cache_read_tokens + cache_write_tokens`; compute mean and stddev of total tokens too.
|
||||
|
||||
3. Flag outliers: any session whose `z = (value − mean) / stddev` exceeds the threshold (default 2.0) on cost (primary) or tokens (secondary). Skip the calc gracefully if stddev is 0.
|
||||
|
||||
4. Print the flagged sessions, sorted by descending cost z-score:
|
||||
- Session id (short), model, started_at.
|
||||
- Cost (USD, 4 decimals) and its z-score.
|
||||
- Total tokens and its z-score (when fetched).
|
||||
- A flag tag: 🔴 if z > 3, 🟡 if z > 2.
|
||||
|
||||
Output rules: a Markdown table of flagged sessions only; currency in USD to 4 decimals; z-scores to 2 decimals; if nothing exceeds the threshold, say "No cost/token outliers above z=<threshold>" and report the top session by cost for context. Cite only API values — never fabricate. If the dashboard is unreachable at `http://localhost:4820`, tell the user to start it with `npm start` from the repo root.
|
||||
@@ -0,0 +1,26 @@
|
||||
---
|
||||
description: Compare two sessions side-by-side with cost and workflow deltas
|
||||
argument-hint: "[sessionA] [sessionB]"
|
||||
---
|
||||
|
||||
Compare the two sessions in **$ARGUMENTS** (first id = Session A, second id = Session B) side-by-side using the Agent Monitor dashboard. If fewer than two ids are given, ask for both.
|
||||
|
||||
1. Fetch cost for each, in parallel:
|
||||
- `curl -s http://localhost:4820/api/pricing/cost/<sessionA>`
|
||||
- `curl -s http://localhost:4820/api/pricing/cost/<sessionB>`
|
||||
Each returns `{ total_cost, breakdown:[{ model, input_tokens, output_tokens, cache_read_tokens, cache_write_tokens, cost, matched_rule }] }`.
|
||||
|
||||
2. Fetch workflow intelligence for each:
|
||||
- `curl -s http://localhost:4820/api/workflows/<sessionA>`
|
||||
- `curl -s http://localhost:4820/api/workflows/<sessionB>`
|
||||
Use `stats` (tool/event/agent counts), `complexity` (score), `effectiveness` (subagent success), `compaction` (impact), and `errorPropagation`.
|
||||
|
||||
3. Print a side-by-side comparison table with a delta column (B − A):
|
||||
- Total cost (USD, 4 decimals) and Δ% .
|
||||
- Tokens: input, output, cache_read, cache_write (sum the breakdown per session).
|
||||
- Cache hit rate = `cache_read / (cache_read + input)`.
|
||||
- Tool count, event count, agent count (from `stats`).
|
||||
- Complexity score (from `complexity`).
|
||||
- Subagent success rate (from `effectiveness`) and compaction count (from `compaction`).
|
||||
|
||||
Output rules: one row per metric with columns Session A | Session B | Δ; use ▲ when B is higher and ▼ when lower; currency in USD to 4 decimals; rates as percentages to 2 decimals. End with a one-line verdict on which session was cheaper/leaner and the main driver. Cite only fields the API returned — never fabricate. If a session id is unknown or the dashboard is unreachable at `http://localhost:4820`, say so and tell the user to start it with `npm start` from the repo root.
|
||||
@@ -0,0 +1,22 @@
|
||||
---
|
||||
description: Surface the top 3 data-backed insights about your Claude Code usage right now
|
||||
---
|
||||
|
||||
Produce the **top 3 insights** about Claude Code usage right now, each backed by real numbers from the Agent Monitor dashboard.
|
||||
|
||||
1. Fetch high-level state:
|
||||
- `curl -s http://localhost:4820/api/stats` → total_sessions, active_sessions, active_agents, total_events, events_today, agents_by_status, sessions_by_status.
|
||||
- `curl -s http://localhost:4820/api/analytics` → tokens (total_input/total_output/total_cache_read/total_cache_write), tool_usage (top 20), daily_events (365d), daily_sessions (365d), event_types, avg_events_per_session, total_subagents.
|
||||
|
||||
2. Derive signal, citing exact field values:
|
||||
- Cache hit rate = `total_cache_read / (total_cache_read + total_input)`.
|
||||
- Activity trend: compare the last 7 days of `daily_sessions`/`daily_events` against the prior 7.
|
||||
- Concentration: the single most-used tool and most-frequent `event_type`, with its share of the total.
|
||||
- Error pressure: `APIError` share of events; subagent fan-out via `total_subagents` and `avg_events_per_session`.
|
||||
|
||||
3. Pick the **3 most decision-relevant** findings (biggest cost lever, sharpest trend, or clearest anomaly). For each print:
|
||||
- A one-line headline with the supporting number.
|
||||
- Why it matters in one sentence.
|
||||
- One concrete action.
|
||||
|
||||
Output rules: rank by impact (most important first); currency in USD to 4 decimals; rates as percentages to 2 decimals; use ▲/▼ for trend direction; cite only fields the API returned — never fabricate. If `curl` cannot reach `http://localhost:4820`, tell the user to start the dashboard with `npm start` from the repo root.
|
||||
@@ -0,0 +1,88 @@
|
||||
---
|
||||
description: >
|
||||
Identify anomalous sessions using Agent Monitor data — cost outliers from
|
||||
the pricing engine, token anomalies (cache miss spikes, compaction baseline
|
||||
surges), unusual event type ratios (PreToolUse/PostToolUse gaps, APIError
|
||||
clusters), behavioral deviations from workflow intelligence (complexity
|
||||
score outliers, error propagation anomalies), and sessions with abnormal
|
||||
metadata (extreme turn_count, high thinking_blocks, zero turn_duration).
|
||||
---
|
||||
|
||||
# Anomaly Alert
|
||||
|
||||
Detect anomalous sessions in Claude Code Agent Monitor data.
|
||||
|
||||
## Input
|
||||
|
||||
The user provides: **$ARGUMENTS**
|
||||
|
||||
This may be:
|
||||
- "all" or empty (default: check all anomaly types)
|
||||
- "cost" for cost anomalies only
|
||||
- "duration" for duration anomalies only
|
||||
- "errors" for error rate anomalies only
|
||||
- A sensitivity level: "strict" (1σ), "normal" (2σ), "relaxed" (3σ)
|
||||
|
||||
## Procedure
|
||||
|
||||
1. **Fetch baseline data** from `http://localhost:4820`:
|
||||
- `GET /api/sessions?limit=500` — historical sessions for baseline
|
||||
- `GET /api/analytics` — aggregated metrics
|
||||
- `GET /api/pricing/cost` — cost data per session
|
||||
|
||||
2. **Compute baselines** for each metric:
|
||||
- Mean, median, standard deviation
|
||||
- P25, P75, P90, P95, P99 percentiles
|
||||
- Interquartile range (IQR) for robust outlier detection
|
||||
|
||||
3. **Detect anomalies** using statistical thresholds:
|
||||
|
||||
### Cost Anomalies
|
||||
- Sessions costing >2σ above mean
|
||||
- Single sessions exceeding daily average
|
||||
- Sudden cost spikes (session-over-session increase >200%)
|
||||
|
||||
### Duration Anomalies
|
||||
- Sessions lasting >2σ above mean duration
|
||||
- Extremely short sessions (<1 minute) that still incur cost
|
||||
- Sessions with unusual active-vs-idle ratios
|
||||
|
||||
### Error Rate Anomalies
|
||||
- Sessions with error rates >2σ above baseline
|
||||
- New error types not seen in previous sessions
|
||||
- Sessions with >3 consecutive tool failures
|
||||
|
||||
### Behavioral Anomalies
|
||||
- Unusual tool combinations not seen before
|
||||
- Sessions with abnormally high compaction counts
|
||||
- Model switches mid-session (if unexpected)
|
||||
- Sessions with no tool usage (pure conversation)
|
||||
|
||||
### Token Anomalies
|
||||
- Input/output token ratio far from historical norm
|
||||
- Cache miss rate significantly higher than average
|
||||
- Token usage growing faster than session count
|
||||
|
||||
4. **Classify each anomaly**:
|
||||
- **🔴 Critical**: Likely indicates a real problem requiring attention
|
||||
- **🟡 Warning**: Unusual but may be expected for certain tasks
|
||||
- **🔵 Info**: Interesting deviation worth noting
|
||||
|
||||
## Output Format
|
||||
|
||||
Present as an **Anomaly Report**:
|
||||
|
||||
```
|
||||
═══════════════════════════════════════════════
|
||||
ANOMALY DETECTION REPORT
|
||||
Analyzed: N sessions | Baseline: last 30 days
|
||||
Anomalies found: N (🔴 N critical, 🟡 N warn, 🔵 N info)
|
||||
═══════════════════════════════════════════════
|
||||
```
|
||||
|
||||
For each anomaly:
|
||||
- Session ID and timestamp
|
||||
- Anomaly type and severity
|
||||
- Observed value vs expected range
|
||||
- Possible explanation
|
||||
- Recommended action (if any)
|
||||
@@ -0,0 +1,66 @@
|
||||
---
|
||||
description: >
|
||||
Benchmark one session (or a small recent set) against the rolling average using
|
||||
Agent Monitor data — cost, total tokens, tool count, and workflow complexity
|
||||
score — and report where each metric lands as a percentile of the population.
|
||||
Tells you whether a session was normal, cheap, or an outlier. Use when judging
|
||||
whether a session was typical or out of band.
|
||||
---
|
||||
|
||||
# Benchmark
|
||||
|
||||
Score a session against the rolling population average and report its percentile on
|
||||
cost, tokens, tool count, and complexity using Agent Monitor data.
|
||||
|
||||
## Input
|
||||
|
||||
The user provides: **$ARGUMENTS**
|
||||
|
||||
This may be:
|
||||
- A single session ID — benchmark that session
|
||||
- "latest" — benchmark the most recent session
|
||||
- "latest N" — benchmark the N most recent sessions, each vs the average
|
||||
- empty — benchmark the most recent session (default)
|
||||
|
||||
## Data Sources
|
||||
|
||||
| Endpoint | Returns |
|
||||
|----------|---------|
|
||||
| `GET /api/sessions?limit=N` | Population of sessions with `cost`, `model`, `started_at`, `metadata` (turn_count, total_turn_duration_ms) — builds the rolling baseline |
|
||||
| `GET /api/pricing/cost/{sessionId}` | `{ total_cost, breakdown:[{ input_tokens, output_tokens, cache_read_tokens, cache_write_tokens, cost }] }` — the target session's cost and tokens |
|
||||
| `GET /api/workflows/{sessionId}` | `complexity` (score), `stats` (tool/event counts), `toolFlow` (distinct tools used) — the target session's tool count and complexity |
|
||||
| `GET /api/analytics` | `avg_events_per_session`, `tool_usage`, `daily_sessions` — corroborates population-level averages |
|
||||
|
||||
## Report Sections
|
||||
|
||||
### 1. Build the Baseline
|
||||
Fetch the population with `GET /api/sessions?limit=200` (the rolling set). For each
|
||||
session gather cost (`GET /api/pricing/cost/{id}` or the list `cost` field), total
|
||||
tokens (sum of the 4 token types from the pricing breakdown), tool count and
|
||||
complexity (`GET /api/workflows/{id}`). Compute mean, median, and standard
|
||||
deviation for each metric across the population.
|
||||
|
||||
### 2. Measure the Target
|
||||
For the requested session, pull the same four metrics:
|
||||
- **Cost** — `total_cost` from `GET /api/pricing/cost/{id}`.
|
||||
- **Total tokens** — `input + output + cache_read + cache_write` summed from the breakdown.
|
||||
- **Tool count** — distinct/total tools from `GET /api/workflows/{id}` `stats`/`toolFlow`.
|
||||
- **Complexity score** — `complexity.score` from `GET /api/workflows/{id}`.
|
||||
|
||||
### 3. Percentile and Deviation
|
||||
For each metric report the target's percentile within the population (share of
|
||||
sessions at or below it) and its z-score `(value − mean) / stddev`. Label each:
|
||||
below average / typical / above average / outlier (|z| > 2).
|
||||
|
||||
### 4. Verdict
|
||||
State whether the session was normal overall. If it is an outlier, name which
|
||||
metric drove it (e.g., complexity p96, cost p91 → an unusually heavy session).
|
||||
|
||||
## Output
|
||||
|
||||
- A Markdown table: metric | session value | population mean | percentile | z-score | label.
|
||||
- Currency in USD to 4 decimals; tokens and tool counts as integers; complexity to 2 decimals.
|
||||
- Use ▲ for above-average and ▼ for below-average vs the mean.
|
||||
- One-line verdict: "Normal session" or "Outlier — driven by <metric> (pNN)".
|
||||
- When benchmarking multiple sessions, one row block per session plus a summary line.
|
||||
- Read-only: percentiles come only from the fetched population; never fabricate the baseline.
|
||||
@@ -0,0 +1,82 @@
|
||||
---
|
||||
description: >
|
||||
Suggest concrete optimizations for Claude Code usage based on historical
|
||||
session data. Covers cost reduction, speed improvement, error prevention,
|
||||
and workflow efficiency. Use for data-driven optimization planning.
|
||||
---
|
||||
|
||||
# Optimization Suggest
|
||||
|
||||
Generate data-driven optimization recommendations for Claude Code usage.
|
||||
|
||||
## Input
|
||||
|
||||
The user provides: **$ARGUMENTS**
|
||||
|
||||
This may be:
|
||||
- "all" or empty (default: comprehensive optimization scan)
|
||||
- "cost" for cost reduction focus
|
||||
- "speed" for performance/speed focus
|
||||
- "quality" for error reduction focus
|
||||
- "efficiency" for workflow efficiency focus
|
||||
|
||||
## Procedure
|
||||
|
||||
1. **Gather optimization data** from `http://localhost:4820`:
|
||||
- `GET /api/sessions?limit=200` — session history
|
||||
- `GET /api/analytics` — tool and token analytics
|
||||
- `GET /api/pricing/cost` — cost data
|
||||
- `GET /api/pricing` — pricing rules for model comparison
|
||||
- Sample event streams for behavioral analysis
|
||||
|
||||
2. **Analyze optimization opportunities**:
|
||||
|
||||
### 💰 Cost Optimization
|
||||
- **Model downgrade opportunities**: Tasks completed with expensive models that could use cheaper ones
|
||||
- Compare success rates per model per task type
|
||||
- Calculate savings from model substitution
|
||||
- **Cache optimization**: Sessions with low cache hit rates
|
||||
- Identify sessions that could benefit from better prompt caching
|
||||
- **Early termination**: Sessions that ran longer than needed
|
||||
- Detect sessions where useful work completed well before session end
|
||||
- **Compaction reduction**: Sessions hitting context limits
|
||||
- Suggest breaking large tasks into smaller sessions
|
||||
|
||||
### ⚡ Speed Optimization
|
||||
- **Tool selection**: Faster alternatives for commonly-used tool patterns
|
||||
- **Subagent parallelization**: Tasks that could run in parallel
|
||||
- **Session planning**: Better upfront context to reduce back-and-forth
|
||||
- **Preemptive context loading**: Frequently needed files/context
|
||||
|
||||
### 🛡 Quality Optimization
|
||||
- **Error prevention**: Common error patterns with preventive measures
|
||||
- **Tool reliability**: Tools with high failure rates and alternatives
|
||||
- **Validation gaps**: Sessions lacking verification steps
|
||||
- **Recovery strategies**: Better error handling patterns
|
||||
|
||||
### 🔄 Workflow Optimization
|
||||
- **Session sizing**: Optimal session scope based on historical success
|
||||
- **Task decomposition**: Complex sessions that should be split
|
||||
- **Automation candidates**: Repetitive workflows to automate
|
||||
- **Knowledge reuse**: Patterns where previous session context could help
|
||||
|
||||
3. **Quantify each recommendation**:
|
||||
- Estimated impact (cost savings $, time savings %, error reduction %)
|
||||
- Implementation effort (low/medium/high)
|
||||
- Confidence level based on data available
|
||||
- Priority score = Impact × Confidence / Effort
|
||||
|
||||
## Output Format
|
||||
|
||||
Present as a prioritized optimization plan:
|
||||
|
||||
| # | Recommendation | Category | Impact | Effort | Priority |
|
||||
|---|---------------|----------|--------|--------|----------|
|
||||
| 1 | Specific action | 💰/⚡/🛡/🔄 | High | Low | ★★★★★ |
|
||||
| 2 | Specific action | ... | ... | ... | ★★★★☆ |
|
||||
|
||||
For the top 5 recommendations, include:
|
||||
- Detailed explanation with supporting data
|
||||
- Step-by-step implementation guide
|
||||
- Expected before/after metrics
|
||||
- How to measure success
|
||||
@@ -0,0 +1,77 @@
|
||||
---
|
||||
description: >
|
||||
Detect recurring patterns using the Agent Monitor's workflow intelligence —
|
||||
toolFlow transitions (tool A → B frequency matrices), recurring workflow
|
||||
patterns, agent co-occurrence pairs, model delegation habits, error
|
||||
propagation paths by agent depth, and compaction triggers. Use to discover
|
||||
habitual usage patterns and anti-patterns.
|
||||
---
|
||||
|
||||
# Pattern Detect
|
||||
|
||||
Identify recurring patterns using the Agent Monitor's workflow intelligence engine.
|
||||
|
||||
## Input
|
||||
|
||||
The user provides: **$ARGUMENTS**
|
||||
|
||||
Options: "all", "tools", "errors", "workflows", "last N sessions".
|
||||
|
||||
## Data Sources
|
||||
|
||||
| Endpoint | Returns |
|
||||
|----------|---------|
|
||||
| `GET /api/sessions?limit=200` | Session list with status, model, cwd, metadata |
|
||||
| `GET /api/analytics` | tool_usage top 20, event_types, agent_types |
|
||||
| `GET /api/workflows/{sessionId}` | 11 datasets per session (see below) |
|
||||
|
||||
### Workflow datasets used for pattern detection
|
||||
|
||||
| Dataset | Pattern insight |
|
||||
|---------|----------------|
|
||||
| `toolFlow` | **Tool transition matrix**: tool A → tool B with counts — reveals sequential habits |
|
||||
| `patterns` | **Detected workflow patterns**: recurring sequences with frequency scores |
|
||||
| `cooccurrence` | **Agent co-occurrence**: which agents frequently run together |
|
||||
| `modelDelegation` | **Model habits**: which models are chosen for which task types |
|
||||
| `errorPropagation` | **Error patterns**: where errors start and how they cascade by agent depth |
|
||||
| `effectiveness` | **Subagent patterns**: which types succeed most, avg duration per type |
|
||||
| `compaction` | **Compaction triggers**: what causes context overflow |
|
||||
| `complexity` | **Complexity patterns**: session complexity scores over time |
|
||||
|
||||
## Pattern Categories
|
||||
|
||||
### 1. Tool Chain Patterns (from `toolFlow`)
|
||||
- **Most common sequences**: Top 10 tool transitions (e.g., Read → Edit: 145 times)
|
||||
- **Starter tools**: First tool used in sessions (indicates task type)
|
||||
- **Finisher tools**: Last tool before Stop event
|
||||
- **Anti-patterns**: Tool → same Tool repeated (retries/failures)
|
||||
- **Co-occurrence**: Tools that always appear together in sessions
|
||||
|
||||
### 2. Workflow Patterns (from `patterns`)
|
||||
- **Named patterns**: Workflow sequences the API has detected with frequency
|
||||
- **Session archetypes**: Common session shapes (short edit, long debug, subagent-heavy)
|
||||
- **Project-specific**: Patterns that appear in specific working directories
|
||||
|
||||
### 3. Error Patterns (from `errorPropagation` + `event_types`)
|
||||
- **Error origins**: Which agent depth level produces most errors
|
||||
- **Cascade patterns**: Errors that trigger chains of follow-up errors
|
||||
- **APIError frequency**: quota hits, rate_limit, overloaded — by time of day
|
||||
- **Recovery patterns**: How errors are typically resolved (tool retry vs agent switch)
|
||||
|
||||
### 4. Agent Patterns (from `cooccurrence` + `effectiveness`)
|
||||
- **Agent pairs**: Which agents are spawned together frequently
|
||||
- **Delegation patterns**: Main agent → subagent task delegation habits
|
||||
- **Success by type**: Which subagent types (task/explore/code-review) work best for which tasks
|
||||
|
||||
### 5. Temporal Patterns (from session timestamps + `daily_sessions`)
|
||||
- **Peak hours**: When sessions cluster
|
||||
- **Duration patterns**: Short vs long session distribution
|
||||
- **Day-of-week trends**: Productive days vs quiet days
|
||||
|
||||
## Output
|
||||
|
||||
**Pattern Report** with top 10 patterns ranked by frequency × impact:
|
||||
- Pattern name and description
|
||||
- Frequency (occurrences across analyzed sessions)
|
||||
- Impact: positive (reinforce), negative (eliminate), or neutral (observe)
|
||||
- Actionable recommendation for each
|
||||
@@ -0,0 +1,82 @@
|
||||
---
|
||||
description: >
|
||||
Detect quality and efficiency regressions over time using Agent Monitor data —
|
||||
rising error rate (APIError events), falling cache hit rate, growing compaction
|
||||
frequency, and climbing cost-per-session. Splits history into an earlier
|
||||
baseline window and a recent window and reports which metrics are getting
|
||||
worse, by how much, and where. Use when checking whether things are degrading
|
||||
or trending in the wrong direction.
|
||||
---
|
||||
|
||||
# Regression Watch
|
||||
|
||||
Detect whether Claude Code sessions are getting worse over time across quality and
|
||||
efficiency metrics, using Agent Monitor data.
|
||||
|
||||
## Input
|
||||
|
||||
The user provides: **$ARGUMENTS**
|
||||
|
||||
This may be:
|
||||
- empty or "all" — check every regression metric (default)
|
||||
- "errors" — error-rate regression only
|
||||
- "cache" — cache hit-rate regression only
|
||||
- "compaction" — compaction-frequency regression only
|
||||
- "cost" — cost-per-session regression only
|
||||
- A window like "last 30d" or "30 vs 90" — set the recent vs baseline window sizes
|
||||
|
||||
## Data Sources
|
||||
|
||||
| Endpoint | Returns |
|
||||
|----------|---------|
|
||||
| `GET /api/analytics` | `daily_events` (365d), `daily_sessions` (365d), `event_types`, `tokens` (total_input, total_output, total_cache_read, total_cache_write — baselines pre-summed), `avg_events_per_session` |
|
||||
| `GET /api/events?session_id=X` | Event stream incl. `APIError`, `Compaction`, `PreToolUse`/`PostToolUse` — used to localize regressions to specific sessions |
|
||||
| `GET /api/pricing/cost` | `{ total_cost, breakdown[...] }` — total cost to derive cost-per-session |
|
||||
| `GET /api/pricing/cost/{sessionId}` | Per-session cost — used to compare recent vs baseline session cost |
|
||||
| `GET /api/workflows/{sessionId}` | `compaction` (impact), `errorPropagation` (by depth), `effectiveness` — per-session quality signals |
|
||||
| `GET /api/sessions?limit=N` | Sessions with `started_at`, `cost`, `metadata` — to bucket sessions into time windows |
|
||||
|
||||
## Report Sections
|
||||
|
||||
### 1. Windowing
|
||||
Split history into a **baseline window** (older) and a **recent window** (newer).
|
||||
Default: recent = last 30 days, baseline = the 30–90 day range before it. Use
|
||||
`daily_events`/`daily_sessions` for series metrics and `GET /api/sessions?limit=N`
|
||||
to assign sessions to each window by `started_at`.
|
||||
|
||||
### 2. Error Rate Regression
|
||||
- Recent error rate = `APIError count / total events` in the recent window
|
||||
(from `event_types` and `daily_events`, or per-session `GET /api/events`).
|
||||
- Compare to the baseline rate. Flag if recent is higher.
|
||||
- Report the absolute and relative change and which sessions contributed most
|
||||
`APIError` events.
|
||||
|
||||
### 3. Cache Hit Rate Regression
|
||||
- Cache hit rate = `total_cache_read / (total_cache_read + total_input)`.
|
||||
- Compute for each window (per-window input/cache_read from session metadata or
|
||||
the pricing breakdown). Flag a **falling** hit rate — that means more
|
||||
uncached input tokens and higher cost.
|
||||
|
||||
### 4. Compaction Frequency Regression
|
||||
- Compaction frequency = `Compaction events / session` per window (from
|
||||
`event_types` / `daily_events`, confirmed via per-session
|
||||
`GET /api/workflows/{id}` `compaction`). Flag a **rising** rate — context is
|
||||
overflowing more often.
|
||||
|
||||
### 5. Cost-per-Session Regression
|
||||
- Cost-per-session = window total cost / window session count, using
|
||||
`GET /api/pricing/cost` overall and `GET /api/pricing/cost/{id}` for the
|
||||
sessions in each window. Flag a **climbing** value.
|
||||
|
||||
### 6. Verdict
|
||||
Roll up which metrics regressed, rank by relative worsening, and name the most
|
||||
likely driver (e.g., cache hit rate fell → cost per session climbed).
|
||||
|
||||
## Output
|
||||
|
||||
- A Markdown table: metric | baseline | recent | Δ | direction (▲ worse / ▼ better) | verdict.
|
||||
- Tag each regressed metric 🔴 (clear regression), 🟡 (mild/within noise), or 🟢 (improved).
|
||||
- Currency in USD to 4 decimals; rates as percentages to 2 decimals.
|
||||
- List the specific session IDs that contributed most to any regression.
|
||||
- End with the single highest-priority regression to address and a concrete next step.
|
||||
- Read-only: only report what the API returns; never fabricate baselines.
|
||||
@@ -0,0 +1,89 @@
|
||||
---
|
||||
description: >
|
||||
Compare two sessions side-by-side using Agent Monitor data — per-model
|
||||
token usage (input/output/cache_read/cache_write + compaction baselines),
|
||||
pricing engine cost breakdowns, workflow intelligence (complexity scores,
|
||||
tool flow transitions, subagent effectiveness), session metadata
|
||||
(thinking_blocks, turn_count, turn_duration_ms, usage_extras), and
|
||||
full event timelines with all 10+ event types.
|
||||
---
|
||||
|
||||
# Session Compare
|
||||
|
||||
Compare two Claude Code sessions side-by-side using Agent Monitor data.
|
||||
|
||||
## Input
|
||||
|
||||
The user provides: **$ARGUMENTS**
|
||||
|
||||
This may be:
|
||||
- Two session IDs: "abc123 def456"
|
||||
- "best vs worst" — compare highest and lowest productivity sessions
|
||||
- "latest 2" — compare the two most recent sessions
|
||||
- A session ID + "vs average" — compare one session against the baseline
|
||||
|
||||
## Procedure
|
||||
|
||||
1. **Identify sessions to compare**:
|
||||
- If two IDs given: fetch both from `http://localhost:4820/api/sessions/{id}`
|
||||
- If "best vs worst": fetch sessions, score by completion + cost efficiency, pick extremes
|
||||
- If "latest 2": `GET /api/sessions?limit=2` (default sort: most recently updated first)
|
||||
- If "vs average": fetch session + compute averages from last 50 sessions
|
||||
|
||||
2. **Gather detailed data** for each session:
|
||||
- Session metadata: `GET /api/sessions/{id}`
|
||||
- Events: `GET /api/events?session_id={id}`
|
||||
- Agents: `GET /api/agents?session_id={id}`
|
||||
- Cost: `GET /api/pricing/cost/{id}`
|
||||
|
||||
3. **Build comparison**:
|
||||
|
||||
### Overview Comparison
|
||||
| Metric | Session A | Session B | Difference |
|
||||
|--------|-----------|-----------|-----------|
|
||||
| Status | completed | error | — |
|
||||
| Model | sonnet-4 | sonnet-4 | same |
|
||||
| Duration | 12m 34s | 45m 12s | +32m 38s |
|
||||
| Total Cost | $0.0234 | $0.1456 | +522% |
|
||||
| Events | 45 | 187 | +315% |
|
||||
| Tools Used | 8 | 12 | +4 |
|
||||
| Error Count | 0 | 7 | +7 |
|
||||
| Agents | 2 | 5 | +3 |
|
||||
|
||||
### Token Comparison
|
||||
| Token Type | Session A | Session B | Difference |
|
||||
|-----------|-----------|-----------|-----------|
|
||||
| Input | N | N | ±N% |
|
||||
| Output | N | N | ±N% |
|
||||
| Cache Read | N | N | ±N% |
|
||||
| Cache Write | N | N | ±N% |
|
||||
| Efficiency | N% | N% | ±N% |
|
||||
|
||||
### Tool Usage Comparison
|
||||
- Tools unique to Session A
|
||||
- Tools unique to Session B
|
||||
- Shared tools with usage count comparison
|
||||
- Error rate per tool in each session
|
||||
|
||||
### Timeline Comparison
|
||||
- Side-by-side event timeline
|
||||
- Where sessions diverged in approach
|
||||
- Key decision points that led to different outcomes
|
||||
|
||||
### Agent Activity Comparison
|
||||
- Agent counts and types
|
||||
- Subagent strategy differences
|
||||
- Agent success rates
|
||||
|
||||
4. **Analysis**:
|
||||
- Why one session was more efficient/successful than the other
|
||||
- Key decisions that made the difference
|
||||
- Lessons to apply to future sessions
|
||||
|
||||
## Output Format
|
||||
|
||||
Present as a side-by-side comparison report with:
|
||||
- Executive comparison summary (which session was "better" and why)
|
||||
- Structured comparison tables with color-coded differences (green = better, red = worse)
|
||||
- A "Lessons Learned" section with actionable takeaways
|
||||
- Overall winner declaration with justification
|
||||
Reference in New Issue
Block a user