feat: Claude Code Monitor — lanes, pipelines and a merged workspace

Internal SmartGift build of a Claude Code monitoring dashboard. Lanes: a durable unit of parallel agent work, one per working directory, tracked across session restarts. Managed lanes are git worktrees the dashboard provisions and can reset or remove behind a three-check destroy guard and a counted preflight; adopted lanes are directories you already own and are never destroyable. Pipelines: a lane moves through pipeline stages. A stage the agent declares with evidence renders green; a stage inferred from the tool-event stream renders dashed amber and never counts as done. Detection is forward-only within a 30-minute window, and never writes the declared stage. Workspace: one page at /run with a lane grid, the selected lane's pipeline, and a full Claude console behind a disclosure.
2026-07-29 17:07:45 +07:00
commit 57dc91585d
783 changed files with 221743 additions and 0 deletions
@@ -0,0 +1,419 @@
+# Agent Conversation Viewer Design
+
+## Overview
+
+Add a conversation viewer to the SessionDetail page, enabling visual inspection of Main Agent and sub-agent interactions (message content and tool call details), with data sourced from real-time JSONL transcript files.
+
+## Problem
+
+The current dashboard tracks agent sessions, events, and tool usage at a summary level, but does not expose the actual conversation content — user messages, assistant replies, tool call parameters, and tool results. Users cannot see what each agent actually did or said, limiting debugging and audit capabilities.
+
+### v2 Additional Problems: Poor Pagination UX + No Real-time Updates
+
+After v1 implementation, two core UX issues emerged:
+
+1. **Pagination doesn't match conversation intuition** — v1 uses offset-based pagination starting from the beginning, so users see the oldest messages first and must page through to reach recent interactions, which doesn't align with chat product conventions.
+2. **No real-time updates** — v1 doesn't subscribe to WebSocket events, so users must manually refresh to see new messages, making it impossible to follow active sessions in real time.
+3. **Sub-agent selection uses database IDs** — v1's `agent_id` parameter relies on database agent IDs, but JSONL files are named with short IDs (e.g. `ad18a79192af10ed1`), causing a mismatch that prevents sub-agent transcripts from loading.
+
+## Design Decisions
+
+| Decision | Choice | Rationale |
+|----------|--------|-----------|
+| Data source | Real-time JSONL reads | Data is always current, no extra storage needed |
+| UI location | Conversation tab within SessionDetail | User-requested; keeps agent tree in the same context |
+| Claude home path | Configurable via `CLAUDE_HOME` env var | Supports non-default paths like `~/.codefuse/engine/cc/` |
+| Message rendering | Collapsible tool calls and thinking blocks | Keeps the view scannable; expand for details |
+| Load strategy (v2) | Chat-flow: load latest N by default, scroll up for history | Matches chat product intuition; users care most about recent interactions |
+| Real-time updates (v2) | WebSocket `new_event` triggers incremental load | Active sessions don't need manual refresh |
+| Agent selection (v2) | Filesystem scan + dropdown | Bypasses database ID mismatch by using file short IDs directly |
+
+## Architecture
+
+### Data Flow
+
+**v1 (deprecated):**
+```
+User clicks "Conversation" tab
+  → Frontend calls GET /api/sessions/:id/transcript[?agent_id=xxx&limit=50&offset=0]
+  → Server resolves JSONL path via claude-home.js
+  → Server reads and parses JSONL file
+  → Server returns structured message list
+  → Frontend renders MessageList (with collapsible blocks)
+```
+
+**v2 Chat-flow (current implementation):**
+```
+Initial load:
+  User opens Conversation tab
+  → GET /api/sessions/:id/transcripts  ← fetch available transcript list
+  → GET /api/sessions/:id/transcript?limit=50  ← default returns latest 50 messages
+  → Frontend renders message list + auto-scrolls to bottom
+
+Real-time updates:
+  CLI Hook → POST /api/hooks/event → processEvent()
+    → broadcast("new_event", {session_id, ...})
+      → WebSocket → ConversationView
+        → GET /api/sessions/:id/transcript?after=N  ← incremental load
+          → Append to bottom + auto-scroll (if user is at bottom)
+
+History load:
+  User scrolls to top
+    → GET /api/sessions/:id/transcript?before=M&limit=50  ← load older messages
+      → Prepend to top + preserve scroll position (no jump)
+```
+
+### Configurable Claude Home Directory
+
+New module `server/lib/claude-home.js` centralizes all Claude directory path logic:
+
+```
+CLAUDE_HOME env var (default: ~/.claude)
+  ├── projects/<encoded-cwd>/<session-id>.jsonl           ← main session transcript
+  │   (encoding rule: all non-alphanumeric chars → "-", e.g. "/Users/txj/.codefuse" → "-Users-txj--codefuse")
+  ├── projects/<encoded-cwd>/<session-id>/subagents/agent-<id>.jsonl  ← sub-agent transcript
+  │   (sub-agent ID format: ad18a79192af10ed1, acompact-f8427be966459435)
+  └── settings.json                                        ← hooks configuration
+```
+
+Existing hardcoded paths in `import-history.js`, `install-hooks.js`, and `settings.js` are migrated to use this module.
+
+---
+
+## API
+
+### GET /api/sessions/:id/transcripts (v2 new)
+
+List available transcript files for a session (main + sub-agents), scanned directly from the filesystem.
+
+**Response (200):**
+
+```json
+{
+  "transcripts": [
+    { "id": "main", "name": "Main Agent", "type": "main", "has_transcript": true },
+    { "id": "ad18a79192af10ed1", "name": "code-reviewer", "type": "subagent", "subagent_type": "code-reviewer", "has_transcript": true },
+    { "id": "acompact-f8427be966459435", "name": "Context Compaction", "type": "compaction", "has_transcript": true }
+  ]
+}
+```
+
+**Design notes:**
+
+- Bypasses database agent IDs; scans the filesystem directly for JSONL file short IDs
+- `id` field maps directly to the filename: `agent-<id>.jsonl`, used as the `agent_id` parameter for the `transcript` API
+- Compaction file name format: `agent-acompact-<hex>.jsonl`, id is `acompact-<hex>`
+- Attempts to read `.meta.json` in the same directory for agent type description
+- Falls back to scanning all `projects/` subdirectories when the exact encoded path doesn't exist
+
+### GET /api/sessions/:id/transcript
+
+Read a session's JSONL transcript file and return a structured message list.
+
+**Query Parameters:**
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `agent_id` | string | null | Transcript short ID (from `transcripts` endpoint); omit for main session |
+| `limit` | number | 50 | Max messages to return (max 200) |
+| `after` | number | null | Incremental mode: only return messages with JSONL line > after (v2 new) |
+| `before` | number | null | History mode: only return the latest N messages with JSONL line < before (v2 new) |
+| `offset` | number | 0 | Legacy pagination offset (compatible, mutually exclusive with after/before) |
+
+**Response (200):**
+
+```json
+{
+  "messages": [
+    {
+      "type": "user",
+      "timestamp": "2026-04-24T10:23:45Z",
+      "content": [
+        { "type": "text", "text": "Please implement the login feature" }
+      ]
+    },
+    {
+      "type": "assistant",
+      "timestamp": "2026-04-24T10:23:52Z",
+      "model": "claude-sonnet-4-6",
+      "usage": { "input_tokens": 1500, "output_tokens": 800 },
+      "content": [
+        { "type": "text", "text": "I'll help you implement the login feature." },
+        { "type": "thinking", "text": "Let me analyze the codebase..." },
+        {
+          "type": "tool_use",
+          "name": "Read",
+          "id": "toolu_abc123",
+          "input": { "file_path": "/src/auth.ts" }
+        }
+      ]
+    }
+  ],
+  "total": 120,
+  "has_more": true,
+  "last_line": 523,
+  "first_line": 474
+}
+```
+
+**v2 New Response Fields:**
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `last_line` | number | JSONL line number of the last message in the current response; used as the `after` parameter for incremental requests |
+| `first_line` | number | JSONL line number of the first message in the current response; used as the `before` parameter for history loading |
+
+**Loading Modes:**
+
+| Mode | Parameters | Behavior | Use Case |
+|------|-----------|----------|----------|
+| Default | No after/before/offset | Return the latest N messages | Initial load |
+| Incremental | `after=N` | Return messages with line > N (up to limit) | WebSocket-triggered new message loading |
+| History | `before=M` | Return the latest N messages with line < M | Scroll-up to load older messages |
+| Compatible | `offset=K` | Skip first K, return next N | Legacy pagination (kept for compatibility) |
+
+**Error Responses:**
+
+| Status | Condition |
+|--------|-----------|
+| 200 | When JSONL file doesn't exist, returns empty `{ messages: [], total: 0, has_more: false, last_line: 0, first_line: 0 }` |
+| 404 | Session ID not found in database |
+
+**Implementation Rules:**
+
+- Only extract entries with `type: "user"` or `type: "assistant"`; skip system/progress entries
+- Match `tool_use` and `tool_result` via `id` field; unpaired tool_use shows no result section
+- Truncate individual content exceeding 10KB, appending `[truncated]`
+- Re-read the file on every request (no server-side caching) to ensure real-time freshness
+- When `cwd` is null, scan all `projects/` subdirectories to find the JSONL for the sessionId
+- Internally use JSONL line numbers as cursors; remove the `line` field from responses, expose `first_line` / `last_line` to the client
+
+---
+
+## Frontend
+
+### SessionDetail Page Changes
+
+Replace the current flat layout with a **tabbed interface**:
+
+```
+[Agents]  [Conversation]  [Timeline]
+```
+
+- **Agents tab** — existing agent hierarchy tree (active by default)
+- **Conversation tab** — new conversation viewer
+- **Timeline tab** — existing event timeline
+
+### Conversation Tab Components
+
+**v2 Chat-flow architecture:**
+
+```
+ConversationView.tsx
+├── TranscriptSelector   — dropdown selector (v2 replaces AgentFilter)
+├── ScrollContainer      — scrollable message container
+│   ├── HistoryLoader    — scroll-up history loading indicator
+│   └── MessageList.tsx
+│       ├── UserMessage      — user message
+│       └── AssistantMessage
+│           ├── TextBlock    — plain text content
+│           ├── ThinkingBlock — collapsible thinking content
+│           └── ToolCallBlock — collapsible tool call + result
+│               ├── ToolUse      — tool name + parameters
+│               └── ToolResult   — execution result / error
+└── NewMsgButton         — "New messages" floating button (v2 new)
+```
+
+### TranscriptSelector (v2 replaces AgentFilter)
+
+- Top dropdown selector: `[Main Agent ▾]` or `[Context Compaction ▾]`
+- Data source: `GET /api/sessions/:id/transcripts` (filesystem scan, not database)
+- Reloads the corresponding transcript on switch
+- Only shown when transcripts > 1
+- Message count displayed alongside: `518 messages`
+
+### Chat-flow Behavior (v2 new)
+
+**Initial load:**
+- Call `transcript?limit=50` to get the latest 50 messages
+- Auto-scroll to bottom after rendering
+- Track `last_line` and `first_line` for subsequent requests
+
+**Real-time updates (WebSocket-driven):**
+- Subscribe to `eventBus` `new_event` events
+- Only process events where `session_id` matches the current session
+- On event, call `transcript?after=last_line&limit=50` for incremental loading
+- If user is at bottom (< 100px from bottom), auto-scroll to latest message
+- If user has scrolled up, show "New messages" floating button; click to scroll to bottom
+
+**Scroll-up history loading:**
+- Listen for scroll events; trigger when `scrollTop < 50` and `has_more` is true
+- Call `transcript?before=first_line&limit=50` to fetch older messages
+- Prepend to top of list; preserve scroll position via `scrollHeight` delta
+- Show spinner while loading; show "↑ Scroll up for older messages" hint at top
+
+**Key Refs:**
+- `lastLineRef` — tracks the JSONL line number of the newest message, used for incremental requests
+- `firstLineRef` — tracks the JSONL line number of the oldest loaded message, used for history loading
+- `scrollContainerRef` — scroll container DOM reference
+- `isAtBottomRef` — boolean flag tracking whether user is at the bottom
+
+### Message Rendering
+
+- **User messages**: right-aligned, blue background, display text content
+- **Assistant messages**: left-aligned, default background, including:
+  - Model name and token usage as faded metadata
+  - Text blocks rendered inline
+  - Thinking blocks: collapsed by default, click to expand (dimmed style)
+  - Tool calls: collapsed by default showing only tool name, click to expand:
+    - Tool name as header with icon
+    - Input parameters formatted as JSON (collapsible)
+    - Tool result with success/error indicator
+
+### Interaction Details
+
+- **Long text truncation**: content over 500 characters is truncated by default, with an "expand" link
+- **Lazy loading (v2)**: initial load of latest 50 messages; scroll-up auto-loads older 50; WebSocket-driven incremental append
+- **Real-time updates (v2)**: on WebSocket `new_event` with matching `session_id`, incrementally load new messages
+- **Auto-scroll (v2)**: auto-scroll to latest when user is at bottom; show floating "New messages" button when user has scrolled up
+- **Empty state**: when JSONL is missing or empty, show "No conversation records found."
+
+---
+
+## Server Module: claude-home.js
+
+```js
+// Centralized Claude home directory path management
+function getClaudeHome() {
+  return process.env.CLAUDE_HOME || path.join(os.homedir(), ".claude");
+}
+
+function getProjectsDir() {
+  return path.join(getClaudeHome(), "projects");
+}
+
+function getSettingsPath() {
+  return path.join(getClaudeHome(), "settings.json");
+}
+
+// Encoding rule: all non-alphanumeric characters replaced with "-"
+// Example: "/Users/txj/.codefuse" → "-Users-txj--codefuse"
+function encodeCwd(cwd) {
+  return cwd.replace(/[^a-zA-Z0-9]/g, "-");
+}
+
+function getTranscriptPath(sessionId, cwd) {
+  if (!cwd) return null;
+  const encoded = encodeCwd(cwd);
+  const candidate = path.join(getProjectsDir(), encoded, `${sessionId}.jsonl`);
+  if (fs.existsSync(candidate)) return candidate;
+  // Fallback: scan projects/ subdirectories
+  return findTranscriptPath(sessionId);
+}
+
+function getSubagentTranscriptPath(sessionId, cwd, agentId) {
+  if (!cwd) return null;
+  const encoded = encodeCwd(cwd);
+  const candidate = path.join(getProjectsDir(), encoded, sessionId, "subagents", `agent-${agentId}.jsonl`);
+  if (fs.existsSync(candidate)) return candidate;
+  // Fallback: scan all project directories
+  return findSubagentTranscriptPath(sessionId, agentId);
+}
+
+function findTranscriptPath(sessionId) {
+  // Fallback: when cwd is unknown, scan projects/ subdirectories
+  const projectsDir = getProjectsDir();
+  if (!fs.existsSync(projectsDir)) return null;
+  const dirs = fs.readdirSync(projectsDir, { withFileTypes: true });
+  for (const d of dirs) {
+    if (!d.isDirectory()) continue;
+    const candidate = path.join(projectsDir, d.name, `${sessionId}.jsonl`);
+    if (fs.existsSync(candidate)) return candidate;
+  }
+  return null;
+}
+
+// v2 new: support prefix fuzzy matching for compaction type
+function findSubagentTranscriptPath(sessionId, agentId) {
+  const projectsDir = getProjectsDir();
+  if (!fs.existsSync(projectsDir)) return null;
+  const dirs = fs.readdirSync(projectsDir, { withFileTypes: true });
+  for (const d of dirs) {
+    if (!d.isDirectory()) continue;
+    const subagentsDir = path.join(projectsDir, d.name, sessionId, "subagents");
+    if (!fs.existsSync(subagentsDir)) continue;
+    // Exact match
+    const exact = path.join(subagentsDir, `agent-${agentId}.jsonl`);
+    if (fs.existsSync(exact)) return exact;
+    // Prefix fuzzy match (compaction type: agentId starts with "acompact-")
+    if (agentId.startsWith("acompact-")) {
+      const files = fs.readdirSync(subagentsDir);
+      const match = files.find(f => f.startsWith("agent-acompact-") && f.endsWith(".jsonl"));
+      if (match) return path.join(subagentsDir, match);
+    }
+  }
+  return null;
+}
+```
+
+---
+
+## File Changes
+
+| File | Action | Description |
+|------|--------|-------------|
+| `server/lib/claude-home.js` | **New** | Claude home directory path management; v2 adds `findSubagentTranscriptPath` prefix fuzzy matching |
+| `server/routes/sessions.js` | Modified | v1: add `GET /sessions/:id/transcript`; v2: add `GET /sessions/:id/transcripts`, transcript endpoint gains `after`/`before` params and `first_line`/`last_line` response |
+| `scripts/import-history.js` | Modified | Use `getClaudeHome()` instead of hardcoded path |
+| `scripts/install-hooks.js` | Modified | Use `getSettingsPath()` instead of hardcoded path |
+| `server/routes/settings.js` | Modified | Use `getClaudeHome()` for hooks detection |
+| `client/src/lib/types.ts` | Modified | v1: add `TranscriptMessage`, `TranscriptContent`; v2: add `TranscriptInfo`, `TranscriptListResult`, `TranscriptResult` gains `last_line`/`first_line` |
+| `client/src/lib/api.ts` | Modified | v1: add `sessions.transcript()`; v2: add `sessions.transcripts()`, `transcript()` gains `after`/`before` params |
+| `client/src/pages/SessionDetail.tsx` | Modified | Add tab switching and Conversation tab; v2: remove `agents` prop from ConversationView |
+| `client/src/components/conversation/ConversationView.tsx` | **New** → v2 rewrite | v1: basic pagination; v2: chat-flow mode (WebSocket incremental + scroll-up history + auto-scroll) |
+| `client/src/components/conversation/MessageList.tsx` | **New** | Message list (with collapsible blocks, command formatting, skill content folding, task notification folding) |
+| `client/src/components/conversation/ToolCallBlock.tsx` | **New** | Collapsible tool call display |
+
+---
+
+## Error Handling
+
+| Scenario | Handling |
+|----------|----------|
+| JSONL file doesn't exist | Return `{ messages: [], total: 0, has_more: false, last_line: 0, first_line: 0 }`; UI shows "No conversation records found." |
+| JSONL line parse failure | Skip the line, continue processing remaining lines |
+| Single content exceeds 10KB | Truncate and append `[truncated]` marker |
+| Sub-agent JSONL doesn't exist | Same as main file — return empty list |
+| Session cwd is null | Use `findTranscriptPath()` to scan project directories |
+| CLAUDE_HOME path invalid | Log warning, return empty list |
+| Incremental load returns no new messages (v2) | `after` request returns empty array, frontend silently ignores |
+| History load failure (v2) | Silent failure, doesn't interrupt user experience |
+| WebSocket disconnection (v2) | Doesn't affect loaded messages; next event after reconnect triggers incremental load |
+
+## Edge Cases
+
+- **Compaction**: After `/compact`, older messages are lost from the JSONL. The viewer only shows what's currently in the file — this is expected behavior. Compact transcripts appear as separate entries in the transcript selector.
+- **Active sessions**: JSONL may be actively written to. Every request re-reads the file for real-time freshness. WebSocket events trigger incremental loading — no polling needed.
+- **Unpaired tool_use/tool_result**: Display the tool call without the result section; no error.
+- **Message order**: JSONL is ordered chronologically; responses preserve the same order (oldest first).
+- **Database ID vs file ID mismatch (v2)**: Database agent IDs use format `<sessionId>-jsonl-<shortId>`, but JSONL filenames use `agent-<shortId>.jsonl`. v2 bypasses database IDs entirely via the `transcripts` endpoint, which scans the filesystem and uses file short IDs.
+- **Compaction filename format (v2)**: In the database, compaction agent IDs use format `<sessionId>-compact-<uuid>`, but filenames use `agent-acompact-<hex>.jsonl`. `findSubagentTranscriptPath` supports prefix fuzzy matching for `agent-acompact-*.jsonl`.
+- **Scroll position preservation (v2)**: When loading history, the scroll position is preserved by computing the `scrollHeight` delta, ensuring the viewport content doesn't jump.
+- **Duplicate events (v2)**: WebSocket may send multiple `new_event` messages; incremental loading uses `after` line number for deduplication, preventing duplicate appends.
+
+## Testing Strategy
+
+| Layer | Test Content |
+|-------|-------------|
+| API unit tests | `GET /sessions/:id/transcript` — normal response, file not found, invalid session, pagination params, agent_id filtering |
+| API unit tests | `GET /sessions/:id/transcript` — v2: `after` incremental loading, `before` history loading, `first_line`/`last_line` response |
+| API unit tests | `GET /sessions/:id/transcripts` — v2: file scanning, compaction type, meta.json reading |
+| API unit tests | `claude-home.js` — path inference logic, env var override, fallback scanning, compaction prefix fuzzy matching |
+| Frontend component tests | `MessageList` rendering, `ToolCallBlock` collapse/expand, command formatting, skill content folding |
+| Frontend component tests | `ConversationView` — v2: initial load, incremental append, history load, scroll detection, new messages indicator |
+
+## Environment Variables
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `CLAUDE_HOME` | `~/.claude` | Claude Code home directory (e.g. `~/.codefuse/engine/cc/`) |
@@ -0,0 +1,220 @@
+# Design: Fix Agent-Monitor server memory leak
+
+- **Date**: 2026-05-22
+- **Author**: zhihua + Claude (brainstorming collaboration)
+- **Status**: Design Approved, pending implementation plan
+
+## Background
+
+After running `npm start` locally, the server process memory grows continuously over time and eventually exhausts host memory when combined with Claude / IDE / browser. The initial proposal was to deploy Agent-Monitor on a remote server and access it via the local browser, but investigation showed this only relocates the problem — the root cause is in the server itself, and a long-running remote instance will also OOM.
+
+This design focuses on **root-cause remediation**, not remote deployment. Once memory is stable post-fix, we can revisit whether remote deployment is still desirable.
+
+## Current diagnosis (with code evidence)
+
+Measured locally:
+
+| Metric | Value |
+|---|---|
+| `data/dashboard.db` | 192 MB |
+| `events` row count | 251,244 |
+| `sessions` count | 1130 (completed 1015 + abandoned 110 + active 5) |
+| Largest single event size | 369 KB |
+| `~/.claude/projects` | 58 MB |
+
+Three leak / performance sources were identified:
+
+### Leak #1: TranscriptCache entry has no per-entry size cap
+
+In `server/lib/transcript-cache.js`, every cache entry holds three push-only arrays:
+
+- `state.turnDurations.push(...)` (l.327)
+- `state.errors.push(...)` (l.332, 343)
+- `state.compaction.entries.push(...)` (l.315)
+
+`_merge()` incremental merging (l.456, 462, 468) likewise only pushes and never trims.
+
+`MAX_CACHE_ENTRIES = 200` bounds the number of entries, but **each entry is unbounded in size**. A long session emits one turnDuration per turn (~50 bytes), so a few thousand turns = MB-scale per entry; 200 entries × tens of MB = **multiple GB**.
+
+### Leak #2: `_set()` stores everything twice (per-entry memory doubled)
+
+`server/lib/transcript-cache.js:51-58`:
+
+```js
+this._set(key, {
+  errors: result?.errors ? [...result.errors] : null,           // top-level shallow copy
+  turnDurations: result?.turnDurations ? [...result.turnDurations] : null,
+  compaction: this._cloneCompaction(result.compaction),
+  ...
+  result,                                                        // contains references to the same fields
+});
+```
+
+The top-level fields are shallow-copied (`[...result.errors]`) new array objects that do not share references with `result.errors`. **Each array exists twice on the heap per cache entry.**
+
+### Performance issue: the sweep does a full scan over events
+
+`server/index.js:329` runs every 60-300s:
+
+```sql
+SELECT DISTINCT e.session_id, json_extract(e.data,'$.transcript_path') AS tp
+FROM events e JOIN sessions s ON s.id=e.session_id
+WHERE s.status='active' AND json_extract(e.data,'$.transcript_path') IS NOT NULL
+GROUP BY e.session_id ORDER BY MAX(e.id) DESC
+```
+
+Doing `json_extract` + DISTINCT + ORDER BY across 250k events rows produces large temporary SQLite memory spikes and is slow.
+
+## Goals and constraints
+
+**Goals**:
+
+1. Server process RSS stays stable over long runs (< 300 MB)
+2. No Agent log loss (events table remains complete; no retention)
+3. Reversible changes confined to `server/`; no changes to hook-handler / UI / WebSocket protocol
+
+**Non-goals** (explicitly out of scope):
+
+- Remote deployment
+- Events table retention / archival
+- DB engine swap / compression / sharding
+- UI / frontend / CLI changes
+
+## Design
+
+### Change A: TranscriptCache per-entry sliding window
+
+Add a configurable cap:
+
+```js
+const MAX_ARRAY_LEN = parseInt(process.env.TRANSCRIPT_CACHE_MAX_ARRAY_LEN, 10) || 1000;
+```
+
+After each push in `_streamRange` parsing (l.315/327/332/343 etc.) and in `_merge` incremental merging (l.456/462/468), trim immediately:
+
+```js
+if (arr.length > MAX_ARRAY_LEN) arr.splice(0, arr.length - MAX_ARRAY_LEN);
+```
+
+Applies to `turnDurations`, `errors`, `compaction.entries`, and `usageExtras.{service_tiers, speeds, inference_geos}` (these Set→Array conversions can also accumulate).
+
+**Why no data loss**:
+
+`routes/hooks.js:583, 633` already inserts `result.errors` / `result.turnDurations` into the events table on every hook trigger, with dedup (`SELECT 1 ... WHERE summary=?` / `WHERE created_at=?`). After cache truncation, the next hook re-reads the transcript file → dedup skips existing rows → only new rows are inserted. The events table stays complete.
+
+**Capacity estimate**:
+- 1 turn ≈ 50 bytes
+- 1000 turns = 50 KB / cache entry
+- 200 entries full ≈ 10 MB
+
+### Change B: Eliminate `_set()` double storage
+
+Simplify the cache entry shape:
+
+```js
+this._cache.set(key, { mtimeMs, size, bytesRead, result });
+```
+
+Drop all top-level `errors` / `turnDurations` / `compaction` / `usageExtras` / `tokensByModel` / `thinkingBlockCount` / `latestModel` fields. `_merge` computes via local variables and writes back only into `result`.
+
+**Expected effect**: ~50% memory reduction per entry.
+
+### Change C: Stop sweeping events for transcript_path
+
+**Schema migration** (`server/db.js`):
+
+```sql
+-- Add column (idempotent)
+ALTER TABLE sessions ADD COLUMN transcript_path TEXT;
+
+-- One-time backfill (runs once at startup, gated by a .migrations marker file to prevent reruns)
+UPDATE sessions SET transcript_path = (
+  SELECT json_extract(data,'$.transcript_path') FROM events
+  WHERE events.session_id=sessions.id
+    AND json_extract(data,'$.transcript_path') IS NOT NULL
+  LIMIT 1
+) WHERE transcript_path IS NULL;
+```
+
+Follow the idempotent migration pattern at `server/db.js:284` (the `agents_new` rebuild).
+
+**Write path** (`server/routes/hooks.js` `ensureSession`):
+
+When `transcript_path` is first seen, run `UPDATE sessions SET transcript_path=? WHERE id=? AND transcript_path IS NULL`.
+
+**Sweep query rewrite** (`server/index.js:329`):
+
+```sql
+SELECT id, transcript_path FROM sessions
+WHERE status='active' AND transcript_path IS NOT NULL
+```
+
+The query at `server/index.js:309` that fetches `transcript_path` on abandonment is also rewritten to read from the sessions table.
+
+**Complexity**: drops from O(total events rows) to O(active sessions ≈ single digits). **Not a single events row is removed.**
+
+## Verification strategy
+
+### Unit tests (new `server/__tests__/transcript-cache-bounded.test.js`)
+
+1. With `MAX_ARRAY_LEN=100`, feed 500 turns → `result.turnDurations.length === 100`, tail retained
+2. After cache truncation, re-extracting → events table dedup skips existing rows, insert count == 0
+3. Coarse memory assertion: 200 entries × 1000 turns, `process.memoryUsage().heapUsed` delta < 30 MB
+
+### Integration tests
+
+- `npm run test:server` green
+- `npm run test:client` green
+- `npm run mcp:typecheck` passes
+
+### Measurement script (one-off)
+
+New `scripts/memory-soak-test.js`:
+
+- Generate fake transcript jsonl with 10000 turns
+- Start the server, simulate 10 concurrent active sessions, fire a hook every 1s
+- Run for 30 minutes, log `process.memoryUsage().rss` per minute
+- Assert: RSS growth at minute 30 < 50 MB
+
+### Verification checklist (pre-merge)
+
+- [ ] Unit + integration tests green
+- [ ] `npm run mcp:typecheck` passes
+- [ ] Local `npm start` for 1h, `ps -o rss=` monitoring shows a flat curve
+- [ ] DB migration idempotent: two consecutive `npm start` runs without errors
+- [ ] Old DB (no `transcript_path` column) → migrate + backfill → sweep works
+- [ ] After cache truncation, the UI events list still shows all old turns/errors
+
+## Risks and rollback
+
+### Risk matrix
+
+| Risk | Likelihood | Impact | Mitigation |
+|---|---|---|---|
+| `MAX_ARRAY_LEN=1000` too small for ultra-long sessions | Low | Medium | Env var tunable to 5000-10000; events table is always complete, UI can still query |
+| Extra dedup SELECTs after cache truncation | Medium | Low | Sweep runs every 60-300s; an extra 100-1000 primary-key lookups per run is acceptable |
+| ALTER TABLE fails on old DB | Very low | High | Use the migration pattern at `db.js:284` — try-catch + column-existence check |
+| transcript_path backfill is slow due to events scan | Low | Low | One-time migration takes ~1s; use EXISTS subquery instead of join |
+| `_set()` shape change breaks other readers | Low | Medium | Grep the repo to confirm all external consumers of `extract()` only read `result.*` |
+
+### Rollback
+
+- All changes are confined to `server/`; **hook-handler / UI / WebSocket protocol untouched**
+- Rollback at any phase = `git revert` of the matching commit
+- DB schema: `ALTER TABLE ... ADD COLUMN` is not reversible, but an unread/unwritten new column is harmless; once code is reverted, sessions just has an extra empty column
+
+## Optional follow-ups (out of scope here)
+
+- Add a `(session_id, event_type, created_at)` composite index on events (UI query performance)
+- Add a `lastProcessedTurnTimestamp` cursor to the cache so `extract` only returns new turns (eliminates dedup SELECTs entirely)
+- `/api/internal/memory` diagnostic endpoint returning `cache.stats()` + `process.memoryUsage()`
+
+## Decision record
+
+| Option | Choice | Rationale |
+|---|---|---|
+| Remote deployment vs fix leak | Fix leak | Remote deployment relocates the problem; the leak hits remote too |
+| Permanent events retention vs retention policy | Permanent | Hard user constraint: guarantee Agent log integrity |
+| Truncate cache vs not truncate | Truncate to MAX_ARRAY_LEN | Events table already persists raw data; the cache is a derived view |
+| Delete events vs rewrite the sweep query | Rewrite the query | Satisfies the "no log loss" constraint |
+| Introduce LRU byte-budget instead of entry count | No | Entry-count cap + per-entry cap is already enough; byte accounting adds complexity |
@@ -0,0 +1,256 @@
+# Tabby — Floating Companion (Design Spec)
+
+**Date:** 2026-05-28
+**Status:** Approved (design) — pending spec review before planning
+**Owner:** Nguyễn Ngọc Trí Vĩ (David)
+**Topic:** A cute-but-functional cat companion that lives in the dashboard's bottom corner, reacts to live session events, and expands into a panel for status, quick actions, and asking questions.
+
+---
+
+## 1. Summary
+
+**Tabby** is a floating cat avatar pinned to the bottom-right corner of the Agent Dashboard on every route. It is two things at once:
+
+1. **A reactive mascot** — an SVG cat whose face, ears, eyes, and posture react in real time to what the monitored Claude Code sessions are doing (a session finishes → tail-up, eyes `^^`; an error/hook fails → arch + ears-back; idle → curls up asleep). Eyes track the cursor when alert.
+2. **An assistant** — click the avatar (or press `⌘B` / `Ctrl+B`) to expand a panel with a live status line, quick navigation actions, and an **Ask** box that answers simple questions from cached dashboard data, with a handoff to the existing **Run** page to ask Claude for real.
+
+The "do the job" path reuses what already exists: `POST /api/run` spawns a real `claude` subprocess and streams over WebSocket. Tabby does **not** introduce any new LLM backend, API key, or server route in P1/P2. P3 adds a single client-only deep-link prefill.
+
+Name **Tabby** matches the app's identity: this is a **Monitor** ("watching your agents"), and Tabby is the alert watcher curled in the corner.
+
+---
+
+## 2. Goals / Non-Goals
+
+### Goals
+- Delightful, on-theme personality layer over live session data — "cute but does the job."
+- Always-present, low-footprint corner avatar that auto-surfaces notable events as transient speech bubbles, then settles.
+- One-keystroke (`⌘B`) expand to a functional panel: status, quick actions, local Ask.
+- Reuse the existing event stream (`eventBus`) and Run flow — no new backend in P1/P2.
+- Fully consistent with the existing dark Tailwind theme (`surface-*`, `accent`, `border`).
+- Accessible: keyboard-operable, `aria-live` bubbles, honors `prefers-reduced-motion`.
+- Degrades safe: if WebSocket is down/delayed, Tabby shows a calm/disconnected state — never errors, never blocks the page.
+
+### Non-Goals (YAGNI)
+- No drag-to-reposition (fixed bottom-right).
+- No sound effects.
+- No new LLM/chat backend or API key (Ask is rule-based locally; real Claude = handoff to Run).
+- No server-side persistence (preferences in `localStorage` only).
+- No multi-avatar / skins / customization.
+- No changes to existing pages beyond the minimal mount + the P3 Run prefill.
+
+---
+
+## 3. Where it lives (architecture)
+
+```
+App.tsx
+  └─ useWebSocket(onMessage = eventBus.publish)   // single shared socket, already exists
+  └─ Layout.tsx
+        ├─ UpdateNotifier        (existing global floater)
+        ├─ Tabby   ◀── NEW: mounted here, sibling of UpdateNotifier
+        └─ <Outlet/>             (page routes)
+```
+
+- **Mount point:** `client/src/components/Layout.tsx`, right next to `<UpdateNotifier/>`. This guarantees Tabby persists across every route and shares the one WebSocket connection.
+- **Data source:** the existing `eventBus` (`client/src/lib/eventBus.ts`).
+  - `eventBus.subscribe(handler)` → every `WSMessage`.
+  - `eventBus.onConnection(handler)` + `eventBus.connected` → WS up/down.
+  - No prop drilling, no new context provider. The brain hook subscribes directly.
+- **Navigation:** quick actions use `react-router` (`useNavigate`) to jump to existing routes (`/sessions`, `/sessions/:id`, `/activity`, `/run`).
+
+### Component layout (new, isolated directory)
+
+```
+client/src/components/Tabby/
+  Tabby.tsx          # Container. Owns open/collapsed/muted state, ⌘B + Esc handlers,
+                     #   localStorage persistence. Composes the three presentational parts.
+  CatAvatar.tsx      # Pure presentational SVG cat. Props: { mood, eyeTarget, reducedMotion }.
+                     #   No data access — fully testable in isolation.
+  SpeechBubble.tsx   # Transient bubble. Props: { text, onDismiss }. aria-live="polite",
+                     #   auto-dismiss ~4.5s. No data access.
+  TabbyPanel.tsx     # Expanded panel: status header + quick actions + Ask box.
+                     #   Receives status summary + handlers as props.
+  useTabbyBrain.ts   # The brain. Subscribes eventBus → derives { mood, statusSummary,
+                     #   bubbleQueue }. Owns all timers (idle/sleep/stuck). The only unit
+                     #   that touches eventBus.
+  intents.ts         # Local Ask: maps a free-text question → templated answer from cached
+                     #   status, or a { runHandoff: prompt } signal. Pure function.
+  quips.ts           # mood/event → randomized phrase pool. The personality. Pure data + picker.
+  tabby.css          # Keyframes (breathe/blink/ear-twitch/arch/tail-flick), translucency,
+                     #   prefers-reduced-motion overrides.
+```
+
+**Boundaries / contracts:**
+- `useTabbyBrain` is the *only* unit that subscribes to `eventBus`. Everything else receives plain props. This keeps the live-data surface in one place and the rest trivially testable.
+- `CatAvatar`, `SpeechBubble`, `TabbyPanel` are pure presentational components — given props, render UI. No side effects.
+- `intents.ts` and `quips.ts` are pure functions over inputs — unit-testable with no DOM.
+
+---
+
+## 4. Data flow
+
+```
+server broadcast ──► useWebSocket ──► eventBus.publish ──► useTabbyBrain subscriber
+                                                                │
+                          (reduce WSMessage + timers into state)│
+                                                                ▼
+                                            { mood, statusSummary, bubbleQueue }
+                                                                │
+                 ┌──────────────────────────────┬──────────────┴───────────────┐
+                 ▼                                ▼                              ▼
+            CatAvatar(mood)              SpeechBubble(next bubble)        TabbyPanel(statusSummary)
+                                                                                │
+                                                                  quick action  │  Ask
+                                                                                ▼
+                                                              useNavigate(route)  |  intents() → answer
+                                                                                              | or → /run?prompt=
+```
+
+`useTabbyBrain` maintains a small in-memory model derived from the stream (it does not refetch):
+- `liveCount` — active sessions/agents currently working.
+- `errorCount` — sessions/agents in error since last clear.
+- `lastEventAt` — timestamp of most recent `new_event`/update (drives `stuck`/`sleeping`).
+- `connected` — from `eventBus.onConnection`.
+- `recentDone` — transient flag set on a `session_updated` → status `completed`, cleared after the happy animation.
+
+The exact `WSMessage.type` union the brain switches on (from `client/src/lib/types.ts`):
+`session_created`, `session_updated`, `agent_created`, `agent_updated`, `new_event`,
+`import.progress`, `update_status`, `run_stream`, `run_status`, `run_input_ack`, `cc_config_changed`.
+Tabby only cares about: `session_created`/`session_updated`/`agent_created`/`agent_updated` (mood + counts),
+`new_event` (activity heartbeat → `lastEventAt`, and hook-failure detection via the event payload),
+`run_status` (run finished → `happy`). The rest are ignored.
+
+These feed both the avatar mood and the panel's status line. Counts are best-effort from the stream; the panel may also read a one-shot from existing stats endpoints if needed for an accurate initial number (open item — see §10).
+
+---
+
+## 5. Mood state machine (rule-based brain)
+
+Mood is a pure function of `(streamModel, timers)`, evaluated on every relevant event and on timer ticks. **Highest-priority matching state wins:**
+
+| Priority | Mood | Trigger | Cat expression |
+|---------:|------|---------|----------------|
+| 1 | `disconnected` | WS down (`eventBus.connected === false`) | faded/desaturated, flat ears, still |
+| 2 | `worried` | `session_updated`/`agent_updated` with status `error`, or a hook-failure `new_event` | arch + puff, ears back, brow down, brief shake |
+| 3 | `stuck` | ≥1 live session AND `now - lastEventAt > STUCK_MS` | ears-up alert stare, `!` |
+| 4 | `happy` | `session_updated` → `completed`, or `run_status` finished (transient, ~4s) | tail-up, eyes `^^`, head-bob |
+| 5 | `thinking` | Ask in flight (panel) | head-tilt, `…` |
+| 6 | `watching` | ≥1 live session, recent activity | eyes track cursor, ears up, tail flick |
+| 7 | `sleeping` | no activity AND idle `> SLEEP_MS` | curled, eyes shut, `zzz` |
+| 8 | `idle` | default / fallback | slow blink, gentle breathe |
+
+Constants (tunable, defined in `useTabbyBrain`): `STUCK_MS` (~10 min), `SLEEP_MS` (~3 min). All timers cleared on unmount.
+
+**Event → mood mapping (concrete):**
+- `onConnection(true)` → recompute (leaves `disconnected`).
+- `onConnection(false)` → `disconnected`.
+- `session_updated` data.status `error` → `worried` (+ increment `errorCount`).
+- `session_updated` data.status `completed` → `happy` (transient) + decrement `liveCount`.
+- `session_created` / `session_updated` data.status `active` → `watching`, recompute `liveCount`.
+- `agent_updated` status `error` → `worried`.
+- `new_event` → refresh `lastEventAt`; hook-failure event types (confirm in build, see §10) → `worried`.
+- `run_status` finished → `happy` (transient).
+- (timers) inactivity → `stuck` (if live) or `sleeping` (if not).
+
+---
+
+## 6. Eyes & motion
+
+- **Eye tracking (`watching`/`idle`):** pupils follow the mouse, clamped inside the eye socket via a small vector-normalize + clamp. Throttled (rAF or ~30ms) to stay cheap.
+- **On event:** eyes glance toward the bubble, then relax back to tracking.
+- **Ears/tail/body:** CSS keyframe animations in `tabby.css`, swapped by a `data-mood` attribute on the avatar root.
+- **`prefers-reduced-motion`:** static eyes (centered), no breathe/shake/arch — mood still conveyed via static pose + face. Detected via `matchMedia`, passed as `reducedMotion` prop.
+
+---
+
+## 7. Auto-surface (speech bubbles)
+
+- Pipeline: event → `quips.pick(mood/event)` → enqueue bubble → show ~4.5s → dismiss → settle.
+- **Rate limit:** at most one bubble every few seconds; coalesce bursts ("3 sessions finished" instead of three bubbles).
+- **Mute toggle:** persisted in `localStorage`. Muted = no bubbles, but faces/animations still react. Toggle lives in the panel.
+- **Accessibility:** bubble container is `aria-live="polite"` so screen readers announce notable events without stealing focus.
+
+Example quips (from `quips.ts`, randomized):
+- happy: "session wrapped 🐾", "nice, that one's done", "4m12s — clean run"
+- worried: "ow, an error", "a hook tripped — peek?"
+- stuck: "this one's been quiet a while…", "still chewing on something?"
+- sleeping: "zzz", "wake me if something happens"
+
+---
+
+## 8. Panel (click / ⌘B)
+
+Opens as a small card anchored above the avatar. Themed with `surface-3`/`border`/`accent`.
+
+**Status header:** `🐾 N live · M errored · ●connected` (from brain's `statusSummary`; `●` reflects WS state, colored by health).
+
+**Quick actions** (each = `useNavigate` to an existing route, or a local toggle):
+- Jump to errored session → `/sessions/:id` (most recent error) or `/sessions?status=error`.
+- Active sessions → `/sessions` (or `/activity`).
+- **Run Claude** → `/run`.
+- Activity feed → `/activity`.
+- Mute / unmute bubbles (local toggle, persisted).
+- Clear alerts (reset `errorCount`).
+
+**Ask box:**
+- P1/P2: `intents()` matches the query against a small set of local intents over cached status — e.g. *what's running*, *any errors*, *how many today*, *slowest* — and returns a templated answer rendered in the panel.
+- Unmatched query → offer: "Ask Claude directly?" → opens `/run?prompt=<query>` (P3).
+
+**Dismiss:** `Esc`, click-outside, or re-press `⌘B`.
+
+---
+
+## 9. Phasing
+
+### P1 — Mascot (delight, zero backend)
+- `CatAvatar.tsx` (full SVG + all moods + eye tracking + reduced-motion).
+- `useTabbyBrain.ts` (eventBus subscription, mood machine, timers, bubble queue).
+- `SpeechBubble.tsx`, `quips.ts`, `tabby.css`.
+- `Tabby.tsx` container mounting avatar + bubble; `⌘B` reserved but panel stubbed.
+- Mounted in `Layout.tsx`.
+- **Outcome:** living, reacting cat in the corner with auto-bubbles. No panel yet.
+
+### P2 — Panel (functional)
+- `TabbyPanel.tsx`: status header + quick actions (router nav) + local Ask.
+- `intents.ts` local intent matching.
+- `localStorage` for `collapsed` + `muted`; mute/clear in panel.
+- `Settings.tsx`: a single on/off toggle for Tabby (persisted), read by `Tabby.tsx`.
+- **Outcome:** click/⌘B opens a useful panel; Ask answers from local data.
+
+### P3 — "Do the job" handoff
+- `Run.tsx`: read `?prompt=` search param → `setPrompt(prefill)` on mount (mirrors the existing `?session=` pattern). Client-only, no server change.
+- Wire Ask's unmatched-query path → `/run?prompt=<query>`.
+- **Outcome:** Tabby can hand a real question to a real `claude` subprocess via the existing Run flow.
+
+---
+
+## 10. Open items (resolve during planning/build)
+1. **Accurate initial counts:** the stream gives deltas; on first mount counts are unknown until events arrive. Decide: (a) start at 0 and let the stream fill in (simplest), or (b) one-shot read from the existing stats endpoint for an accurate seed. Leaning (a) for P1, optional (b) in P2 panel.
+2. **Hook-failure detection:** confirm which `event` `eventType` values represent hook failures vs. normal lifecycle, so `worried` only fires on real problems. Verify against `server/routes/hooks.js` + DB event types during build.
+3. **Errored-session deep link:** confirm `/sessions` supports a `status=error` query or whether to navigate to the specific `/sessions/:id`.
+
+---
+
+## 11. Theme & accessibility notes
+- Colors strictly from existing tokens: `surface-0..5`, `border`/`border-light`, `accent`/`accent-hover`. Cat palette: warm accent-tinted body that reads on the dark `surface-0` background; soft glow via `accent-muted`.
+- Fonts inherit (`Inter` / `JetBrains Mono`) — bubble/status text uses existing classes.
+- Keyboard: `⌘B`/`Ctrl+B` toggle, `Esc` close, panel actions tab-focusable.
+- `prefers-reduced-motion`: disables continuous animation.
+- z-index above content, below modals; never traps focus when collapsed.
+
+---
+
+## 12. Verification (per CLAUDE.md)
+- **Frontend:** `npm run test:client`.
+  - Unit tests for `useTabbyBrain` mood transitions (each event → expected mood, priority ordering, timer-driven `stuck`/`sleeping`).
+  - Unit tests for `intents()` (known queries → templated answers; unknown → runHandoff).
+  - Unit test for `quips.pick` (returns a string for every mood).
+- **No server change in P1/P2** → `npm run test:server` not required for those phases. P3 touches only `Run.tsx` (client) → still client-only; run `test:client`.
+- Manual: load dashboard, trigger a run, observe mood/bubble transitions; toggle reduced-motion; toggle mute; ⌘B/Esc.
+
+---
+
+## 13. File change summary
+**New:** `client/src/components/Tabby/{Tabby,CatAvatar,SpeechBubble,TabbyPanel}.tsx`, `client/src/components/Tabby/{useTabbyBrain.ts,intents.ts,quips.ts,tabby.css}`, plus `__tests__` for brain/intents/quips.
+**Edited:** `client/src/components/Layout.tsx` (mount, P1) · `client/src/pages/Settings.tsx` (on/off toggle, P2) · `client/src/pages/Run.tsx` (`?prompt=` prefill, P3) · i18n files (`tabby:*` keys, as strings are added).
@@ -0,0 +1,74 @@
+# Stage auto-detection — design
+
+**Status:** approved 2026-07-28. Sub-project B of three (C = worktree lanes, shipped on `feat/worktree-lanes`; A = merged Workspace page, next). Built after C because C settled the lane data model.
+
+## Problem
+
+A lane's stage only moves when the driving agent calls `ccam stage <name>`. Every un-instrumented session — which is most of them — sits at `idle` forever while its agent works, so the pipeline map shows nothing. The dashboard already receives every tool call the agent makes; it just never reads them.
+
+## Goal
+
+Infer a lane's stage from the hook stream it already ingests, and show it **without ever claiming it as evidence**.
+
+## The rule that shapes everything
+
+**Inference never renders green.** A node the dashboard inferred reaches `passed-no-evidence` (amber) at most; `done` requires a declared stage carrying evidence via `ccam stage --evidence`. If inference could paint a node green, the amber/green distinction — the reason this feature exists — would be worthless.
+
+Declared always outranks detected. A lane that has declared `review` ignores a detection for `implement`.
+
+## Signals that actually exist here
+
+Verified against a real 121 MB install before designing:
+
+| Source | Rows | Notes |
+|---|---|---|
+| `events.tool_name` | Bash 29 470, Read 20 236, Edit 4 678, Write 1 330, Agent 1 188, TaskUpdate 908, Skill 127 | the bulk of the signal |
+| `events.data.tool_input` | present on every `PostToolUse` | full Bash command strings, Edit/Write paths, Skill names |
+| `TodoWrite` | **0** | this Claude Code build uses `TaskCreate`/`TaskUpdate` instead — do not design around TodoWrite |
+
+So the signal is `tool_name` plus a regex over `tool_input`. No model call, no extra query.
+
+## Where the rules live
+
+In the pipeline template, not in code. `server/data/pipelines/default.json` gains an optional `detect` array per node:
+
+```json
+{ "id": "tests", "detect": [{ "tool": "Bash", "match": "\\b(npm (run )?test|pytest|vitest|jest|go test|cargo test)\\b" }] }
+```
+
+A rule is `{tool, match?}`: `tool` matches `events.tool_name` exactly; `match` is a regex tested against a flattened string of the tool's input. A rule with no `match` fires on the tool alone. A custom template supplied through `DASHBOARD_PIPELINES_DIR` may define its own rules, so a team can teach the dashboard their own conventions without touching the code.
+
+## Where it runs
+
+`server/lib/stage-detect.js` exposes one pure function, `detect(pipeline, event) → {nodeId, signal} | null`. It is called from the existing fail-safe block in `touchLaneFromHook` (`server/routes/hooks.js`) that already resolves the lane — no new pass over the hook path, no new query, and the same swallow-everything guarantee, because a hook must never fail on account of bookkeeping.
+
+## Anti-flapping
+
+- **Forward only.** A detection whose node index is not greater than the current detected index is dropped. Reading a file after editing it must not pull a lane back to `plan`.
+- **Write only on change.** Bash alone accounts for 29 470 rows in a real install; the lane row is written only when the detected node actually advances.
+- **Declared wins.** If the lane's declared stage sits at or beyond the detection, nothing is written.
+
+## Data model
+
+Three additive columns on `lanes`, each behind its own `try { SELECT col } catch { ALTER }` probe so a partial migration self-heals: `detected_stage`, `detected_signal`, `detected_at`. `stage` keeps its exact current meaning — the declared stage.
+
+`lanePayload` gains `detected_stage`, `detected_signal`, and a per-node `detected: boolean` inside `pipeline_nodes`.
+
+## What the user sees
+
+A detected node renders amber with a **dashed** border, distinguishing it from an amber solid node (declared without evidence). Its tooltip names the signal: `tests ← npm run test:server`. The lane card shows `auto: tests` when the detection is ahead of the declaration. The header line still shows the declared stage, because that is what the agent asserted.
+
+## Deliberately not in scope
+
+- No inference of `done`, and no inference of gate results. A gate is a judgement; only an agent may claim one.
+- No back-filling of history. Detection starts when this ships; existing lanes gain nothing retroactively.
+- No inference from `workflows.phases`. It exists and would work, but it covers only Workflow-tool runs and would need its own reconciliation with the declared stage — a separate feature if wanted.
+- No writing to `stage`. Ever. Detection lives in its own columns so that turning the feature off loses nothing.
+
+## Testing
+
+- Rule matcher: one test per shipped rule, plus a rule with no `match`, an invalid regex in a template (must be skipped, not crash the hook), and a tool the rules do not mention.
+- Monotonic guard: an out-of-order detection is dropped; a same-node detection writes nothing.
+- Declared precedence: a lane declared at `review` ignores an `implement` detection.
+- Fail-safety: a malformed event cannot throw out of the hook path.
+- **Inference never renders green:** given a lane with only detections and no declarations, no node in `pipeline_nodes` may have state `done`. This is the test that guards the feature's whole premise.
@@ -0,0 +1,59 @@
+# Merged Workspace page — design
+
+**Status:** approved 2026-07-28. Sub-project A of three (C = worktree lanes, shipped; B = stage detection, planned). Built last because it consumes both.
+
+## Problem
+
+Lanes and Run are two pages that describe the same activity. `Run` spawns a `claude` process in a directory and streams its output; `Lanes` shows what a lane is doing and where it is in its pipeline. A user watching an agent work has to hold both in their head, and the lane card cannot even send a prompt — `start` opens a promptless conversation run and `message` has no input field.
+
+## Goal
+
+One page. A lane strip across the top, the selected lane's pipeline beneath it, and that lane's Claude console below — with every capability the Run page has today.
+
+## Decisions already taken
+
+- **The merged page lives at `/run`.** `/lanes` redirects there. One sidebar entry.
+- **Every run belongs to a lane.** Choosing a working directory that no lane owns creates one (`kind='adopted'`) rather than running loose. A lane is, after all, just a working directory the dashboard is watching.
+- **Everything from Run survives:** slash-command autocomplete in the prompt editor, model / permission-mode / effort selectors, the token meter with cost, run history, and attach-to-a-live-run.
+- **Layout:** lane strip (horizontal, scrollable, with the counters and Add) → pipeline map of the selected lane → console. Selecting a lane switches both the pipeline and the console.
+
+## Architecture
+
+`client/src/pages/Run.tsx` is 3658 lines holding an envelope model, a merge/typewriter engine, a slash-autocomplete prompt editor, a token meter, cwd suggestions, run history and the page shell. It is extracted into pieces that the new page composes:
+
+| Unit | Responsibility |
+|---|---|
+| `client/src/hooks/useRunStream.ts` | envelope state for one run id: subscribe `run_stream` / `run_status` / `run_input_ack`, merge envelopes, typewriter |
+| `client/src/components/run/RunConsole.tsx` | render the envelope stream, the prompt editor with slash autocomplete, the token meter, stop/clear |
+| `client/src/components/run/RunSetup.tsx` | mode / model / permission / effort / cwd / resume pickers, binary status, the limitations banner |
+| `client/src/components/run/RunHistory.tsx` | past runs, live runs, attach |
+| `client/src/pages/Workspace.tsx` | lane strip + `PipelineMap` + the three above |
+
+**The extraction is mechanical and must not change behaviour.** Each unit moves in its own commit with the existing Run tests passing untouched except for import paths. Only once `Run.tsx` is a thin composition does the new page get built. Extraction and composition never share a commit — that is the difference between a reviewable refactor and an unreviewable rewrite.
+
+## Server glue
+
+Four small pieces, each independently useful:
+
+1. **Runs start through the lane.** The UI always calls `POST /api/lanes/:id/start`, which already exists, sits behind the same-origin guard, and records `run_id` on the lane. `POST /api/run` stays for the CLI and other callers; the UI simply stops using it. Lane `start` gains `mode` so a headless one-shot is still possible.
+2. **`POST /api/lanes/ensure`** — `{cwd, title?}` returns the lane owning that path or creates an `adopted` one. Avoids the UI having to catch a 409 and re-read, and keeps the create-then-start pair from racing.
+3. **`dashboard_runs.lane_id`** — one additive column, set when a run is started through a lane, so history can be filtered per lane instead of guessed at by `cwd`.
+4. **A finished run releases its lane.** Today nothing clears `lanes.run_id` when a run ends on its own: the lane reads `running` forever and `message` keeps targeting a dead run. The run-spawner already knows the moment of exit (`actualExitedAt`, added on the worktree branch); on that event, clear the owning lane's `run_id` and set its status back to `idle`. This is a bug the merge exposes rather than causes.
+
+## What the console must not do
+
+**It never touches the lane's stage.** Typing `/code-review` in the UI does not move the lane to `review`; only `ccam stage` declares, and only detection (sub-project B) infers. The console is a window onto a process, not a driver of the pipeline. Keeping that boundary is what stops the pipeline from becoming a lie.
+
+## Risks and how they are contained
+
+- **The extraction is the whole risk.** 3658 lines, one of them the typewriter engine, with a screens snapshot over the page. Containment: one unit per commit, tests untouched but for imports, snapshot diffs read rather than regenerated, and the composition deferred until the last extraction is green.
+- **Two consoles for one lane.** Only one run is live per lane (`start` 409s when one exists), so the console shows exactly one stream.
+- **A lane created just to try a command** leaves an `adopted` lane behind. Acceptable: `adopted` lanes are never destroyable, forgetting one is a click, and the alternative — runs that belong to nothing — is what this design set out to remove.
+
+## Testing
+
+- Each extraction: the existing Run tests pass with only import changes, and the screens snapshot for `/run` is unchanged until the page itself changes.
+- `useRunStream`: envelopes merge in order; a `run_status` terminal event stops the stream; the subscription is disposed on unmount.
+- `POST /api/lanes/ensure`: returns the existing lane for a path already owned, for a path nested inside one, and creates exactly one lane under concurrent calls.
+- Run-exit releases the lane: after a run ends by itself, the lane's `run_id` is null and its status is `idle`.
+- The console does not move the stage: after a full run through the console, the lane's `stage` is what it was.
@@ -0,0 +1,133 @@
+# Worktree-backed lanes + Shipyard-style lifecycle — design
+
+**Status:** approved 2026-07-28. Sub-project C of three (B = stage auto-detection, A = merged Workspace page) — each gets its own spec, plan and execution cycle. C is being built first because it fixes the lane data model that the other two build on.
+
+## Problem
+
+A lane today is a pointer at a directory that already exists. Two agents working in parallel therefore work in the *same* checkout and collide — the exact failure Shipyard solves by giving every lane its own clone. CCAM has no provisioning at all: no way to create a lane's working copy, no way to reset it between features, no way to remove it, and no way to reclaim the database a finished lane leaves behind (a real install reached 121 MB).
+
+## Goals
+
+- A lane can own a **git worktree** that CCAM creates, resets and removes.
+- The lifecycle verbs mirror Shipyard's, because that vocabulary is proven: `add`, `clear`, `reset`, `remove`, plus `purge` as CCAM's analogue of Shipyard's per-lane `dropdb`.
+- Every destructive action is confirmed **against counted facts**, not adjectives.
+- Directories the user already had must be impossible for CCAM to destroy.
+
+## Non-goals
+
+- No dependency bootstrap. A fresh worktree has no `node_modules`, no `.env`, no `.claude/settings.local.json` — all gitignored. Shipyard solves this with per-project `bootstrap`/`migrate`/`seed` hooks, which is a whole subsystem. Out of scope; documented as a limitation.
+- No per-lane ports, databases, or Docker services.
+- No orchestration. Unchanged from the existing feature: the driving Claude session declares its own stage.
+- `VACUUM` is not part of `purge` (see Database reclamation).
+
+## Data model
+
+Additive columns on `lanes`, each guarded by the repo's `try { SELECT col } catch { ALTER }` probe:
+
+| column | meaning |
+|---|---|
+| `kind` | `adopted` \| `managed`. Default `adopted`, so every pre-existing row migrates into the safe class. |
+| `source_repo` | absolute path of the checkout a managed worktree was created from |
+| `base_branch` | the branch the worktree was cut from, e.g. `development` |
+| `slug` | sanitised from the title; used for both the directory and the branch name |
+
+`branch`, `cwd`, `stage`, `stages` and the rest keep their current meaning. `cwd` stays `UNIQUE`.
+
+Environment: `LANES_ROOT` (default `~/.claude/ccam-lanes`), `LANE_BASE_BRANCH`, `LANE_BRANCH_PREFIX` (default `feat/`).
+
+Layout: worktree at `$LANES_ROOT/<repo-basename>__<slug>`, branch `<prefix><slug>`. Numbered `lane1..lane9` slots were considered and rejected — CCAM is multi-repo, and slot numbers carry no meaning without Shipyard's per-lane ports.
+
+## Safety model
+
+`adopted` lanes expose no destructive verb. No reset, no remove-with-worktree, no branch deletion. The UI hides those controls; the API refuses them.
+
+A `managed` lane may be destroyed only when **all three** independent checks pass:
+
+1. `kind === 'managed'`
+2. the lane's `cwd`, fully resolved (symlinks included), lies inside `LANES_ROOT`
+3. `git worktree list --porcelain` run in `source_repo` actually lists that path
+
+Shipyard gets away with one check (`case "$DIR" in */lane$N`) because its directory names are fixed. Dropping numbered slots costs that guarantee, so three cheaper checks replace it. Every destructive function in `server/lib/worktree.js` re-runs the three checks itself rather than trusting its caller.
+
+CCAM never runs `rm -rf` on a lane. Removal goes through `git worktree remove`; if git refuses, the error surfaces unchanged.
+
+## Verbs
+
+| verb | steps | destructive |
+|---|---|---|
+| `add` | resolve base → `git worktree add -b <prefix><slug> <dir> <base>` → insert lane row `kind=managed` | no |
+| `adopt` | today's `POST /api/lanes` — point a lane at an existing directory, `kind=adopted` | no |
+| `clear` | reset stage/status fields only (already implemented) | no |
+| `reset` | kill and await the run → `git fetch origin --prune` → checkout base → `reset --hard <base>` → `clean -fd` → delete the feature branch → recreate it from base → clear lane state | **yes** |
+| `remove` | kill and await the run → unlock if locked → `git worktree remove --force` → `git worktree prune` → delete the branch → delete the lane row | **yes** |
+| `purge` | delete the lane's sessions and their events plus the orphan `token_usage` rows | **yes** |
+
+`clean -fd` deliberately omits `-x`, exactly as Shipyard does: gitignored files (`node_modules`, `.env`) survive a reset, untracked-but-not-ignored files do not.
+
+Branch deletion never touches `main`, `master`, or the lane's `base_branch`, and only runs after the worktree holding that branch is gone.
+
+## Preflight
+
+`GET /api/lanes/:id/preflight?action=reset|remove|purge` returns counted facts, never prose:
+
+```json
+{
+  "action": "reset",
+  "lane": 3, "branch": "feat/criteria-form", "kind": "managed",
+  "dirty": 4, "untracked": 11, "unpushed": 2,
+  "head": "9b3e74a",
+  "blocked": ["unpushed-commits"],
+  "warnings": ["no-remote"]
+}
+```
+
+For `purge`: `sessions`, `events`, `tokenRows`, `bytesEstimate`, and `activeSessionSkipped`.
+
+The confirmation modal renders those numbers. The action then re-verifies: the client echoes back the `head` and counts it was shown, and the server returns `409` if they moved. `unpushed > 0` blocks `reset`, and `remove` when a managed worktree is actually at risk, unless the request carries `{force: true}`. `unpushed` counts what the action would really discard — with no remote that is `<base_branch>..HEAD`, this lane's own work, not the repository's whole history.
+
+## Concurrency
+
+One mutex per lane serialises destructive actions, mirroring `repo_lock()` in the AutomaticWorkflow bot — where concurrent git operations on a shared checkout produced a real `git checkout` exit 128, not a theoretical one. A destructive action first kills the lane's run and **awaits its exit** before touching git.
+
+`add` can take seconds on a large repo, so it returns `202` with `status=provisioning` and finishes in the background, broadcasting `lane_update` on completion — the same pattern Shipyard uses to drive its spinner.
+
+## Edge cases and their resolutions
+
+- **Branch already exists:** if unused, `git worktree add` without `-b`; if checked out elsewhere, refuse and name the other path. Slug collisions get a `-2` suffix.
+- **Base branch missing on origin:** resolve `origin/<base>` → local `<base>` → the source repo's current HEAD.
+- **Source repo is itself a worktree, or bare:** works; git resolves through `--git-common-dir`.
+- **Repo with no commits:** `worktree add` fails and the lane lands `failed` with git's stderr in `notes`. Preflight does not pre-empt it — the lane is created first, then provisioning reports the git failure, and the row is forgotten with `DELETE /api/lanes/:id`.
+- **Worktree directory deleted by hand:** the lane reports `missing` and only `remove` is offered, taking the prune path.
+- **`cwd` uniqueness:** `remove` deletes the row, so re-adding the same slug is clean; `reset` keeps the path.
+
+## Database reclamation
+
+`events` cascades from `sessions`, but `token_usage` has no foreign key — `purge` must delete those rows explicitly or leave orphans. The currently-live session is never purged.
+
+SQLite does not shrink on `DELETE`. `purge` runs `DELETE` plus `PRAGMA optimize` and reports the reclaimable size; `VACUUM` is a separate, explicitly-labelled maintenance action because it locks the whole database for seconds. Hiding a database-wide lock inside a button labelled "clean up" would be a trap.
+
+## Surfaces
+
+**API:** `POST /api/lanes/worktree` (add), `GET /api/lanes/:id/preflight`, and `reset` / `purge` joining the existing `POST /api/lanes/:id/:action` set, all behind the existing same-origin guard.
+
+**CLI:** `ccam lanes add --repo <path> [--title <t>] [--base <branch>]`, `ccam lanes reset|remove|purge <id> [--force]`.
+
+**UI:** a `managed` / `adopted` badge on the lane card; destructive buttons rendered only for `managed`; the existing `ConfirmModal` showing the preflight table.
+
+## Testing
+
+Against a real git repository fixture created in a temp directory — no mocks, because every bug worth catching here lives in git's actual behaviour:
+
+- worktree created, listed, removed, pruned clean
+- each of the three safety refusals: an `adopted` lane, a path outside `LANES_ROOT`, a path git does not list as a worktree
+- `reset` keeps gitignored files and removes untracked ones
+- the unpushed-commit guard blocks, and `force` overrides it
+- preflight's counts equal what the action actually changes
+- `purge` removes sessions, events and token rows, and skips the live session
+- `add` returns 202 and broadcasts `lane_update` when provisioning finishes
+
+## Known limitations
+
+- A fresh worktree has no installed dependencies or local env files (see Non-goals).
+- Nine worktrees of a large repository cost nine working trees of disk; the `add` preflight estimates the size first.
+- `--repo` may point anywhere the user can read. That is their own machine; validation is limited to "absolute, exists, is a git repo", and every route stays behind the loopback guard.
@@ -0,0 +1,140 @@
+# Workspace UI rebuild — design
+
+**Status:** approved 2026-07-29. Sub-project D, built on top of A (merged Workspace page).
+Reference: the Shipyard "Feature Harness" screen the user supplied.
+
+## Problem
+
+The merged Workspace page shipped with the right information and the wrong shape.
+Everything a lane knows — declared stage, inferred stage, progress, liveness,
+needs-you — is already on the card (`client/src/components/lanes/LaneCard.tsx`)
+and already correct on the wire. None of it is legible: the lane strip is a
+horizontal scroller of cramped cards, the pipeline sits above a console that
+dominates the viewport, and the `auto: <stage>` chip that proves detection works
+is 10px of amber text nobody sees.
+
+Measured on the live install while writing this: lane 5 carried
+`detected_stage: "tests"` with a real signal, and the user's report was
+"the lane does not auto-detect". Detection was never broken. The display was.
+
+Two facts also make lanes look emptier than they are:
+
+- `branch` and `ci_status` are columns nobody writes, so those rows are always
+  blank even for a managed worktree sitting on a real branch.
+- Detection is forward-only with no expiry, so a lane parks at the highest stage
+  it ever touched. Lane 5 reached `tests` and can never show `implement` again,
+  even while the agent is editing code.
+
+## Goal
+
+The reference screen's legibility, on CCAM's real data: a lane's state readable
+from across the room, the pipeline large enough to trace, and the console present
+but out of the way until wanted.
+
+## Decisions taken
+
+- **Card grid, not a strip.** Responsive 1 / 2 / 3 columns.
+- **The console collapses.** It keeps every capability from A; it starts
+  collapsed and opens for the selected lane. Watching lanes is the default
+  posture, driving one is the exception.
+- **Only real data.** No placeholder tiles for facts CCAM does not have
+  (tickets, preview ports, per-lane credentials). Branch/commit/CI are added
+  because they can be read for real — see below.
+- **Detection expires.** A detection older than a TTL stops holding the floor.
+
+## Layout
+
+Top to bottom, one column:
+
+```
+header:    title · [N lanes][N running][N need you][N dead] · [+ Add lane]
+detail:    selected lane · declared + inferred headline · large PipelineMap · legend
+console:   collapsed by default; expands to RunSetup + RunConsole + RunHistory
+grid:      lane cards, 1/2/3 columns
+```
+
+Selecting a card switches the detail panel and the console together, exactly as
+A wired it. The console is unchanged behind its new disclosure — no prop of
+`RunConsole`, `RunSetup` or `RunHistory` moves.
+
+## The card
+
+Reference layout, CCAM's fields, nothing invented:
+
+| Row | Content | Source |
+|---|---|---|
+| header | `LANE <id>` · liveness dot · status | `id`, `liveness`, `status` |
+| title | title, falling back to `cwd` | existing |
+| progress | declared stage chip · bar · `%` · time on stage | `stage`, `progress`, `stage_seconds` |
+| inferred | dashed amber `auto: <stage>` with the signal as tooltip | `detected_stage`, `detected_signal` |
+| tags | `kind` (adopted/managed), CI when known | `kind`, `ci_status` |
+| git | branch · short head · last commit subject · dirty/untracked counts | new, see below |
+| alert | needs-you banner | `needs_action` |
+| actions | start · stop · clear · reset · remove | existing lane actions |
+
+`reset` and `remove` keep their preflight + `expect` echo through
+`DestructiveLaneModal`. This redesign does not touch the destroy guard.
+
+## Git facts
+
+A new read-only endpoint, `GET /api/lanes/:id/git`, returning
+`{branch, head, subject, dirty, untracked}` or `{available: false}` when the
+lane's `cwd` is not a git repo or is unreadable.
+
+Deliberately **not** folded into `GET /api/lanes`: that payload is polled and
+broadcast, and shelling out to git once per lane on the hot path would put a
+subprocess burst behind every hook-driven `lane_update`. The card fetches its
+own facts when it mounts and on a slow interval, and renders without them until
+they arrive.
+
+`server/lib/worktree.js` already has `statusCounts(dir)` returning
+`{dirty, untracked, head}` and a `git()` wrapper that scrubs the inherited
+`GIT_*` environment. Both are reused as-is; the endpoint adds only the branch
+name and the commit subject. No second git helper, no shell strings.
+
+## Detection expiry
+
+`recordDetection` gains one rule: a `detected_stage` whose `detected_at` is
+older than `DETECTION_TTL_MS` (default 30 minutes) no longer blocks a new
+detection — the forward-only comparison is skipped and the fresh signal wins.
+Within the window nothing changes: forward-only and declared-wins hold exactly
+as they do today.
+
+This keeps the anti-flapping property that motivated forward-only (a `Read`
+right after an `Edit` must not drag the lane backwards) while admitting the
+thing it got wrong: a work session ends, and the next one starts somewhere else
+in the pipeline.
+
+**Unchanged, and not negotiable:** detection still never writes `lanes.stage`,
+and an inferred node still never renders `done`.
+
+## Signal legibility
+
+`detected_signal` currently captures the whole flattened tool input, so the chip's
+tooltip reads `cd /very/long/path && npm run test:server 2>&1 | grep …`. The
+matcher already knows which regex fired; the signal becomes the matched span plus
+a little context rather than the entire command. Cosmetic, but it is the text the
+tooltip exists to show.
+
+## Risks
+
+- **The console's disclosure is the only structural risk.** Mounting it inside a
+  collapsed container must not unmount `useRunStream` and lose a live stream.
+  The subscription stays mounted; only the visual container collapses.
+- **Git calls per card.** Bounded by the number of lanes on screen and a slow
+  refresh; failure is silent and the card renders without those rows.
+- **The screens snapshot over `/run` will change.** It is read, not regenerated
+  blindly.
+
+## Testing
+
+- The card renders every field from a fixture lane, and renders without the git
+  block when the endpoint reports `available: false`.
+- A detected node still never carries `data-state="done"` — the premise guard
+  from sub-project B is re-asserted at the new layout.
+- Collapsing and expanding the console does not tear down the run subscription:
+  a stream envelope delivered while collapsed is present when it re-expands.
+- `GET /api/lanes/:id/git` returns the facts for a real repo fixture and
+  `available: false` for a plain directory, and never shells out through a shell.
+- A detection older than the TTL is accepted even when it is behind the current
+  `detected_stage`; one inside the window is still refused.