feat: Claude Code Monitor — lanes, pipelines and a merged workspace
Internal SmartGift build of a Claude Code monitoring dashboard. Lanes: a durable unit of parallel agent work, one per working directory, tracked across session restarts. Managed lanes are git worktrees the dashboard provisions and can reset or remove behind a three-check destroy guard and a counted preflight; adopted lanes are directories you already own and are never destroyable. Pipelines: a lane moves through pipeline stages. A stage the agent declares with evidence renders green; a stage inferred from the tool-event stream renders dashed amber and never counts as done. Detection is forward-only within a 30-minute window, and never writes the declared stage. Workspace: one page at /run with a lane grid, the selected lane's pipeline, and a full Claude console behind a disclosure.
This commit is contained in:
@@ -0,0 +1,53 @@
|
||||
# SDD ledger — plan: docs/superpowers/plans/2026-07-27-lanes-pipeline.md
|
||||
|
||||
Base commit: 8e41e80 (branch feat/lanes-pipeline)
|
||||
|
||||
Task 1: review — spec MET, quality SOUND. 1 Important, 1 Minor.
|
||||
Task 1: ruled — Important (.gitignore `data/` -> `/data/`) does NOT enter fix loop. Plan's Global Constraints say "Preserve existing behavior. Additive schema only"; a gitignore anchor changes no behavior, and the reviewer's "purely additive" alternative edits the same file AND cannot work alone (git cannot re-include a path under an excluded directory without also negating the parent). `/data/` is the correct minimal fix. Stands.
|
||||
Task 1: minor (deferred): lanes-lib.test.js test title says "amber" where it means "without evidence" (wording inherited from the plan).
|
||||
Task 1: complete (commits 8e41e80..8b9e477, review clean after ruling)
|
||||
|
||||
Task 2: review — spec MET except one deviation; all 5 load-bearing behaviors verified correct. 1 Important, 1 Minor.
|
||||
Task 2: ruled — Important (lane SQL prepared inline in lanes.js instead of added to db.js `stmts`) does NOT enter fix loop: the plan contradicted itself (File Structure said "stmts entries", the Task 2 code block and its Interfaces line "Owns all SQL for lanes" say inline). The code block is authoritative; the stale File Structure line has been corrected in the plan so later reviews do not re-raise it.
|
||||
Task 2: minor (deferred): lanes-lib.test.js @file comment still says it only covers pipelines.js.
|
||||
Task 2: complete (commits 8b9e477..f29d904, review clean after ruling)
|
||||
|
||||
Task 3: review — spec MET; 1 "Critical" (WS payload asymmetry) ruled plan-mandated, 3 Important, 3 Minor.
|
||||
Task 3: ruled — delete broadcasting `{removed: id}` instead of `{lane}` is deliberate and consumed by Task 7's Lanes.tsx; the plan's Produces bullet was stale prose and has been corrected. Not a defect.
|
||||
Task 3: ruled — per-lane `SELECT MAX(created_at)` stands: events(session_id) is indexed and lanes number in the dozens (one per worktree), not thousands.
|
||||
Task 3: ruled — broadcast-before-response stands: `broadcast()` in server/websocket.js is already defensive.
|
||||
Task 3: minor (deferred): double payload() call per mutation (broadcastLane re-reads the lane); lanes-api.test.js tests share state via an outer laneId.
|
||||
Task 3: fix round 1/5 (2 addressed, 0 open — 409 now branches on SQLITE_CONSTRAINT_UNIQUE with message fallback; new WS test asserts lane_update on create and delete; commits 8e76c90..76c6f26)
|
||||
Task 3: complete (commits a69b3d8..76c6f26, review clean)
|
||||
|
||||
Task 4: review — spec FAIL (1 Critical: needs_action cleared by any session), 2 Important, 1 Minor.
|
||||
Task 4: ruled — the Critical was real. The plan's own contract said "cleared on the next non-Notification hook for that lane"; amended to "from the session currently bound to that lane, evaluated before rebinding" so two agents sharing one worktree cannot cancel each other's "needs you".
|
||||
Task 4: fix round 1/5 (3 addressed, 0 open — clear now gated on the pre-existing lane.session_id; cross-session + default-message tests added, verified to fail against the buggy code; per-hook lane-scan ceiling documented, no cache; commits 83aa655..3f7121d)
|
||||
Task 4: complete (commits 76c6f26..3f7121d, review clean) — 780 server tests pass
|
||||
|
||||
Task 5: first attempt reported DONE_WITH_CONCERNS claiming the sandbox blocks loopback — WRONG, and nothing was committed (pre-commit test gate held). Real cause: the plan's own test harness used blocking spawnSync while the test HTTP server ran in the same process, so the event loop stalled and the CLI child's request was never served. Verified loopback works between processes (detached node server + curl + separate node fetch, all exit 0). Plan's code block corrected to async spawn.
|
||||
Task 5: review — spec PASS, 2 Important (ccam lanes add untested; temp DB leaked), 3 Minor.
|
||||
Task 5: fix round 1/5 (2 addressed, 1 open — lanes-add test added, health poll replaced the 100ms sleep, DB cleanup added but not exception-safe; commits 19dd31b..3deb420)
|
||||
Task 5: fix round 2/5 (1 addressed, 0 open — after() teardown wrapped in try/finally; commit 35138b4; verified inline)
|
||||
Task 5: complete (commits 3f7121d..35138b4, review clean) — 783 server tests pass. `ccam lanes add` was an authorised addition so the CLI's empty-state hint names a command that exists.
|
||||
|
||||
Task 6: review — spec MET; security verified adversarially (cross-origin POST to /:id/start returns 403; prompt travels via stdin, model/effort/resumeSessionId are separate argv with no shell; /:id/stage not captured by /:id/:action; unknown action rejected before any lookup or mutation). 1 Important, 2 Minor.
|
||||
Task 6: fix round 1/5 (2 addressed, 0 open — message now consults getRun and returns 409 for a recorded-but-dead run instead of 500; unknown-action test asserts stage/status unchanged; new 409 test; commits f225fe8..b3f9e2f)
|
||||
Task 6: minor (deferred): lane.cwd is not re-validated at action time — a vanished directory fails cleanly at spawn.
|
||||
Task 6: complete (commits 35138b4..b3f9e2f, review clean) — 788 server tests pass
|
||||
|
||||
Task 7: review — 4 Critical, 2 Important, 2 Minor. Three of the Criticals were the PLAN's fault (it mandated wrapping LaneCard in a <button>, an unconfirmed remove, and hardcoded English strings). Review was right; plan was wrong.
|
||||
Task 7: fix round 1/5 (7 addressed, 0 open — selection is now a keyboard-operable role="button" div with stopPropagation on actions; remove goes through the existing ConfirmModal; every string i18n'd with real zh/vi/ko translations; Lanes case added to screens.snapshot.test.tsx; unknown-lane lane_update refetches so counters stay server-truthful; amber-vs-green test now asserts the colour tokens; start's empty-prompt behaviour documented in a tooltip; commits f29f597..a05065d)
|
||||
Task 7: minor (deferred): the Lanes screen snapshot captures the empty state only — a populated card + pipeline map is not snapshotted.
|
||||
Task 7: minor (deferred): the card cannot send a prompt — driving a lane from the UI needs a prompt/message input; today `start` opens a promptless conversation run and `message` has no input field. Follow-up feature, not a defect.
|
||||
Task 7: complete (commits b3f9e2f..a05065d, review clean) — verified by controller: 279/279 client tests, `npm run build` clean
|
||||
|
||||
Task 8: review — every documented command/flag/env var/endpoint/state-rule fact-checked against the shipped code and correct, EXCEPT one Critical: docs advertised `ccam lanes add --pipeline <id>` which the CLI never parsed.
|
||||
Task 8: ruled — fix the CODE, not the docs: a pipeline template you cannot select from the CLI is a template nobody uses, and the server already accepted the field. Authorised a scoped code change inside the docs task.
|
||||
Task 8: fix round 1/5 (1 addressed, 0 open — --pipeline parsed and forwarded only when provided, usage string updated, two tests via a DASHBOARD_PIPELINES_DIR fixture with cleanup, docs parenthetical corrected; commits 6cca59c..3b196ef)
|
||||
Task 8: minor (deferred): README VN/CN/KO mirrors are now behind the English README; the repo's update-project-docs convention expects them synced.
|
||||
Task 8: complete (commits a05065d..3b196ef, review clean) — 790 server tests pass
|
||||
|
||||
Final whole-branch review (Opus): READY — no Critical/Important/Minor findings. Cross-cutting types, CLI-vs-server cwd resolution, WS payload branches, migration safety on an existing DB, the same-origin guard on the spawning routes, and both confirmation gates all verified. All 8 deferred/parked items triaged acceptable-to-defer, none load-bearing.
|
||||
Controller end-to-end smoke test (real server, real CLI, real hook POSTs) — lane created, `ccam stage plan --evidence` then `ccam stage review` reported: node states came back intake=passed-no-evidence, plan=done, implement/tests=passed-no-evidence, review=current, rest=pending; progress 57%; stage_seconds live; `ccam lanes` table and counters correct; a Notification hook bound the lane and a later hook from the newly-bound session cleared needs_action exactly as the amended contract specifies.
|
||||
Side effect found and REVERTED: starting the fork's server auto-configured Claude Code hooks in ~/.claude/settings.json (8 entries pointing at this fork). Removed surgically; the pre-existing `rtk hook claude` PreToolUse entry was left untouched. Polluted copy kept at /tmp/settings.before-ccam-cleanup.json.
|
||||
@@ -0,0 +1,31 @@
|
||||
# SDD ledger — plan: docs/superpowers/plans/2026-07-28-stage-detection.md
|
||||
|
||||
Base commit: d0044d1 (branch feat/stage-detection)
|
||||
|
||||
Task 1 (B1): written by Codex terra (foreground; four earlier background runs were killed by the harness's background wall-clock limit). Review — spec MET, all 9 scenarios covered by 5 tests; totality verified by a reviewer that actually threw null/numbers/arrays/circular refs/symbols/malformed rules/a throwing property getter at it (zero throws), and regex compilation confirmed once-per-pipeline (1 RegExp construction across 1000 events, WeakMap keyed on pipeline identity). 2 Important, 1 Minor.
|
||||
Task 1: fix round 1/5 (3 addressed — the signal was unbounded at 5002 chars for a real Bash command and is now whitespace-collapsed and capped at 120 with an ellipsis; `flattenInput` had concatenated EVERY top-level string, so a real Edit event's `old_string`/`new_string` (whole code blocks) would have been matched against and then shown as the reason for the inference — now restricted to an allowlist of identifying fields; compileRules gained direct totality tests. Totality re-verified by the fixer with its own script.) A Codex attempt at this fix hit the 540 s wall and produced nothing, so it was done by a Claude subagent.
|
||||
Task 1: complete (869 server tests) — commits 6bb445e + the fix commit
|
||||
Tooling note: Codex completed 3 of 10 attempted runs in this environment; when it finishes, its work is good (it was the only implementer that ran clause-deletion experiments unprompted), but each failure costs ~9 minutes, so implementation moved to Claude subagents. The useful half of the Codex protocol was kept for every implementer: the agent runs only its own focused test file and does NOT commit; the controller runs the full suite and commits, so the 861-test pre-commit hook runs once per task instead of twice.
|
||||
|
||||
Task 2 (B2): rules on plan/implement/tests/review/ship in the default template (intake/gate/done deliberately bare — a gate is a judgement, `done` is a claim); three per-column probes for detected_stage/detected_signal/detected_at; `recordDetection` with the forward-only + declared-wins guard; `detected` decorating nodeStates' output without touching how any state is computed.
|
||||
Task 2: the implementer found an integration bug in the PLAN, not in its own work, and flagged it instead of fixing outside its file list: `loadAll()` in pipelines.js normalises each node to {id,label,icon,gate,aliases} and DROPPED the new `detect` array, so every rule shipped in the template was inert. Controller reproduced it directly — raw template matched `tests`, `getPipeline('default')` matched null. Neither B1's review nor B2's own tests could have caught it: B1 tested the matcher with raw fixtures that bypass the loader, and B2 tested the rules as JSON. Fixed by preserving `detect` defensively in the loader, plus an END-TO-END test that calls detect(getPipeline('default'), event) through the real loader — the pin that was missing.
|
||||
Task 2: controller verified through the real path afterwards: rules live, an Edit's signal is its file_path (not the code from old_string/new_string), `gate` still uninferrable, and getPipeline() still returns a stable object so stage-detect's WeakMap regex cache keeps compiling once per pipeline.
|
||||
Task 2: complete (878 server tests)
|
||||
|
||||
Task 3 (B3): detect + recordDetection + broadcast placed inside the existing fail-safe try/catch in touchLaneFromHook, right after the lane is resolved; broadcast only when recordDetection reports written:true, with a comment naming the 29,470-Bash-event volume behind that rule. 5 new API tests including the premise guard at the API level.
|
||||
Task 3: controller smoke-tested on a real server. First attempt looked like a total failure (detected_stage null) — cause was NOT the code: a server from an earlier branch still held port 4820, so the new instance refused to start and the probes hit stale code. After stopping it: detected_stage=tests with the right signal; 15 identical events plus one backward Edit left updated_at completely unchanged (write-on-change holds under real traffic); the backward Edit did not drag the lane back; every node still `pending` with a `detected` flag and no node `done`.
|
||||
Task 3: complete (883 server tests)
|
||||
|
||||
Task 4 (B4): detected nodes render amber-dashed via a class that REPLACES the state-driven one, so even a hypothetical {detected:true, state:"done"} payload renders amber and never green — the safe direction. `auto:` chip only when the detection leads the declaration. tsc (not vitest) caught a missing field in an unrelated test fixture, which is why `npm run build` is in every client task's verification list.
|
||||
Task 4: complete (308 client tests)
|
||||
Task 5 (B5): `ccam lanes` gains a detected suffix using the same lead-comparison as the card, so terminal and browser never disagree; docs/LANES.md documents the signals, the field allowlist, the 120-char cap, the rules node by node, the deliberate blanks on intake/gate/done, forward-only + write-on-change, and the evidence boundary with its reason. The implementer flagged rather than papered over the fact that GET /api/lanes has NO OpenAPI path docs at all and no Lane schema — it added a standalone schema for the three fields instead of inventing either.
|
||||
Task 5: complete (885 server tests)
|
||||
|
||||
FINAL whole-branch review (Opus): READY WITH FIXES — 1 Critical, 5 Important, 7 Minor. The evidence boundary itself held: no lane state could be constructed in which a detection renders green or writes lanes.stage, proved by mutation and live probing rather than by reading, with per-event cost measured (1 µs for an ordinary Bash event, zero queries when nothing changes).
|
||||
FINAL Critical — clearLane() did not reset the detection columns. A lane reset back to base kept claiming `auto: ship` with 7 of 8 nodes amber for an empty tree, and then refused every subsequent detection with `behind-detected` — and since no node past `ship` carries a rule, detection was dead for that lane permanently.
|
||||
FINAL corrected MY OWN RULING: I had called the ENOLANE race in the hook path a non-issue "because the outer try/catch swallows it". Right conclusion, wrong reasoning — entering the catch skips the rest of the function, so `needs_action` was left lit and a rebound `session_id` unwritten. The fix is ordering/isolation, not error handling.
|
||||
FINAL found a second inert rule, the same defect class as the loader bug: `plan`'s `Write docs/.*plan.*\.md` was shadowed by `implement`'s unconditional `Write` because detect() takes the LAST match, so writing a plan document reported `implement`. Documented as live, could never fire.
|
||||
FINAL proved the boundary was unpinned: removing `&& !stages[n.id]` from withDetected left 56/56 tests passing, because both tests calling themselves PREMISE GUARD sat on lanes with no declarations, making "no node is done" vacuously true.
|
||||
Fix wave (one commit, 9da58ea, Opus): all 9 items fixed with seven verify-by-deletion experiments and their exact failure messages. It DECLINED my instruction to move the detection block below the bookkeeping, correctly: that block ends in `if (!Object.keys(patch).length) return;`, so detection would have become unreachable on the common hook. It used an inner try/catch instead and corrected the docs sentence. `implement`'s Write rule is now constrained with a negative lookahead, verified at the boundaries (mydocs/, docs2/ still implement).
|
||||
Fix-wave re-review: READY. Re-ran the G-1 mutation itself in a throwaway copy (32 pass / 1 fail, same tally), rebuilt the I-2 resolution table independently, and traced the alias mechanism to confirm the I-3 test would fail against the old code. All three of the fixer's open items judged non-blocking.
|
||||
B complete: 10 commits c78e7f7..9da58ea, 892 server tests, 308 client tests.
|
||||
@@ -0,0 +1,59 @@
|
||||
# SDD ledger — plan: docs/superpowers/plans/2026-07-28-worktree-lanes.md
|
||||
|
||||
Base commit: 7195741 (branch feat/worktree-lanes)
|
||||
|
||||
Task 1: first attempt committed with --no-verify after misdiagnosing a hook failure as "test isolation". Controller reproduced it via `git commit --amend`: git hooks export GIT_DIR/GIT_INDEX_FILE, every git child inherited them, and in a worktree `.git` is a FILE so `.git/index` gave ENOTDIR. Real bug in worktree.js, not the environment. Fixed by scrubbing 9 GIT_* vars (+ GIT_TERMINAL_PROMPT=0) in the git() helper and the test fixture, with a regression test that sets bogus GIT_DIR/GIT_INDEX_FILE. Recommitted through the hook.
|
||||
Task 1: review — spec MET (one accepted deviation: reset-in-place instead of delete+recreate); adversarial checks all passed: path containment defeats symlink/prefix/`..`/missing-path attacks via realpath on both sides + path.relative boundary; branch deletion cannot be tricked into main/master/base even by a lying lane row; env scrub complete; no vacuous tests. 2 Important.
|
||||
Task 1: fix round 1/5 (2 addressed, 0 open — ERESETBRANCH verifies the worktree really landed on the feature branch; ENOBASE verifies the base ref before ANY mutation, with a test proving the worktree and its dirty files are untouched on that path; commits 436adfa..03a6e63)
|
||||
Task 1: complete (commits 7195741..03a6e63, review clean) — 801 server tests pass, every commit through the pre-commit gate
|
||||
|
||||
Task 2: review — spec met on the surface, 2 Critical underneath. (a) the migration probed only `kind` while adding four columns, so a crash after the first ALTER left three columns permanently missing on a real install — the plan's own fault; (b) `kind` was validated in createLane only, so updateLane could silently corrupt the boundary that decides whether CCAM may delete a directory. Plus 3 Important (lock Map never pruned, two missing tests).
|
||||
Task 2: fix round 1/5 (5 addressed, 0 open — per-column independent probes that self-heal a partial migration; shared validateKind() used by both create and update, rejecting before any write so a mixed patch cannot half-apply; lock entry deleted when its chain settles if still current; regression tests for update-kind, per-lane (not global) locking, and an old-schema database plus a simulated mid-migration crash; commits fdf4da4..6138944)
|
||||
Task 2: complete (commits 03a6e63..6138944, review clean) — 808 server tests pass
|
||||
|
||||
Task 3: review — read-only confirmed, shapes/route order correct. 3 Important: purge WHERE clause duplicated between counter and deleter; the purge test could not fail (every session it created was `active`, so all three exclusions were deletable with the test still green); `unpushed: 0` was a confident lie when no upstream is configured — the exact under-report the preflight exists to prevent.
|
||||
Task 3: fix round 1/5 (2 addressed, 1 NOT — shared `purgeCandidateSessions()` now the single expression of the rule; `unpushedCount` counts commits on no remote via `--not --remotes` and surfaces a distinct `no-remote` fact; but the purge test got WEAKER, not stronger: the implementer deleted the sessions entirely and asserted zeros against an empty table, while its report claimed it had added direct DB inserts. Controller verified: zero INSERTs in the file. commits 51c1c4c..85aa098)
|
||||
Task 3: fix round 2/5 (1 addressed, 0 open — handed to Codex gpt-5.6-terra effort medium, which built the discriminating fixture (counted / active / bound / sibling-prefix sessions + seeded events and token rows), ran all three clause-deletion experiments and reported the failure each produced; commit d247c37)
|
||||
Task 3: controller re-verified independently — removed the bound-session exclusion by hand, test failed `2 !== 1`, file restored, `git diff server/lib/lanes.js` empty. 7/7 lifecycle tests, 815 server tests.
|
||||
Task 3: complete (commits 6138944..d247c37, review clean)
|
||||
Tooling note: Codex cannot run through `codex-rescue` here — the subagent's Bash sandbox makes Codex's own bwrap fail with `loopback: Failed RTM_NEWADDR`. Working invocation, with the user's explicit approval to drop the sandbox for it: `codex exec --dangerously-bypass-approvals-and-sandbox -m gpt-5.6-terra -c model_reasoning_effort=medium "<prompt>"` run from the controller's Bash with dangerouslyDisableSandbox.
|
||||
|
||||
Task 4: implemented by Codex gpt-5.6-terra (effort medium) via `codex exec`, review by Claude. Review — no Critical; same-origin guard genuinely applied, slug cannot escape LANES_ROOT, failure path leaves the lane managed/failed/removable with git's stderr and no orphan directory (reviewer reproduced it against a real repo). Codex also updated ARCHITECTURE.md, docs/API.md, server/README.md and added an OpenAPI fragment — ruled NOT scope creep: .claude/skills/update-project-docs mandates exactly those files for an API change. 1 Important, 3 Minor.
|
||||
Task 4: fix round 1/5 (4 addressed, 0 open — boot sweep marks any still-'provisioning' lane failed with an explanatory note (a lane killed mid-provision could otherwise never age into 'dead', since a fresh managed lane has no session_id for classifyLiveness to measure); 409 EDUPCWD and the `base` default now documented in both OpenAPI and docs/API.md; directory-suffix loop capped at 50 with a 409 and a test that pre-creates exactly the 51 colliding directories needed to reach it; commits d524cd3..cf8ebbf)
|
||||
Task 4: re-review traced the boot ordering — the sweep runs as a microtask off `server.listen`'s resolve, before the poll phase can dispatch a connection, and is one synchronous better-sqlite3 UPDATE with no yield point, so no request can slip a new provisioning lane into the sweep. Confirmed it writes only status/notes/updated_at, leaving kind/cwd/branch/source_repo/base_branch/slug intact.
|
||||
Task 4: minor (deferred): recoverInterruptedProvisioning writes raw SQL instead of going through updateLane/PATCHABLE; the boot handler logs err.message without the stack.
|
||||
Task 4: complete (commits d247c37..cf8ebbf, review clean) — 821 server tests pass
|
||||
|
||||
Task 5: implemented by Codex terra, reviewed by Claude Opus. Review — guard work (three checks, force gating, purge honesty, error mapping, lock) clean, but 2 Critical: (a) "kill the run and await its exit" was vacuous — killRun sets status='killed' synchronously after SIGTERM, so the poll on status returned instantly and `git clean -fd` / `worktree remove --force` ran milliseconds later while a live Claude could still be writing into that directory; (b) `remove` had been silently narrowed to managed lanes, breaking the shipped Remove button for adopted lanes with no client path at all (api.ts had no DELETE method), and the existing test that encoded the old contract was rewritten instead of the regression being reported.
|
||||
Task 5: fix round 1/5 (7 addressed, 2 NEW breakages — added `actualExitedAt`, written only in the child's exit handler, polled with a 7.5 s deadline (> killRun's 5 s escalation) and failing loudly via ERUNTIMEOUT before any git; restored `remove` for both kinds (managed tears down the worktree, adopted deletes the row only); made `expect` mandatory with the full field set; guarded DELETE; ESTALE now carries expected/current. commits 24841d4..58a94e8)
|
||||
Task 5: fix round 2/5 (4 addressed, 0 open — the mandatory `expect` (my requirement) had broken the Remove button for EVERY lane with 400 EEXPECT, so the client now fetches preflight and echoes it, and surfaces errors instead of swallowing them; a child that fails to spawn emits only `error`, never `exit`, so `actualExitedAt` also set there — otherwise `claude` missing from PATH froze every destructive action for the whole reap window; killRun's SIGKILL escalation tested `!child.killed`, which Node sets true on a successful SIGTERM, so it could never fire — now keyed on real exit; PATCH guarded. commits 58a94e8..fa77725)
|
||||
Task 5: controller committed one leftover line Codex left uncommitted (`preflight: r({})` in the screens-snapshot API mock, cac0d2a) and re-ran the client suite from a clean tree: 279/279.
|
||||
Task 5: minor (deferred to C7): LaneCard has no Force affordance, so an unpushed reset/remove 409s with no way to retry from the UI; reset and purge are not on the card at all yet; adopted `remove` still demands force when the adopted directory has unpushed commits even though nothing is destroyed.
|
||||
Task 5: complete (commits cf8ebbf..cac0d2a, review clean) — 834 server tests, 279 client tests
|
||||
|
||||
Task 6: written by Codex terra, whose process was killed by a wall-clock limit TWICE before it could run a single test or commit — the work survived staged. A Claude subagent then read the staged diff, verified it and committed it unchanged (00fc9fa). So the code reached review having never been run by its author; the reviewer was told that and given permission to run the CLI test file itself.
|
||||
Task 6: review — no Critical, no Important. Verified field-by-field that the CLI's LANE_PREFLIGHT_FIELDS matches the server's expectedFields exactly and that `expect` is built from the freshly fetched preflight (not client-guessed); `--yes` cannot be bypassed (returns with exit 1 before any POST is constructed); `--force` only ever unlocks the unpushed gate; the adopted-lane refusal is plain language and its test asserts the file's CONTENTS survive; no spawnSync, no bare sleeps, no vacuous tests. Reviewer independently ran the CLI suite: 10/10.
|
||||
Task 6: minor (deferred): `ccam help` shows `[--yes]` while the prose shows it unbracketed; no test covers the ESTALE 409 or the provisioning failed/timeout print paths; the destructive tests share a mutable `managedLane` across describe blocks.
|
||||
Task 6: complete (commits cac0d2a..00fc9fa, review clean) — 839 server tests
|
||||
|
||||
Task 7: Codex wrote the component work and was killed by a wall-clock limit a third time, leaving it uncommitted and missing both the modal tests and all the documentation; a Claude agent audited it, wrote those, and committed (2a58f5a).
|
||||
Task 7: that agent also found and fixed a real pre-existing bug OUTSIDE its brief: client/src/i18n/index.ts never registered the `lanes` namespace, so no string on the Lanes page had ever resolved — every locale, including the vi/ko/zh translations shipped in the earlier plan's Task 7, rendered as raw keys. Both that task's reviewer and its re-reviewer had passed i18n as complete; both had only checked that the locale files contained the keys, never that the namespace was loaded.
|
||||
Task 7: review — spec MET throughout; docs fact-checked line by line against the code (three safety checks, `clean -fd` without `-x`, preflight field lists, bytesEstimate derivation) and found truthful, including correctly documenting that LANE_BASE_BRANCH / LANE_BRANCH_PREFIX do NOT exist rather than inventing support. 1 Critical.
|
||||
Task 7: Critical (open, goes to C8) — `no-remote` sits in the same `blocked[]` array as hard blockers, so a managed lane in a repo with no remote can never be reset or removed from the UI: unpushedCount counts every commit when there is no remote, giving `blocked = ["unpushed-commits","no-remote"]`, the modal treats anything but `unpushed-commits` as a hard block, and its Force checkbox only appears when `blocked.length === 1`. The server gates nothing on `no-remote` and the CLI works fine — the UI locks only itself, permanently, for a common case.
|
||||
Task 7: minor (goes to C8): LaneCard renders `t("status.<status>")` but no locale has any `status.*` key, and i18next returns the key on a miss, so the `||` fallback never fires — every lane shows literal text like `status.active` right now.
|
||||
Task 7: complete (commits 00fc9fa..2a58f5a, 1 Critical carried into C8) — 287 client tests, 840 server tests
|
||||
|
||||
Task 8 (added after Task 7's Critical): implemented by a Claude subagent. Root-cause fix rather than a patch — `blocked[]` had been conflating three kinds of thing, so it now carries only the action-preventing conditions (adopted, missing, unreadable) plus `unpushed-commits` (the one Force overrides), while purely informational facts (`no-remote`) moved to a new `warnings[]`. The modal needed no logic change at all: it was locked out purely because the server mislabelled `no-remote` as blocking. Also landed: status.* i18n keys for all four locales, real support for LANE_BASE_BRANCH and LANE_BRANCH_PREFIX (the spec had promised them, the code had hardcoded main and feat/), CLI tests for the ESTALE print path and for failed provisioning, and `--yes` presented as mandatory everywhere.
|
||||
Task 8: review — APPROVE, no Critical or Important. Reviewer enumerated every blocked[] combination the server can emit and confirmed the modal and the server now agree on all eight; ran the ESTALE test three times (12/12 each, ~300 ms) and confirmed it is structural rather than timing-based, since GET /preflight takes no lock while POST does; verified the env-var tests actually set the variables and observe their values rather than asserting a default that would pass anyway; fact-checked the docs against the code and found no inaccurate sentence.
|
||||
Task 8: minor (deferred, for final-review triage): `lanes.status` is not validated at the API layer — `POST /:id/stage` and `ccam stage --status <s>` accept an arbitrary string, so a rogue value would reproduce the raw-key badge bug this task fixed. The robust fix is a t() defaultValue in LaneCard rather than restricting what an agent may declare.
|
||||
Task 8: complete (commits 2a58f5a..3139de9, review clean) — 844 server tests, 292 client tests
|
||||
|
||||
FINAL whole-branch review (Opus): READY WITH FIXES — 2 Critical, 6 Important, 9 Minor. Safety core sound: every caller of resetWorktree/removeWorktree/purgeLaneSessions/deleteLane enumerated across server/, bin/, scripts/, mcp/ (the CLI goes over HTTP, so nothing outside the routes reaches the git layer), the guard sits INSIDE both destructive functions rather than only upstream, and the migration was verified against real SQLite including a simulated mid-migration crash.
|
||||
FINAL: the reviewer earned its verdict by mutation — it deleted `await assertDestroyable(lane)` from removeWorktree and all 844 tests stayed green, proving the remove-path guard was pinned by nothing. The same deletion in resetWorktree failed a test.
|
||||
FINAL: controller raised the reviewer's Important I3 to Critical — `cwd LIKE ? || '/%'` was unescaped and `_` is a LIKE wildcard, while every managed lane directory this branch creates is named `<repo>__<slug>`. A lane at /root/myrepo__feat-foo purged sessions belonging to /root/myrepoXXfeat-foo, and the preflight counted the victims too, so the confirmation was consistently wrong rather than detectably wrong. The old fixture (/tmp/wt-purge) had no underscore, so nothing could have caught it.
|
||||
Fix wave (one commit, 513235a, Opus): all 16 findings addressed, +997/-151 across 28 files, 844→857 server tests, 292→297 client tests. Deletion experiments run and reported for C2, C3, I1 and I2. The fixer also found a FOURTH instance of "the UI refusing what the server permits" in bin/ccam.js and disclosed it rather than fixing it silently.
|
||||
Fix-wave re-review (Opus): READY. Verified 857/857 itself, re-ran the C2 mutation in a throwaway copy, and built the four-way table (adopted/missing/unreadable/no-remote × reset/remove/purge × modal/server/CLI) — no square where the UI is stricter than the server, no third instance of the defect.
|
||||
FINAL: the re-reviewer found what neither the fix wave nor I had: a FOURTH refusal shape for C2 that git does NOT incidentally refuse — an adopted lane legitimately pointing at a worktree the user registered themselves. With the guard removed it destroyed the directory silently with no error at all. Check 1 is the sole protection for that case, so the guard was load-bearing well beyond the three shapes the tests cover.
|
||||
FINAL: it also disproved the fix wave's own reasoning on `unreadable` + `remove` by reproducing a corrupt-.git worktree: `git worktree remove --force` and even `--force --force` both refuse (code 128), so the promise written into `destructive.notice.unreadable` in all four locales and into two doc lines — that removal is forced and git's entry cleared — is FALSE for that shape. Decision to unblock stands (blocking recreates the defect class), but the copy over-promises.
|
||||
FINAL: registered as a known fact about the branch, not a surprise — `removeWorktree`'s new prune path deliberately does NOT call assertDestroyable; it runs check 1 plus a lexical check 2 and substitutes `worktree prune` for check 3, fires only when the directory does not exist, and is pinned by a test that still refuses a missing cwd outside LANES_ROOT.
|
||||
Open follow-ups, none blocking, surfaced to the user: (1) reword the `unreadable` notice + two doc lines to promise an attempt rather than success, add a prune-style fallback so an unreadable lane is genuinely removable, add the two missing modal tests; (2) wrap `start` in withLaneLock so its atomicity is an invariant rather than a property of the current await-free code; (3) add an openapi.yaml drift check to CI; (4) separate triage for the pre-existing unguarded `POST /api/lanes/` and `POST /api/lanes/:id/stage`.
|
||||
@@ -0,0 +1,204 @@
|
||||
# Worktree-lanes: final-review follow-ups
|
||||
|
||||
Base: bf312c6 (branch `feat/worktree-lanes`). One commit for the whole set.
|
||||
|
||||
## Follow-up 1: unreadable worktrees were genuinely unremovable
|
||||
|
||||
### (a) Made an unreadable managed lane genuinely removable
|
||||
|
||||
`server/lib/worktree.js`: `removeWorktree` unconditionally called
|
||||
`git worktree remove --force`, which git refuses outright — even with a
|
||||
second `--force` — when the worktree's OWN `.git` pointer fails its own
|
||||
validation (a corrupt/garbage `.git` file). All three `assertDestroyable`
|
||||
checks pass for this shape (the directory exists, resolves inside
|
||||
`LANES_ROOT`, and is still listed by the source repo), so the lane was stuck:
|
||||
`removeWorktree` threw, the route 500'd, the lane row survived.
|
||||
|
||||
Fix: wrapped the `git worktree remove --force` call in a try/catch. On
|
||||
failure, `findWorktreeAdminDir(sourceRepo, cwd)` locates the worktree's
|
||||
administrative directory under the source repo's common dir
|
||||
(`<common>/worktrees/<name>`) by reading each entry's `gitdir` file — that
|
||||
file's content is the absolute path to the worktree's own `.git` file, read
|
||||
from the *source repo's* side, so it still resolves correctly even though the
|
||||
worktree's own `.git` file is corrupt. If found, `fs.rmSync` deletes only
|
||||
that administrative directory (never the worktree directory itself), which
|
||||
deregisters the worktree from `git worktree list`. Branch delete and lane-row
|
||||
deletion then proceed exactly as for every other remove. If no matching
|
||||
admin directory is found (paranoid case — should not happen for a lane that
|
||||
passed all three checks), the original git error is rethrown rather than
|
||||
silently continuing, so the failure is never swallowed.
|
||||
|
||||
This is a genuine deregistration, not a workaround: it never touches the
|
||||
worktree's own files, and after it runs, `git worktree remove --force` on
|
||||
the same path correctly reports `'...' is not a working tree` — proof the
|
||||
repo no longer thinks it manages that directory.
|
||||
|
||||
### Experiment (raw output)
|
||||
|
||||
Built a real corrupt worktree under a throwaway repo and ran the exact
|
||||
sequence a reviewer had already reproduced, then the new fallback:
|
||||
|
||||
```
|
||||
=== remove --force (expect fail) ===
|
||||
fatal: validation failed, cannot remove working tree: '/tmp/tmp.4kZLeHvThU/lanes/src__corrupt/.git' is not a .git file, error code 5
|
||||
exit=128
|
||||
|
||||
=== manually delete admin dir ===
|
||||
(no output)
|
||||
|
||||
=== worktree list after manual admin removal ===
|
||||
worktree /tmp/tmp.4kZLeHvThU/src
|
||||
HEAD 0a1ed2bb01a8ac88b758180d569997b8c0bb22e8
|
||||
branch refs/heads/main
|
||||
|
||||
=== branch still exists? ===
|
||||
feat/corrupt
|
||||
|
||||
=== worktree directory + corrupted .git file still present? ===
|
||||
total 16
|
||||
drwxrwxr-x 2 smartgiftailab smartgiftailab 4096 ... .
|
||||
drwxrwxr-x 3 smartgiftailab smartgiftailab 4096 ... ..
|
||||
-rw-rw-r-- 1 smartgiftailab smartgiftailab 8 ... .git
|
||||
-rw-rw-r-- 1 smartgiftailab smartgiftailab 6 ... README.md
|
||||
garbage
|
||||
|
||||
=== git worktree remove again now (expect: not a working tree, confirms deregistered) ===
|
||||
fatal: '/tmp/tmp.4kZLeHvThU/lanes/src__corrupt' is not a working tree
|
||||
exit=128
|
||||
```
|
||||
|
||||
This confirms: (1) `remove --force` genuinely refuses a corrupt worktree,
|
||||
matching the earlier report exactly (error code 5 here vs 7 in the original
|
||||
report — git version difference, same refusal); (2) deleting only
|
||||
`.git/worktrees/<name>` deregisters the worktree from `git worktree list`
|
||||
without touching the worktree directory or its files; (3) after
|
||||
deregistration git itself confirms the path is no longer a working tree.
|
||||
|
||||
The actual code path (`removeWorktree` against a real corrupt worktree
|
||||
inside `LANES_ROOT`, through `findWorktreeAdminDir`) was then exercised by
|
||||
the new server test below and produced the same result end to end.
|
||||
|
||||
### (b) Reworded the false promise
|
||||
|
||||
`destructive.notice.unreadable` in `client/src/i18n/locales/{en,zh,vi,ko}/lanes.json`
|
||||
no longer promises forced success. New English text: "The lane directory
|
||||
cannot be read as a Git worktree. Removal is attempted; if Git itself
|
||||
refuses, its worktree record is cleared directly instead. Either way, the
|
||||
directory itself is never touched." zh/vi/ko were translated with the same
|
||||
meaning (not machine-transliterated word-for-word), matching each locale's
|
||||
existing terminology for "lane"/"worktree"/"record" already used elsewhere
|
||||
in the same file.
|
||||
|
||||
`docs/API.md` (blocked/remove paragraph) and `docs/LANES.md` (the three
|
||||
safety checks section) both had the same "force-removes an unreadable one"
|
||||
claim; both now describe the attempt-then-fallback behavior and name the
|
||||
corrupt-`.git`-pointer case explicitly.
|
||||
|
||||
### (c) Added the two missing modal tests
|
||||
|
||||
`client/src/components/lanes/__tests__/DestructiveLaneModal.test.tsx`:
|
||||
- `disables RESET when the worktree directory is unreadable` — pins
|
||||
`unreadable` staying in `HARD_BLOCKERS.reset`.
|
||||
- `ENABLES remove when the worktree directory is unreadable` — pins
|
||||
`unreadable` staying absent from `HARD_BLOCKERS.remove`, and that the
|
||||
reworded notice text renders and the modal still echoes back the exact
|
||||
`expect` block on confirm.
|
||||
|
||||
### Regression test for (a)
|
||||
|
||||
`server/__tests__/worktree.test.js`: new test builds a real worktree, writes
|
||||
garbage into its `.git` file (reproducing the exact corruption), asserts
|
||||
`git status` inside it fails (sanity), calls `removeWorktree` directly, then
|
||||
asserts: the source repo no longer lists it, its branch is gone, **and** the
|
||||
directory plus a file written into it still exist afterward untouched.
|
||||
|
||||
## Follow-up 2: `start` was atomic by accident, not by invariant
|
||||
|
||||
`server/routes/lanes.js`: the `start` case read `lane.run_id` and later wrote
|
||||
a new one with no `await` in between — atomic today only because nothing
|
||||
yields the event loop in that stretch. Wrapped the whole check-then-spawn in
|
||||
`withLaneLock(lane.id, async () => {...})`, re-fetching the lane inside the
|
||||
lock (a concurrent `remove` could have deleted the row while queued, so a
|
||||
`missing` case now returns 404 `ENOLANE` instead of dereferencing a null
|
||||
lane — a real edge case introduced by the lock itself, not present before).
|
||||
The `409 ERUNLIVE` code and message are unchanged.
|
||||
|
||||
Regression test (`server/__tests__/lane-lifecycle.test.js`,
|
||||
`"start is serialized behind the per-lane lock..."`): holds the same lane's
|
||||
lock directly from the test (`withLaneLock(lane.id, () => new Promise(...))`),
|
||||
fires `POST .../start` while that lock is held, confirms the request has NOT
|
||||
settled 50ms later and that no run_id was written, then releases the held
|
||||
lock and confirms `start` only proceeds (spawns, returns 200) after release.
|
||||
This proves the route genuinely shares the per-lane lock rather than proving
|
||||
only that the current zero-await code happens to be atomic.
|
||||
|
||||
## Follow-up 3: openapi.yaml drift check in CI
|
||||
|
||||
`.github/workflows/ci.yml`: added a step to the existing "🧹 Check Formatting"
|
||||
job — `npm run openapi:yaml` (regenerate) followed by
|
||||
`git diff --exit-code openapi.yaml`. No new job, no git hook (the pre-commit
|
||||
hook is already slow, per instruction). Verified locally: regenerating
|
||||
produces zero diff against the currently committed file.
|
||||
|
||||
## Follow-up 4: missing `sameOriginGuard` on two mutating routes
|
||||
|
||||
`server/routes/lanes.js`: added `sameOriginGuard` to `POST /api/lanes/`
|
||||
(lane creation) and `POST /api/lanes/:id/stage` (stage reporting) —
|
||||
previously the only two mutating lane routes without it.
|
||||
|
||||
Checked both callers before changing anything:
|
||||
- `bin/ccam.js`'s `post()` helper (backing `ccam lanes add` and `ccam stage`)
|
||||
only ever sends a `Content-Type` header, never `Origin` or `Referer` — the
|
||||
guard passes any request with no Origin header through unconditionally
|
||||
(same rule already relied on by every other guarded lane route). Verified
|
||||
end to end by re-running `server/__tests__/lanes-cli.test.js` (13/13) after
|
||||
the change — both `ccam lanes add --repo` and the destructive-lifecycle
|
||||
CLI flows (which call `stage` indirectly via `clear`) still pass.
|
||||
- `grep -rn "api/lanes" mcp/ scripts/` — no hits. Nothing else calls these
|
||||
routes.
|
||||
|
||||
No legitimate caller was broken; no need to weaken the guard or stop and
|
||||
ask.
|
||||
|
||||
Regression tests added to `server/__tests__/lanes-api.test.js`:
|
||||
`"rejects cross-origin lane creation"` and `"rejects cross-origin stage
|
||||
reporting"`, mirroring the existing cross-origin PATCH/DELETE tests exactly
|
||||
(assert `403 EBADORIGIN`, and for stage, that the lane's stage was not
|
||||
changed).
|
||||
|
||||
## Commands run, with tallies
|
||||
|
||||
- `node --test server/__tests__/worktree.test.js` → 19/19 (was 18)
|
||||
- `node --test server/__tests__/lane-lifecycle.test.js` → 29/29 (was 28)
|
||||
- `node --test server/__tests__/lanes-api.test.js` → 22/22 (was 20)
|
||||
- `node --test server/__tests__/lanes-cli.test.js` → 13/13 (unchanged, re-run
|
||||
as a caller check for follow-up 4)
|
||||
- `npm run test:server` → **861/861** (baseline 857)
|
||||
- `cd client && npx vitest run src/components/lanes/__tests__/DestructiveLaneModal.test.tsx`
|
||||
→ 16/16 (was 14)
|
||||
- `npm run test:client` → **299/299** (baseline 297)
|
||||
- `npm run build` → succeeded (client build, `tsc -b && vite build`)
|
||||
- `bash .claude/skills/file-headers/scripts/check-headers.sh` → exit 0
|
||||
- `node scripts/generate-openapi-yaml.js && git diff --exit-code openapi.yaml`
|
||||
→ exit 0, no diff
|
||||
- `npm run format:check` → all files pass
|
||||
|
||||
## Left out / deferred, with reasons
|
||||
|
||||
- Did not touch the "Each verb → remove" bullet in `docs/LANES.md` (a
|
||||
different paragraph from the one named in the brief) — it only describes
|
||||
the ordinary managed-removal steps and makes no claim about the
|
||||
corrupt-`.git` case, so it was not inaccurate and editing it would have
|
||||
been unrequested scope.
|
||||
- Did not add a CLI-level test reproducing the corrupt-`.git` removal
|
||||
through `ccam lanes remove` — the brief asked for "a test proving (a)"
|
||||
which the new `worktree.test.js` case already does directly against
|
||||
`removeWorktree` (the same function the route and CLI both call); adding a
|
||||
second, slower end-to-end CLI version would duplicate coverage without
|
||||
proving anything new.
|
||||
- Did not add a real concurrent-`claude`-process test for follow-up 2 (two
|
||||
genuine `POST /start` calls racing against real spawned processes) — the
|
||||
codebase's own existing tests avoid ever spawning `claude` twice
|
||||
concurrently for exactly this reason (slow, and the real `claude` binary's
|
||||
behavior isn't what's under test). The lock-holding test proves the same
|
||||
invariant deterministically without that cost.
|
||||
@@ -0,0 +1,240 @@
|
||||
# Remove Native SQLite Dependency — Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED: Use superpowers:subagent-driven-development (if subagents available) or superpowers:executing-plans to implement this plan. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Replace `better-sqlite3` (native C++ module requiring Python/build tools) with a compatibility layer over Node.js built-in `node:sqlite`, so `npm install` succeeds on any machine without native compilation tools.
|
||||
|
||||
**Architecture:** Create `server/compat-sqlite.js` — a thin wrapper that gives `DatabaseSync` (from `node:sqlite`) the same API as `better-sqlite3`. Move `better-sqlite3` to `optionalDependencies` so it's preferred when prebuilds are available but doesn't block install. The `server/db.js` loader tries `better-sqlite3` first, falls back to the compat wrapper. Update minimum Node version to 22.
|
||||
|
||||
**Tech Stack:** Node.js `node:sqlite` (DatabaseSync), existing Express/WS server
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
| File | Action | Responsibility |
|
||||
|------|--------|---------------|
|
||||
| `server/compat-sqlite.js` | **Create** | Wrapper class: DatabaseSync → better-sqlite3 API |
|
||||
| `server/db.js` | **Modify** (line 1) | Try better-sqlite3, fallback to compat wrapper |
|
||||
| `scripts/clear-data.js` | **Modify** (line 8) | Same fallback import |
|
||||
| `package.json` | **Modify** | Move better-sqlite3 to optionalDependencies, bump engines to >=22 |
|
||||
| `server/__tests__/api.test.js` | **Modify** (lines 952-959) | Fix `db.pragma()` calls to work with both backends |
|
||||
|
||||
---
|
||||
|
||||
## Chunk 1: Core Implementation
|
||||
|
||||
### Task 1: Create `server/compat-sqlite.js`
|
||||
|
||||
**Files:**
|
||||
- Create: `server/compat-sqlite.js`
|
||||
|
||||
- [ ] **Step 1: Write the compat wrapper**
|
||||
|
||||
The wrapper must bridge these API differences:
|
||||
|
||||
| better-sqlite3 | node:sqlite (DatabaseSync) |
|
||||
|----------------|---------------------------|
|
||||
| `new Database(path)` | `new DatabaseSync(path)` |
|
||||
| `db.pragma("key = value")` | `db.exec("PRAGMA key = value")` |
|
||||
| `db.pragma("key")` → value | `db.prepare("PRAGMA key").get()` → `{key: value}` |
|
||||
| `db.pragma("key", { simple: true })` → value | same as above, extract single value |
|
||||
| `db.transaction(fn)` → wrapper fn | manual `BEGIN`/`COMMIT`/`ROLLBACK` |
|
||||
| `db.prepare(sql)` → stmt with `.run()`, `.get()`, `.all()` | identical API |
|
||||
| `db.exec(sql)` | identical |
|
||||
| `db.close()` | identical |
|
||||
|
||||
```js
|
||||
// server/compat-sqlite.js
|
||||
const { DatabaseSync } = require("node:sqlite");
|
||||
|
||||
class Database {
|
||||
constructor(filePath) {
|
||||
this._db = new DatabaseSync(filePath);
|
||||
}
|
||||
|
||||
exec(sql) {
|
||||
this._db.exec(sql);
|
||||
return this;
|
||||
}
|
||||
|
||||
pragma(str, options) {
|
||||
if (str.includes("=")) {
|
||||
this._db.exec(`PRAGMA ${str}`);
|
||||
return undefined;
|
||||
}
|
||||
const row = this._db.prepare(`PRAGMA ${str}`).get();
|
||||
if (!row) return undefined;
|
||||
const keys = Object.keys(row);
|
||||
if (options?.simple || keys.length === 1) return row[keys[0]];
|
||||
return row;
|
||||
}
|
||||
|
||||
prepare(sql) {
|
||||
return this._db.prepare(sql);
|
||||
}
|
||||
|
||||
transaction(fn) {
|
||||
const db = this._db;
|
||||
const wrapper = (...args) => {
|
||||
db.exec("BEGIN");
|
||||
try {
|
||||
const result = fn(...args);
|
||||
db.exec("COMMIT");
|
||||
return result;
|
||||
} catch (err) {
|
||||
db.exec("ROLLBACK");
|
||||
throw err;
|
||||
}
|
||||
};
|
||||
return wrapper;
|
||||
}
|
||||
|
||||
close() {
|
||||
this._db.close();
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = Database;
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Verify the wrapper works standalone**
|
||||
|
||||
Run: `node -e "const DB = require('./server/compat-sqlite'); const db = new DB(':memory:'); db.pragma('journal_mode = WAL'); db.exec('CREATE TABLE t (id INTEGER PRIMARY KEY, v TEXT)'); const s = db.prepare('INSERT INTO t (v) VALUES (?)'); console.log(s.run('hi')); console.log(db.prepare('SELECT * FROM t').all()); const tx = db.transaction((items) => { for (const i of items) s.run(i); }); tx(['a','b','c']); console.log(db.prepare('SELECT COUNT(*) as c FROM t').get()); db.close(); console.log('OK')"`
|
||||
|
||||
Expected: `OK` printed at the end with correct query results.
|
||||
|
||||
- [ ] **Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add server/compat-sqlite.js
|
||||
git commit -m "feat: add node:sqlite compat wrapper for better-sqlite3 API"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Update `server/db.js` to use fallback import
|
||||
|
||||
**Files:**
|
||||
- Modify: `server/db.js:1`
|
||||
|
||||
- [ ] **Step 1: Replace the import**
|
||||
|
||||
Change line 1 from:
|
||||
```js
|
||||
const Database = require("better-sqlite3");
|
||||
```
|
||||
To:
|
||||
```js
|
||||
let Database;
|
||||
try {
|
||||
Database = require("better-sqlite3");
|
||||
} catch {
|
||||
Database = require("./compat-sqlite");
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Verify server starts**
|
||||
|
||||
Run: `node -e "process.env.DASHBOARD_DB_PATH = require('path').join(require('os').tmpdir(), 'test-fallback-' + Date.now() + '.db'); const { db, stmts } = require('./server/db'); console.log('stmts keys:', Object.keys(stmts).length); stmts.insertSession.run('test-1', 'Test', 'active', null, null, null); console.log(stmts.getSession.get('test-1')); db.close(); console.log('OK')"`
|
||||
|
||||
Expected: Prints statement count (39), session row, and `OK`.
|
||||
|
||||
- [ ] **Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add server/db.js
|
||||
git commit -m "feat: fallback to node:sqlite when better-sqlite3 unavailable"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Update `scripts/clear-data.js` to use fallback import
|
||||
|
||||
**Files:**
|
||||
- Modify: `scripts/clear-data.js:8`
|
||||
|
||||
- [ ] **Step 1: Replace the import**
|
||||
|
||||
Change line 8 from:
|
||||
```js
|
||||
const Database = require("better-sqlite3");
|
||||
```
|
||||
To:
|
||||
```js
|
||||
let Database;
|
||||
try {
|
||||
Database = require("better-sqlite3");
|
||||
} catch {
|
||||
Database = require("../server/compat-sqlite");
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Commit**
|
||||
|
||||
```bash
|
||||
git add scripts/clear-data.js
|
||||
git commit -m "fix: use fallback sqlite import in clear-data script"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Update `package.json`
|
||||
|
||||
**Files:**
|
||||
- Modify: `package.json`
|
||||
|
||||
- [ ] **Step 1: Move better-sqlite3 to optionalDependencies, bump engines**
|
||||
|
||||
Move `"better-sqlite3": "^11.7.0"` from `dependencies` to `optionalDependencies`.
|
||||
Change engines from `"node": ">=18.0.0"` to `"node": ">=22.0.0"`.
|
||||
|
||||
- [ ] **Step 2: Commit**
|
||||
|
||||
```bash
|
||||
git add package.json
|
||||
git commit -m "chore: make better-sqlite3 optional, require Node >= 22"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Fix test pragma calls
|
||||
|
||||
**Files:**
|
||||
- Modify: `server/__tests__/api.test.js:952-959`
|
||||
|
||||
- [ ] **Step 1: Fix pragma calls in Database Integrity tests**
|
||||
|
||||
The tests call `db.pragma("journal_mode", { simple: true })` and `db.pragma("foreign_keys", { simple: true })`. The compat wrapper supports `{ simple: true }`, so these should work as-is. However, WAL mode isn't available for in-memory databases (returns "memory"). The test creates a file-based DB via `TEST_DB`, so WAL should work.
|
||||
|
||||
No change needed — verify by running tests.
|
||||
|
||||
- [ ] **Step 2: Run full test suite**
|
||||
|
||||
Run: `node --test server/__tests__/api.test.js`
|
||||
|
||||
Expected: All tests pass.
|
||||
|
||||
- [ ] **Step 3: Run setup to verify npm install succeeds without Python**
|
||||
|
||||
Run: `npm run setup`
|
||||
|
||||
Expected: Install succeeds (better-sqlite3 may warn but won't fail since it's optional).
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Update documentation
|
||||
|
||||
**Files:**
|
||||
- Modify: `SETUP.md` (if it mentions better-sqlite3 or Python requirements)
|
||||
|
||||
- [ ] **Step 1: Check and update SETUP.md**
|
||||
|
||||
Remove any mentions of Python or build tools as requirements. Note that Node >= 22 is required.
|
||||
|
||||
- [ ] **Step 2: Commit all remaining changes**
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "docs: update setup requirements for native-free SQLite"
|
||||
```
|
||||
@@ -0,0 +1,875 @@
|
||||
# JSONL Reading Performance Optimization
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Eliminate redundant full-file reads of JSONL transcript files by caching extracted token data and using incremental reads.
|
||||
|
||||
**Architecture:** Add a lightweight in-memory cache keyed by `(transcriptPath, mtime, size)` that stores the extracted `{tokensByModel, compaction}` result. On each hook event, stat the file first — if unchanged, return cached result. For files that did change, use byte-offset tracking to only read new lines appended since last parse. The periodic compaction scanner shares this same cache.
|
||||
|
||||
**Tech Stack:** Node.js `fs.statSync`, in-memory `Map` cache, byte-offset tracking via `fs.openSync`/`fs.readSync`.
|
||||
|
||||
---
|
||||
|
||||
## Performance Problem Analysis
|
||||
|
||||
### Current Behavior
|
||||
|
||||
Three code paths read JSONL files **fully, synchronously, with zero caching**:
|
||||
|
||||
| Path | File | Trigger | Frequency |
|
||||
|------|------|---------|-----------|
|
||||
| `extractTokensFromTranscript()` | `server/routes/hooks.js:15-62` | Every POST `/api/hooks/event` with `transcript_path` | 1-10x/min per active session |
|
||||
| `findCompactionsInFile()` | `scripts/import-history.js:658-674` | 2-minute periodic scan | Every 2 min × active sessions |
|
||||
| `parseSessionFile()` | `scripts/import-history.js:22-131` | Server startup import | Once per JSONL file at startup |
|
||||
|
||||
### Why This Hurts
|
||||
|
||||
1. **`extractTokensFromTranscript` is the hot path.** Called on *every* hook event. For a session producing 5 events/min with a 10K-line JSONL (typical long session), that's 5 full file reads + 50K `JSON.parse` calls per minute.
|
||||
|
||||
2. **JSONL files are append-only** (until compaction rewrites them). Between hook events, only a few new lines are appended. Reading the entire file to re-sum tokens that haven't changed is pure waste.
|
||||
|
||||
3. **`readFileSync` blocks the event loop.** Long sessions (50K+ lines, several MB) block the Express request handler for tens of milliseconds, stalling concurrent hook ingestion and API responses.
|
||||
|
||||
4. **Periodic scanner duplicates work.** `findCompactionsInFile` re-reads the same files that `extractTokensFromTranscript` already parsed seconds ago.
|
||||
|
||||
### Quantified Impact (estimated)
|
||||
|
||||
| Session Length | Lines | File Size | Parse Time (sync) | Events/min | Wasted CPU/min |
|
||||
|---------------|-------|-----------|--------------------|------------|----------------|
|
||||
| Short (30min) | 500 | ~100KB | ~2ms | 3 | ~6ms |
|
||||
| Medium (2hr) | 5,000 | ~1MB | ~15ms | 5 | ~75ms |
|
||||
| Long (8hr+) | 20,000 | ~4MB | ~50ms | 8 | ~400ms |
|
||||
| Marathon (24hr) | 50,000+ | ~10MB+ | ~120ms+ | 10 | ~1.2s |
|
||||
|
||||
With multiple concurrent sessions, this compounds. The 2-minute scanner adds another full read per active session on top.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
| File | Responsibility | Action |
|
||||
|------|---------------|--------|
|
||||
| `server/lib/transcript-cache.js` | In-memory cache + incremental reader for JSONL files | **Create** |
|
||||
| `server/lib/__tests__/transcript-cache.test.js` | Unit tests for cache + incremental read logic | **Create** |
|
||||
| `server/routes/hooks.js` | Hook event handler — swap `extractTokensFromTranscript` to use cache | **Modify** (lines 15-62, 353-354) |
|
||||
| `scripts/import-history.js` | Periodic compaction scanner — swap `findCompactionsInFile` to use cache | **Modify** (lines 658-674) |
|
||||
| `server/index.js` | Wire cache into periodic scanner; add cache stats to settings | **Modify** (lines 104-128) |
|
||||
| `server/routes/settings.js` | Expose cache stats in `/api/settings/info` | **Modify** |
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Create the Transcript Cache Module
|
||||
|
||||
**Files:**
|
||||
- Create: `server/lib/transcript-cache.js`
|
||||
- Test: `server/lib/__tests__/transcript-cache.test.js`
|
||||
|
||||
### Design
|
||||
|
||||
```
|
||||
Cache entry = {
|
||||
mtime: number, // file modification time (ms)
|
||||
size: number, // file size in bytes
|
||||
bytesRead: number, // how far we've read into the file
|
||||
tokensByModel: {}, // accumulated token sums
|
||||
compaction: null|{}, // compaction entries found so far
|
||||
}
|
||||
|
||||
On read request:
|
||||
1. fs.statSync(path) → get mtime + size
|
||||
2. Cache hit? (same mtime + size) → return cached result
|
||||
3. File shrunk or mtime changed with smaller size? → compaction rewrite → full re-read, reset cache
|
||||
4. File grew? (size > bytesRead) → incremental read from bytesRead → parse new lines → merge into cached totals
|
||||
5. Store updated entry, return result
|
||||
```
|
||||
|
||||
- [ ] **Step 1: Create test file with first test — cache miss triggers full read**
|
||||
|
||||
```javascript
|
||||
// server/lib/__tests__/transcript-cache.test.js
|
||||
const { describe, it, beforeEach, afterEach } = require("node:test");
|
||||
const assert = require("node:assert/strict");
|
||||
const fs = require("fs");
|
||||
const path = require("path");
|
||||
const os = require("os");
|
||||
|
||||
let tmpDir;
|
||||
let TranscriptCache;
|
||||
|
||||
function writeJsonl(filePath, entries) {
|
||||
fs.writeFileSync(filePath, entries.map((e) => JSON.stringify(e)).join("\n") + "\n");
|
||||
}
|
||||
|
||||
describe("TranscriptCache", () => {
|
||||
beforeEach(() => {
|
||||
tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), "tc-test-"));
|
||||
// Fresh require to reset module-level state
|
||||
delete require.cache[require.resolve("../../lib/transcript-cache")];
|
||||
TranscriptCache = require("../../lib/transcript-cache");
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
fs.rmSync(tmpDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it("should extract tokens on first read (cache miss)", () => {
|
||||
const file = path.join(tmpDir, "session.jsonl");
|
||||
writeJsonl(file, [
|
||||
{ message: { model: "claude-sonnet-4-20250514", usage: { input_tokens: 100, output_tokens: 50 } } },
|
||||
{ message: { model: "claude-sonnet-4-20250514", usage: { input_tokens: 200, output_tokens: 75 } } },
|
||||
]);
|
||||
|
||||
const cache = new TranscriptCache();
|
||||
const result = cache.extract(file);
|
||||
|
||||
assert.deepStrictEqual(result.tokensByModel, {
|
||||
"claude-sonnet-4-20250514": { input: 300, output: 125, cacheRead: 0, cacheWrite: 0 },
|
||||
});
|
||||
assert.strictEqual(result.compaction, null);
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `node --test server/lib/__tests__/transcript-cache.test.js`
|
||||
Expected: FAIL — module not found
|
||||
|
||||
- [ ] **Step 3: Implement TranscriptCache with full-read path**
|
||||
|
||||
```javascript
|
||||
// server/lib/transcript-cache.js
|
||||
const fs = require("fs");
|
||||
|
||||
class TranscriptCache {
|
||||
constructor() {
|
||||
this._cache = new Map();
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract token usage and compaction data from a JSONL transcript file.
|
||||
* Uses stat-based caching — returns cached result if file hasn't changed.
|
||||
* Returns null if file doesn't exist or has no data.
|
||||
*/
|
||||
extract(transcriptPath) {
|
||||
if (!transcriptPath) return null;
|
||||
try {
|
||||
const stat = fs.statSync(transcriptPath);
|
||||
const key = transcriptPath;
|
||||
const cached = this._cache.get(key);
|
||||
|
||||
// Cache hit: file unchanged
|
||||
if (cached && cached.mtimeMs === stat.mtimeMs && cached.size === stat.size) {
|
||||
return cached.result;
|
||||
}
|
||||
|
||||
// Full read (cache miss or file was rewritten/compacted)
|
||||
const result = this._fullRead(transcriptPath);
|
||||
this._cache.set(key, {
|
||||
mtimeMs: stat.mtimeMs,
|
||||
size: stat.size,
|
||||
bytesRead: stat.size,
|
||||
tokensByModel: result ? { ...result.tokensByModel } : null,
|
||||
compaction: result ? result.compaction : null,
|
||||
result,
|
||||
});
|
||||
return result;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
_fullRead(filePath) {
|
||||
const content = fs.readFileSync(filePath, "utf8");
|
||||
return this._parseContent(content);
|
||||
}
|
||||
|
||||
_parseContent(content) {
|
||||
const tokensByModel = {};
|
||||
let compaction = null;
|
||||
for (const line of content.split("\n")) {
|
||||
if (!line) continue;
|
||||
try {
|
||||
const entry = JSON.parse(line);
|
||||
if (entry.isCompactSummary) {
|
||||
if (!compaction) compaction = { count: 0, entries: [] };
|
||||
compaction.count++;
|
||||
compaction.entries.push({
|
||||
uuid: entry.uuid || null,
|
||||
timestamp: entry.timestamp || null,
|
||||
});
|
||||
}
|
||||
const msg = entry.message || entry;
|
||||
const model = msg.model;
|
||||
if (!model || model === "<synthetic>" || !msg.usage) continue;
|
||||
if (!tokensByModel[model]) {
|
||||
tokensByModel[model] = { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 };
|
||||
}
|
||||
tokensByModel[model].input += msg.usage.input_tokens || 0;
|
||||
tokensByModel[model].output += msg.usage.output_tokens || 0;
|
||||
tokensByModel[model].cacheRead += msg.usage.cache_read_input_tokens || 0;
|
||||
tokensByModel[model].cacheWrite += msg.usage.cache_creation_input_tokens || 0;
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
}
|
||||
const hasTokens = Object.keys(tokensByModel).length > 0;
|
||||
if (!hasTokens && !compaction) return null;
|
||||
return { tokensByModel: hasTokens ? tokensByModel : null, compaction };
|
||||
}
|
||||
|
||||
/** Number of entries currently cached */
|
||||
get size() {
|
||||
return this._cache.size;
|
||||
}
|
||||
|
||||
/** Remove a specific path from cache (e.g. when session ends) */
|
||||
invalidate(transcriptPath) {
|
||||
this._cache.delete(transcriptPath);
|
||||
}
|
||||
|
||||
/** Clear all cached entries */
|
||||
clear() {
|
||||
this._cache.clear();
|
||||
}
|
||||
|
||||
/** Return cache stats for diagnostics */
|
||||
stats() {
|
||||
return {
|
||||
entries: this._cache.size,
|
||||
paths: [...this._cache.keys()],
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = TranscriptCache;
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `node --test server/lib/__tests__/transcript-cache.test.js`
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add server/lib/transcript-cache.js server/lib/__tests__/transcript-cache.test.js
|
||||
git commit -m "feat: add TranscriptCache module with stat-based caching for JSONL reads"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Add Cache Hit and Compaction Detection Tests
|
||||
|
||||
**Files:**
|
||||
- Modify: `server/lib/__tests__/transcript-cache.test.js`
|
||||
|
||||
- [ ] **Step 1: Add test — second read with unchanged file returns cached result**
|
||||
|
||||
```javascript
|
||||
it("should return cached result when file is unchanged", () => {
|
||||
const file = path.join(tmpDir, "session.jsonl");
|
||||
writeJsonl(file, [
|
||||
{ message: { model: "claude-sonnet-4-20250514", usage: { input_tokens: 100, output_tokens: 50 } } },
|
||||
]);
|
||||
|
||||
const cache = new TranscriptCache();
|
||||
const r1 = cache.extract(file);
|
||||
const r2 = cache.extract(file);
|
||||
|
||||
assert.deepStrictEqual(r1, r2);
|
||||
// Same object reference proves cache hit (no re-parse)
|
||||
assert.strictEqual(r1, r2);
|
||||
});
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add test — detects appended lines after file grows**
|
||||
|
||||
```javascript
|
||||
it("should detect new data when file grows", (t) => {
|
||||
const file = path.join(tmpDir, "session.jsonl");
|
||||
writeJsonl(file, [
|
||||
{ message: { model: "claude-sonnet-4-20250514", usage: { input_tokens: 100, output_tokens: 50 } } },
|
||||
]);
|
||||
|
||||
const cache = new TranscriptCache();
|
||||
const r1 = cache.extract(file);
|
||||
assert.strictEqual(r1.tokensByModel["claude-sonnet-4-20250514"].input, 100);
|
||||
|
||||
// Append more data (simulates Claude writing to transcript)
|
||||
fs.appendFileSync(
|
||||
file,
|
||||
JSON.stringify({ message: { model: "claude-sonnet-4-20250514", usage: { input_tokens: 200, output_tokens: 75 } } }) + "\n"
|
||||
);
|
||||
|
||||
const r2 = cache.extract(file);
|
||||
assert.strictEqual(r2.tokensByModel["claude-sonnet-4-20250514"].input, 300);
|
||||
assert.strictEqual(r2.tokensByModel["claude-sonnet-4-20250514"].output, 125);
|
||||
});
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Add test — detects compaction (file shrinks)**
|
||||
|
||||
```javascript
|
||||
it("should do full re-read when file shrinks (compaction rewrite)", () => {
|
||||
const file = path.join(tmpDir, "session.jsonl");
|
||||
writeJsonl(file, [
|
||||
{ message: { model: "claude-sonnet-4-20250514", usage: { input_tokens: 500, output_tokens: 200 } } },
|
||||
{ message: { model: "claude-sonnet-4-20250514", usage: { input_tokens: 300, output_tokens: 100 } } },
|
||||
]);
|
||||
|
||||
const cache = new TranscriptCache();
|
||||
cache.extract(file);
|
||||
|
||||
// Simulate compaction — file is rewritten with fewer entries + summary
|
||||
writeJsonl(file, [
|
||||
{ isCompactSummary: true, uuid: "abc-123", timestamp: "2026-03-20T10:00:00Z" },
|
||||
{ message: { model: "claude-sonnet-4-20250514", usage: { input_tokens: 50, output_tokens: 20 } } },
|
||||
]);
|
||||
|
||||
const r2 = cache.extract(file);
|
||||
assert.strictEqual(r2.tokensByModel["claude-sonnet-4-20250514"].input, 50);
|
||||
assert.strictEqual(r2.compaction.count, 1);
|
||||
assert.strictEqual(r2.compaction.entries[0].uuid, "abc-123");
|
||||
});
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Add test — returns null for missing file**
|
||||
|
||||
```javascript
|
||||
it("should return null for non-existent file", () => {
|
||||
const cache = new TranscriptCache();
|
||||
assert.strictEqual(cache.extract("/nonexistent/file.jsonl"), null);
|
||||
assert.strictEqual(cache.extract(null), null);
|
||||
assert.strictEqual(cache.extract(""), null);
|
||||
});
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Add test — compaction-only extraction (for findCompactionsInFile replacement)**
|
||||
|
||||
```javascript
|
||||
it("should expose compaction entries via extractCompactions()", () => {
|
||||
const file = path.join(tmpDir, "session.jsonl");
|
||||
writeJsonl(file, [
|
||||
{ message: { model: "claude-sonnet-4-20250514", usage: { input_tokens: 100, output_tokens: 50 } } },
|
||||
{ isCompactSummary: true, uuid: "c1", timestamp: "2026-03-20T09:00:00Z" },
|
||||
{ message: { model: "claude-sonnet-4-20250514", usage: { input_tokens: 50, output_tokens: 20 } } },
|
||||
{ isCompactSummary: true, uuid: "c2", timestamp: "2026-03-20T10:00:00Z" },
|
||||
]);
|
||||
|
||||
const cache = new TranscriptCache();
|
||||
const compactions = cache.extractCompactions(file);
|
||||
|
||||
assert.strictEqual(compactions.length, 2);
|
||||
assert.strictEqual(compactions[0].uuid, "c1");
|
||||
assert.strictEqual(compactions[1].uuid, "c2");
|
||||
});
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Run all tests**
|
||||
|
||||
Run: `node --test server/lib/__tests__/transcript-cache.test.js`
|
||||
Expected: All pass
|
||||
|
||||
- [ ] **Step 7: Commit**
|
||||
|
||||
```bash
|
||||
git add server/lib/__tests__/transcript-cache.test.js
|
||||
git commit -m "test: add cache hit, compaction, and edge case tests for TranscriptCache"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Add Incremental Read (Byte-Offset Tracking)
|
||||
|
||||
**Files:**
|
||||
- Modify: `server/lib/transcript-cache.js`
|
||||
- Modify: `server/lib/__tests__/transcript-cache.test.js`
|
||||
|
||||
This is the key optimization. JSONL files are append-only between compactions. Instead of re-reading the full file, read only the bytes appended since our last read.
|
||||
|
||||
- [ ] **Step 1: Add test — incremental read only parses new bytes**
|
||||
|
||||
```javascript
|
||||
it("should only read new bytes on incremental update (not full file)", () => {
|
||||
const file = path.join(tmpDir, "session.jsonl");
|
||||
const line1 = JSON.stringify({ message: { model: "m1", usage: { input_tokens: 100, output_tokens: 50 } } }) + "\n";
|
||||
fs.writeFileSync(file, line1);
|
||||
|
||||
const cache = new TranscriptCache();
|
||||
cache.extract(file);
|
||||
|
||||
// Append a second line
|
||||
const line2 = JSON.stringify({ message: { model: "m1", usage: { input_tokens: 200, output_tokens: 75 } } }) + "\n";
|
||||
fs.appendFileSync(file, line2);
|
||||
|
||||
// Spy: check bytesRead advanced by only line2 length
|
||||
const r2 = cache.extract(file);
|
||||
assert.strictEqual(r2.tokensByModel["m1"].input, 300);
|
||||
|
||||
const entry = cache._cache.get(file);
|
||||
assert.strictEqual(entry.bytesRead, Buffer.byteLength(line1 + line2, "utf8"));
|
||||
});
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Update `extract()` to use incremental read path**
|
||||
|
||||
In `server/lib/transcript-cache.js`, update the `extract` method:
|
||||
|
||||
```javascript
|
||||
extract(transcriptPath) {
|
||||
if (!transcriptPath) return null;
|
||||
try {
|
||||
let stat;
|
||||
try {
|
||||
stat = fs.statSync(transcriptPath);
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
const key = transcriptPath;
|
||||
const cached = this._cache.get(key);
|
||||
|
||||
// Cache hit: file unchanged
|
||||
if (cached && cached.mtimeMs === stat.mtimeMs && cached.size === stat.size) {
|
||||
return cached.result;
|
||||
}
|
||||
|
||||
// File shrunk or was rewritten (compaction) → full re-read
|
||||
if (!cached || stat.size < cached.bytesRead) {
|
||||
const result = this._fullRead(transcriptPath);
|
||||
this._cache.set(key, {
|
||||
mtimeMs: stat.mtimeMs,
|
||||
size: stat.size,
|
||||
bytesRead: stat.size,
|
||||
tokensByModel: result ? this._cloneTokens(result.tokensByModel) : null,
|
||||
compaction: result ? this._cloneCompaction(result.compaction) : null,
|
||||
result,
|
||||
});
|
||||
return result;
|
||||
}
|
||||
|
||||
// File grew → incremental read from last position
|
||||
const newBytes = this._readFrom(transcriptPath, cached.bytesRead, stat.size);
|
||||
if (newBytes) {
|
||||
const incremental = this._parseContent(newBytes);
|
||||
const merged = this._merge(cached, incremental);
|
||||
const result = {
|
||||
tokensByModel: Object.keys(merged.tokensByModel).length > 0 ? merged.tokensByModel : null,
|
||||
compaction: merged.compaction,
|
||||
};
|
||||
if (!result.tokensByModel && !result.compaction) {
|
||||
this._cache.set(key, { ...cached, mtimeMs: stat.mtimeMs, size: stat.size, bytesRead: stat.size, result: null });
|
||||
return null;
|
||||
}
|
||||
this._cache.set(key, {
|
||||
mtimeMs: stat.mtimeMs,
|
||||
size: stat.size,
|
||||
bytesRead: stat.size,
|
||||
tokensByModel: this._cloneTokens(result.tokensByModel),
|
||||
compaction: this._cloneCompaction(result.compaction),
|
||||
result,
|
||||
});
|
||||
return result;
|
||||
}
|
||||
|
||||
// newBytes was empty (e.g. only newlines appended)
|
||||
this._cache.set(key, { ...cached, mtimeMs: stat.mtimeMs, size: stat.size, bytesRead: stat.size });
|
||||
return cached.result;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
_readFrom(filePath, offset, totalSize) {
|
||||
const len = totalSize - offset;
|
||||
if (len <= 0) return null;
|
||||
const buf = Buffer.alloc(len);
|
||||
const fd = fs.openSync(filePath, "r");
|
||||
try {
|
||||
fs.readSync(fd, buf, 0, len, offset);
|
||||
} finally {
|
||||
fs.closeSync(fd);
|
||||
}
|
||||
return buf.toString("utf8");
|
||||
}
|
||||
|
||||
_merge(cached, incremental) {
|
||||
const tokensByModel = cached.tokensByModel ? { ...cached.tokensByModel } : {};
|
||||
if (incremental && incremental.tokensByModel) {
|
||||
for (const [model, tokens] of Object.entries(incremental.tokensByModel)) {
|
||||
if (!tokensByModel[model]) {
|
||||
tokensByModel[model] = { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 };
|
||||
}
|
||||
tokensByModel[model].input += tokens.input;
|
||||
tokensByModel[model].output += tokens.output;
|
||||
tokensByModel[model].cacheRead += tokens.cacheRead;
|
||||
tokensByModel[model].cacheWrite += tokens.cacheWrite;
|
||||
}
|
||||
}
|
||||
|
||||
let compaction = cached.compaction ? this._cloneCompaction(cached.compaction) : null;
|
||||
if (incremental && incremental.compaction) {
|
||||
if (!compaction) compaction = { count: 0, entries: [] };
|
||||
compaction.count += incremental.compaction.count;
|
||||
compaction.entries.push(...incremental.compaction.entries);
|
||||
}
|
||||
|
||||
return { tokensByModel, compaction };
|
||||
}
|
||||
|
||||
_cloneTokens(tokensByModel) {
|
||||
if (!tokensByModel) return null;
|
||||
const clone = {};
|
||||
for (const [model, t] of Object.entries(tokensByModel)) {
|
||||
clone[model] = { ...t };
|
||||
}
|
||||
return clone;
|
||||
}
|
||||
|
||||
_cloneCompaction(compaction) {
|
||||
if (!compaction) return null;
|
||||
return { count: compaction.count, entries: compaction.entries.map((e) => ({ ...e })) };
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run tests**
|
||||
|
||||
Run: `node --test server/lib/__tests__/transcript-cache.test.js`
|
||||
Expected: All pass
|
||||
|
||||
- [ ] **Step 4: Add `extractCompactions()` convenience method**
|
||||
|
||||
```javascript
|
||||
/**
|
||||
* Extract only compaction entries from a JSONL file (replacement for findCompactionsInFile).
|
||||
* Uses the same cache — no duplicate reads.
|
||||
*/
|
||||
extractCompactions(transcriptPath) {
|
||||
const result = this.extract(transcriptPath);
|
||||
if (!result || !result.compaction) return [];
|
||||
return result.compaction.entries;
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Run all tests**
|
||||
|
||||
Run: `node --test server/lib/__tests__/transcript-cache.test.js`
|
||||
Expected: All pass
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add server/lib/transcript-cache.js server/lib/__tests__/transcript-cache.test.js
|
||||
git commit -m "feat: add incremental byte-offset reads and extractCompactions to TranscriptCache"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Wire Cache into Hook Handler
|
||||
|
||||
**Files:**
|
||||
- Modify: `server/routes/hooks.js` (lines 1-62, 353-354)
|
||||
|
||||
Replace the standalone `extractTokensFromTranscript` function with the shared `TranscriptCache` instance.
|
||||
|
||||
- [ ] **Step 1: Create shared cache instance and replace function**
|
||||
|
||||
At the top of `server/routes/hooks.js`, replace:
|
||||
|
||||
```javascript
|
||||
// OLD (lines 15-62): the entire extractTokensFromTranscript function
|
||||
```
|
||||
|
||||
With:
|
||||
|
||||
```javascript
|
||||
const TranscriptCache = require("../lib/transcript-cache");
|
||||
const transcriptCache = new TranscriptCache();
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Update the call site at line 353-354**
|
||||
|
||||
Replace:
|
||||
```javascript
|
||||
const result = extractTokensFromTranscript(data.transcript_path);
|
||||
```
|
||||
|
||||
With:
|
||||
```javascript
|
||||
const result = transcriptCache.extract(data.transcript_path);
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Export the cache instance for use by periodic scanner**
|
||||
|
||||
At the bottom of hooks.js, change:
|
||||
```javascript
|
||||
module.exports = router;
|
||||
```
|
||||
To:
|
||||
```javascript
|
||||
module.exports = router;
|
||||
module.exports.transcriptCache = transcriptCache;
|
||||
```
|
||||
|
||||
Wait — that overwrites the router export. Instead, attach it to the router:
|
||||
|
||||
```javascript
|
||||
router.transcriptCache = transcriptCache;
|
||||
module.exports = router;
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run existing server tests to verify no regression**
|
||||
|
||||
Run: `npm run test:server`
|
||||
Expected: All existing tests pass
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add server/routes/hooks.js
|
||||
git commit -m "refactor: replace extractTokensFromTranscript with TranscriptCache in hook handler"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Wire Cache into Periodic Compaction Scanner
|
||||
|
||||
**Files:**
|
||||
- Modify: `server/index.js` (lines 86, 104-128)
|
||||
|
||||
The 2-minute periodic scanner currently calls `findCompactionsInFile()` which does its own full synchronous read. Replace it with the shared cache from the hooks router.
|
||||
|
||||
- [ ] **Step 1: Update import and use shared cache**
|
||||
|
||||
In `server/index.js`, in the `if (!isTest)` block where the periodic scanner is set up (~line 85):
|
||||
|
||||
Replace the import:
|
||||
```javascript
|
||||
const { importCompactions, findCompactionsInFile } = require("../scripts/import-history");
|
||||
```
|
||||
|
||||
With:
|
||||
```javascript
|
||||
const { importCompactions } = require("../scripts/import-history");
|
||||
const { transcriptCache } = require("./routes/hooks");
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Replace `findCompactionsInFile` calls with cache**
|
||||
|
||||
Replace (inside the setInterval, ~line 113):
|
||||
```javascript
|
||||
const compactions = findCompactionsInFile(row.tp);
|
||||
```
|
||||
|
||||
With:
|
||||
```javascript
|
||||
const compactions = transcriptCache.extractCompactions(row.tp);
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run server tests**
|
||||
|
||||
Run: `npm run test:server`
|
||||
Expected: All pass
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add server/index.js
|
||||
git commit -m "refactor: periodic compaction scanner uses shared TranscriptCache instead of standalone file reads"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 6: Cache Eviction for Ended Sessions
|
||||
|
||||
**Files:**
|
||||
- Modify: `server/routes/hooks.js`
|
||||
|
||||
When a session completes, its JSONL file won't be read again. Evict it from cache to prevent unbounded memory growth.
|
||||
|
||||
- [ ] **Step 1: Add test for cache invalidation**
|
||||
|
||||
Add to `server/lib/__tests__/transcript-cache.test.js`:
|
||||
|
||||
```javascript
|
||||
it("should remove entry on invalidate()", () => {
|
||||
const file = path.join(tmpDir, "session.jsonl");
|
||||
writeJsonl(file, [
|
||||
{ message: { model: "m1", usage: { input_tokens: 100, output_tokens: 50 } } },
|
||||
]);
|
||||
|
||||
const cache = new TranscriptCache();
|
||||
cache.extract(file);
|
||||
assert.strictEqual(cache.size, 1);
|
||||
|
||||
cache.invalidate(file);
|
||||
assert.strictEqual(cache.size, 0);
|
||||
});
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run test**
|
||||
|
||||
Run: `node --test server/lib/__tests__/transcript-cache.test.js`
|
||||
Expected: Pass (invalidate was already implemented in Task 1)
|
||||
|
||||
- [ ] **Step 3: Add eviction when session ends in hooks.js**
|
||||
|
||||
In `server/routes/hooks.js`, find the Stop event handler section. After the session is updated to "completed", add:
|
||||
|
||||
```javascript
|
||||
// Evict transcript from cache — session is done, no more reads expected
|
||||
if (data.transcript_path) {
|
||||
transcriptCache.invalidate(data.transcript_path);
|
||||
}
|
||||
```
|
||||
|
||||
Place this right after the `stmts.updateSession.run(...)` call for the Stop event that sets status to "completed".
|
||||
|
||||
- [ ] **Step 4: Run server tests**
|
||||
|
||||
Run: `npm run test:server`
|
||||
Expected: All pass
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add server/routes/hooks.js server/lib/__tests__/transcript-cache.test.js
|
||||
git commit -m "feat: evict transcript cache entry when session completes"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 7: Expose Cache Stats in Settings API
|
||||
|
||||
**Files:**
|
||||
- Modify: `server/routes/settings.js`
|
||||
|
||||
Add cache stats to the `/api/settings/info` endpoint for observability.
|
||||
|
||||
- [ ] **Step 1: Import cache and add stats to info response**
|
||||
|
||||
In `server/routes/settings.js`, add to the `GET /api/settings/info` handler:
|
||||
|
||||
```javascript
|
||||
const { transcriptCache } = require("./hooks");
|
||||
```
|
||||
|
||||
In the response object, add:
|
||||
```javascript
|
||||
transcript_cache: transcriptCache.stats(),
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run server tests**
|
||||
|
||||
Run: `npm run test:server`
|
||||
Expected: All pass
|
||||
|
||||
- [ ] **Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add server/routes/settings.js
|
||||
git commit -m "feat: expose transcript cache stats in settings info endpoint"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 8: Integration Smoke Test
|
||||
|
||||
**Files:**
|
||||
- Modify: `server/__tests__/api.test.js`
|
||||
|
||||
Add a test that simulates the full hook event flow with transcript file reads to verify the cache integration works end-to-end.
|
||||
|
||||
- [ ] **Step 1: Add integration test for cached transcript reading**
|
||||
|
||||
Add a new describe block to `server/__tests__/api.test.js`:
|
||||
|
||||
```javascript
|
||||
describe("transcript cache integration", () => {
|
||||
it("should extract tokens from transcript file via hook event", async () => {
|
||||
// Create a temp JSONL transcript file
|
||||
const tmpTranscript = path.join(os.tmpdir(), `test-transcript-${Date.now()}.jsonl`);
|
||||
const entries = [
|
||||
JSON.stringify({ message: { model: "claude-sonnet-4-20250514", usage: { input_tokens: 100, output_tokens: 50, cache_read_input_tokens: 10, cache_creation_input_tokens: 5 } } }),
|
||||
JSON.stringify({ message: { model: "claude-sonnet-4-20250514", usage: { input_tokens: 200, output_tokens: 75, cache_read_input_tokens: 20, cache_creation_input_tokens: 10 } } }),
|
||||
];
|
||||
fs.writeFileSync(tmpTranscript, entries.join("\n") + "\n");
|
||||
|
||||
try {
|
||||
// Send hook event with transcript_path
|
||||
const sessionId = `cache-test-${Date.now()}`;
|
||||
const res = await post("/api/hooks/event", {
|
||||
hook_type: "Stop",
|
||||
data: {
|
||||
session_id: sessionId,
|
||||
transcript_path: tmpTranscript,
|
||||
cwd: "/tmp",
|
||||
},
|
||||
});
|
||||
assert.strictEqual(res.status, 200);
|
||||
|
||||
// Verify tokens were stored
|
||||
const costRes = await fetch(`/api/pricing/cost/${sessionId}`);
|
||||
if (costRes.status === 200 && costRes.body.breakdown) {
|
||||
const sonnet = costRes.body.breakdown.find((b) => b.model.includes("sonnet"));
|
||||
if (sonnet) {
|
||||
assert.strictEqual(sonnet.input_tokens, 300);
|
||||
assert.strictEqual(sonnet.output_tokens, 125);
|
||||
}
|
||||
}
|
||||
} finally {
|
||||
fs.unlinkSync(tmpTranscript);
|
||||
}
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run full server test suite**
|
||||
|
||||
Run: `npm run test:server`
|
||||
Expected: All pass
|
||||
|
||||
- [ ] **Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add server/__tests__/api.test.js
|
||||
git commit -m "test: add integration smoke test for transcript cache via hook events"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 9: Final Build Verification
|
||||
|
||||
- [ ] **Step 1: Run all server tests**
|
||||
|
||||
Run: `npm run test:server`
|
||||
Expected: All pass
|
||||
|
||||
- [ ] **Step 2: Run client build to check nothing broke**
|
||||
|
||||
Run: `npm run build`
|
||||
Expected: Clean build, no errors
|
||||
|
||||
- [ ] **Step 3: Manual smoke test**
|
||||
|
||||
Start the dev server (`npm run dev`) and verify:
|
||||
1. Hook events still process correctly
|
||||
2. Token counts update in the UI
|
||||
3. `/api/settings/info` shows `transcript_cache` stats
|
||||
4. No errors in server console
|
||||
|
||||
- [ ] **Step 4: Final commit if any cleanup needed**
|
||||
|
||||
---
|
||||
|
||||
## Summary of Expected Impact
|
||||
|
||||
| Metric | Before | After |
|
||||
|--------|--------|-------|
|
||||
| File reads per hook event | 1 full read (every line) | 0 reads (cache hit) or partial read (new bytes only) |
|
||||
| Parse calls per hook event | N lines × JSON.parse | 0 (cache hit) or K new lines only |
|
||||
| Periodic scanner file reads | 1 full read per active session every 2min | 0 (shared cache already has data) |
|
||||
| Memory overhead | None | ~1KB per active session (tokens + metadata) |
|
||||
| Event loop blocking | Up to 120ms for large files | <1ms (stat only) on cache hit |
|
||||
|
||||
For a typical long session (20K lines, 4MB), this reduces per-event CPU cost from ~50ms to <1ms — a **50x improvement** on the hot path.
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,117 @@
|
||||
# Stage Auto-Detection Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Infer a lane's stage from the hook stream the dashboard already ingests, and surface it as an explicitly-inferred amber node that can never read as done.
|
||||
|
||||
**Architecture:** Design doc: `docs/superpowers/specs/2026-07-28-stage-detection-design.md` — read it once before Task 1. Rules live in the pipeline template JSON, not in code. One pure function (`server/lib/stage-detect.js`) turns an event into a candidate node; the existing fail-safe block in `touchLaneFromHook` applies it under a forward-only, write-on-change guard; three additive columns hold the result; the client renders it dashed-amber and never green.
|
||||
|
||||
**Tech Stack:** Node 18+, Express, better-sqlite3, `node:test` (server), React 18 + TypeScript + Vitest (client).
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- Branch: `feat/stage-detection`, cut from the head of `feat/worktree-lanes`. Never work on `master`.
|
||||
- Every `.js/.ts/.tsx` file created or modified MUST start with a file overview comment plus the exact line `@author Nguyễn Ngọc Trí Vĩ <vinnt@smartgift.vn>`. Verify with `bash .claude/skills/file-headers/scripts/check-headers.sh` (must exit 0).
|
||||
- **Detection never writes `lanes.stage`.** It writes only `detected_stage`, `detected_signal`, `detected_at`. Declared stage keeps its exact current meaning.
|
||||
- **Inference never renders `done`.** A detected node reaches `passed-no-evidence` at most.
|
||||
- **The hook path must never throw.** Everything added to `server/routes/hooks.js` lives inside the existing try/catch that already swallows lane bookkeeping errors. Claude Code waits on `POST /api/hooks/event`.
|
||||
- Schema changes are additive with one probe per column (`try { SELECT col } catch { ALTER } `), so a crash mid-migration self-heals on the next boot.
|
||||
- Preserve existing behavior: no existing route, response field, WebSocket type, or CLI command changes meaning. `lane_update` stays the only lane WS type.
|
||||
- Server CommonJS. No new npm dependencies. Server tests `node:test` + `node:assert/strict`; client tests Vitest + Testing Library. Exact-value assertions; no bare sleeps.
|
||||
- Docs move with behavior: `docs/LANES.md`, `docs/API.md`, `server/openapi-extra/lanes.js` (+ regenerate `openapi.yaml`), `server/README.md`, `ARCHITECTURE.md` as applicable.
|
||||
- The pre-commit hook runs Prettier plus both suites and takes minutes. Let it finish. NEVER `--no-verify` — on the previous branch it caught a real bug that had been dismissed as an environment quirk.
|
||||
- Baseline at branch point: 857 server tests, 297 client tests, all passing. Each task must leave `git status --short` empty.
|
||||
|
||||
---
|
||||
|
||||
## Task 1 (B1): the rule matcher
|
||||
|
||||
Pure logic, no DB, no HTTP. Everything else depends on its shape.
|
||||
|
||||
**Files:** Create `server/lib/stage-detect.js`; create `server/__tests__/stage-detect.test.js`.
|
||||
|
||||
**Produces:**
|
||||
- `flattenInput(toolInput): string` — a searchable string from a tool's input object (concatenate string values one level deep, plus `command`, `file_path`, `skill`, `prompt` if present). Must tolerate `null`, a string, an array, and deeply nested objects without throwing.
|
||||
- `detect(pipeline, event): {nodeId, signal} | null` — `event` is `{tool_name, tool_input}`. Walks the pipeline's nodes, returns the LAST node whose `detect` rules match (later stage wins when two match, so `git push` beats `Edit`), with `signal` a short human string like `` `npm run test:server` ``. Returns null when nothing matches, when the pipeline has no rules, or when the event has no `tool_name`.
|
||||
- `compileRules(pipeline)` — internal, but exported for testing: precompiles each rule's regex ONCE per pipeline and skips (never throws on) an invalid pattern from a user-supplied template.
|
||||
|
||||
- [ ] **Step 1: write the failing tests.** Cover: `Bash` + `npm run test:server` → `tests`; `Edit` → `implement`; `Skill` + `brainstorming` → `plan`; `Bash` + `git push` → `ship`; a rule with no `match` fires on tool alone; `Read` (mentioned by no rule) → null; a template whose rule holds an invalid regex is skipped and the rest still work; `flattenInput` survives null/string/array/nested; two matching nodes → the later one wins.
|
||||
- [ ] **Step 2: run, confirm they fail** (`node --test server/__tests__/stage-detect.test.js`) — module missing.
|
||||
- [ ] **Step 3: implement.** No DB, no `require` of anything but `node:` builtins.
|
||||
- [ ] **Step 4: run, confirm they pass.**
|
||||
- [ ] **Step 5:** header audit, `npm run test:server`, commit — `feat(lanes): rule matcher for inferring a stage from a tool event`.
|
||||
|
||||
---
|
||||
|
||||
## Task 2 (B2): rules in the template, columns in the database
|
||||
|
||||
**Files:** Modify `server/data/pipelines/default.json`; modify `server/db.js`; modify `server/lib/lanes.js`; modify `server/__tests__/lanes-lib.test.js`.
|
||||
|
||||
**Produces:**
|
||||
- `detect` arrays on the default template's nodes. Ship exactly these, and no others — every rule must be defensible:
|
||||
- `plan`: `Skill` matching `brainstorming|writing-plans`; `Write` matching `docs/.*plan.*\.md`
|
||||
- `implement`: `Edit`; `Write`
|
||||
- `tests`: `Bash` matching `\b(npm (run )?test|pytest|vitest|jest|go test|cargo test)\b`
|
||||
- `review`: `Skill` matching `code-review|requesting-code-review`; `Bash` matching `git diff|gh pr diff`
|
||||
- `ship`: `Bash` matching `git push|gh pr create`
|
||||
- `intake`, `gate`, `done`: no rules. A gate is a judgement and `done` is a claim; neither may be inferred.
|
||||
- Columns `detected_stage`, `detected_signal`, `detected_at` on `lanes`, one probe each.
|
||||
- `recordDetection(id, {nodeId, signal})` in `server/lib/lanes.js` — applies the guard and returns `{written: boolean, reason?: string}`. It writes only when ALL hold: the detection's node index is strictly greater than the current `detected_stage`'s index; and the lane's DECLARED stage index is strictly less than the detection's. Otherwise it returns `written: false` with a reason (`behind-detected`, `behind-declared`, `unknown-node`) and touches nothing.
|
||||
- `lanePayload` gains `detected_stage`, `detected_signal`, and `detected: boolean` on each entry of `pipeline_nodes` (true for the detected node and for nodes before it that carry no declaration).
|
||||
|
||||
- [ ] **Step 1: write the failing tests** in `lanes-lib.test.js`: a forward detection writes; a backward detection returns `behind-detected` and writes nothing; a detection at or behind the declared stage returns `behind-declared`; an unknown node id returns `unknown-node`; **a lane with detections and no declarations has no `done` node in `pipeline_nodes`**; the migration adds all three columns to a database holding an old-schema `lanes` row, and a simulated mid-migration crash self-heals.
|
||||
- [ ] **Step 2: run, confirm they fail.**
|
||||
- [ ] **Step 3: implement** — template rules, the three probes, `recordDetection`, the payload fields.
|
||||
- [ ] **Step 4: run, confirm they pass;** then `npm run test:server`.
|
||||
- [ ] **Step 5:** header audit, commit — `feat(lanes): detection rules in the template, detected columns on lanes`.
|
||||
|
||||
---
|
||||
|
||||
## Task 3 (B3): wire it into the hook stream
|
||||
|
||||
**Files:** Modify `server/routes/hooks.js`; modify `server/__tests__/lanes-api.test.js`.
|
||||
|
||||
**Produces:** no new exports. Inside the EXISTING `touchLaneFromHook` try/catch, after the lane is resolved: build the event from the hook payload (`data.tool_name`, `data.tool_input`), call `detect(getPipeline(lane.pipeline), event)`, and on a hit call `recordDetection`. Broadcast `lane_update` only when `recordDetection` reports `written: true` — Bash alone produced 29 470 events in a real install, so a broadcast per event is not acceptable.
|
||||
|
||||
- [ ] **Step 1: write the failing tests:** a `PostToolUse` hook carrying `Bash: npm run test:server` under a lane's cwd sets `detected_stage` to `tests`; a following `Read` event leaves it unchanged; an event for a path under no lane changes nothing; a lane whose declared stage is already `ship` ignores an `implement` detection; a hook whose `data` is malformed (`tool_input` a string, `tool_name` missing) still returns 200 and leaves the lane untouched.
|
||||
- [ ] **Step 2: run, confirm they fail.**
|
||||
- [ ] **Step 3: implement.** Nothing may be added outside the existing try/catch. Add one short comment naming the write-on-change rule and why (the event volume).
|
||||
- [ ] **Step 4: run, confirm they pass;** then `npm run test:server`.
|
||||
- [ ] **Step 5:** header audit, commit — `feat(lanes): infer a lane's stage from its hook stream`.
|
||||
|
||||
---
|
||||
|
||||
## Task 4 (B4): show it, and never as done
|
||||
|
||||
**Files:** Modify `client/src/lib/types.ts`, `client/src/components/lanes/PipelineMap.tsx`, `client/src/components/lanes/LaneCard.tsx`, `client/src/i18n/locales/{en,zh,vi,ko}/lanes.json`; modify `client/src/components/lanes/__tests__/PipelineMap.test.tsx`.
|
||||
|
||||
**Produces:** `LaneNode` gains `detected?: boolean`; `Lane` gains `detected_stage` and `detected_signal`. A node with `detected: true` renders amber with a **dashed** border, visually distinct from both green `done` and solid-amber `passed-no-evidence`. Its `title` names the signal (`tests ← npm run test:server`). `LaneCard` shows an `auto: <stage>` chip only when the detected stage is ahead of the declared one. All strings via i18n in all four locales.
|
||||
|
||||
- [ ] **Step 1: write the failing tests:** a detected node carries `data-detected="true"` and a dashed-border class token; its class differs from both the `done` and the plain `passed-no-evidence` node; the tooltip contains the signal; **no node with `detected: true` ever carries `data-state="done"`**.
|
||||
- [ ] **Step 2: run, confirm they fail** (`cd client && npx vitest run src/components/lanes/__tests__/PipelineMap.test.tsx`).
|
||||
- [ ] **Step 3: implement.**
|
||||
- [ ] **Step 4:** `npm run test:client`, `npm run build`. If the screens snapshot moves, read the diff and accept it only if it is exactly the intended change.
|
||||
- [ ] **Step 5:** header audit, commit — `feat(lanes): render inferred stages as dashed amber, never as done`.
|
||||
|
||||
---
|
||||
|
||||
## Task 5 (B5): docs, CLI surface, and the honesty pass
|
||||
|
||||
**Files:** Modify `docs/LANES.md`, `docs/API.md`, `server/openapi-extra/lanes.js`, `openapi.yaml` (regenerated), `server/README.md`, `bin/ccam.js`, `server/__tests__/lanes-cli.test.js`.
|
||||
|
||||
**Produces:** `ccam lanes` gains a column or suffix showing the inferred stage when it leads the declared one. `docs/LANES.md` gains a Stage detection section stating: what signals are read; that rules live in the template and how to add one via `DASHBOARD_PIPELINES_DIR`; that detection is forward-only; that declared beats detected; and — prominently — **that an inferred stage never counts as evidence and never renders as done**, with the reason. `docs/API.md` and the OpenAPI fragment document the three new payload fields.
|
||||
|
||||
- [ ] **Step 1: write the failing CLI test:** `ccam lanes` prints the inferred stage for a lane whose detection leads its declaration, and does not print one when the declaration leads.
|
||||
- [ ] **Step 2: run, confirm it fails.**
|
||||
- [ ] **Step 3: implement the CLI change and write the docs.** Every rule you document must match `server/data/pipelines/default.json` exactly — read the file, do not recall it.
|
||||
- [ ] **Step 4:** `npm run test:server`, `npm run test:client`, `node scripts/generate-openapi-yaml.js` then confirm `git diff openapi.yaml` is empty.
|
||||
- [ ] **Step 5:** header audit, commit — `docs(lanes): document stage detection and its evidence boundary`.
|
||||
|
||||
---
|
||||
|
||||
## Out of scope
|
||||
|
||||
- Inferring `done` or any gate outcome.
|
||||
- Back-filling detections for existing lanes.
|
||||
- Reading `workflows.phases` as a signal — real, but it needs its own reconciliation story with the declared stage.
|
||||
- Any write to `lanes.stage` from inference.
|
||||
@@ -0,0 +1,119 @@
|
||||
# Merged Workspace Page Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Merge the Lanes page and the Run page into one Workspace page at `/run` — lane strip, pipeline map, and a full Claude console for the selected lane — without losing any Run capability.
|
||||
|
||||
**Architecture:** Design doc: `docs/superpowers/specs/2026-07-28-workspace-page-design.md` — read it once before Task 1. `client/src/pages/Run.tsx` (3658 lines) is extracted into a hook and three components in three separate mechanical commits, each leaving the existing tests green with only import changes. Only then is the new page composed. Four small server pieces support it: runs start through the lane, an `ensure` endpoint, a `lane_id` on run history, and releasing the lane when a run ends.
|
||||
|
||||
**Tech Stack:** React 18 + TypeScript + Vite + Tailwind, Vitest + Testing Library (client); Node 18+, Express, better-sqlite3, `node:test` (server).
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- Branch: `feat/workspace-page`, cut from the head of `feat/stage-detection` (or of `feat/worktree-lanes` if B has not landed). Never work on `master`.
|
||||
- Every `.js/.ts/.tsx` file created or modified MUST start with a file overview comment plus the exact line `@author Nguyễn Ngọc Trí Vĩ <vinnt@smartgift.vn>`. Verify with `bash .claude/skills/file-headers/scripts/check-headers.sh` (must exit 0).
|
||||
- **Tasks A1-A3 are pure moves.** No behaviour may change, no logic may be "improved" in passing. The existing Run tests must pass with changes to import paths ONLY. If a test needs a real edit to keep passing, stop and report it — that means the move was not pure.
|
||||
- **Extraction and composition never share a commit.** A1, A2, A3 are refactors; A5 builds the page.
|
||||
- **The console never writes a lane's stage.** No code path from the console may call `POST /:id/stage`. Only `ccam stage` declares; only detection infers.
|
||||
- Schema changes are additive with a per-column probe (`try { SELECT col } catch { ALTER }`).
|
||||
- Preserve existing behavior: `POST /api/run` and every existing WebSocket message type keep working exactly as they do — the CLI and other callers depend on them. `lane_update` stays the only lane WS type.
|
||||
- Server CommonJS; client React + TypeScript. No new npm dependencies. Exact-value assertions; bounded polling, no bare sleeps. i18n strings in all four locales (`en`, `zh`, `vi`, `ko`), genuinely translated.
|
||||
- The screens snapshot (`client/src/pages/__tests__/screens.snapshot.test.tsx`) covers `/run`. Read every snapshot diff before accepting it; never regenerate blindly.
|
||||
- The pre-commit hook runs Prettier plus both suites and takes minutes. Let it finish. NEVER `--no-verify`.
|
||||
- Baseline at branch point: 857 server tests, 297 client tests (add B's counts if B landed first). Each task leaves `git status --short` empty.
|
||||
|
||||
---
|
||||
|
||||
## Task 1 (A1): extract `useRunStream`
|
||||
|
||||
**Files:** Create `client/src/hooks/useRunStream.ts`; create `client/src/hooks/__tests__/useRunStream.test.tsx`; modify `client/src/pages/Run.tsx`.
|
||||
|
||||
**Produces:** `useRunStream(runId: string | null)` returning `{envelopes, status, lastAck}`. It owns everything `Run.tsx` currently does with `run_stream` / `run_status` / `run_input_ack`: the envelope merge (`mergeEnvelope`, `findLastStreamingAssistant`, `findAssistantByMessageId`, `mutateAssistantAt`), the typewriter (`useTypewriterEnvelopes`), and the `eventBus.subscribe` lifecycle. Move those functions; do not rewrite them.
|
||||
|
||||
- [ ] **Step 1: write the hook's tests first** — they are new coverage for code that had none: envelopes for the subscribed run id merge in arrival order; an envelope for a different run id is ignored; a streaming assistant envelope updates in place rather than appending; a terminal `run_status` stops further merging; unmounting disposes the subscription (assert the disposer returned by `eventBus.subscribe` was called).
|
||||
- [ ] **Step 2: run, confirm they fail** — `cd client && npx vitest run src/hooks/__tests__/useRunStream.test.tsx`.
|
||||
- [ ] **Step 3: move the code.** Cut the named functions out of `Run.tsx` into the hook, export them if the tests need them, and have `Run.tsx` call the hook. Delete the now-dead copies. Change nothing else.
|
||||
- [ ] **Step 4: verify the move was pure** — `npm run test:client` (all pre-existing Run tests green, no test bodies edited), `npm run build`, and `git diff client/src/pages/__tests__/` must show no snapshot change.
|
||||
- [ ] **Step 5:** header audit, commit — `refactor(run): extract useRunStream from the Run page`.
|
||||
|
||||
---
|
||||
|
||||
## Task 2 (A2): extract `RunConsole`
|
||||
|
||||
**Files:** Create `client/src/components/run/RunConsole.tsx`; modify `client/src/pages/Run.tsx`.
|
||||
|
||||
**Produces:** `<RunConsole runId prompt onPromptChange onSubmit onStop slashCommands busy />` rendering the envelope stream, the prompt editor with its slash autocomplete (`PromptEditor`, `detectAutocomplete`, `scoreSlashMatch`, `subsequenceMatch`, `commandSourceLabel`, `commandSourceTone`), and the token meter (`TokenMeter`, `computeTokens`, `formatNum`). It consumes `useRunStream` from A1. Props only — no direct API calls, so the same console can serve the Run page and the Workspace page.
|
||||
|
||||
- [ ] **Step 1: write the failing test** — `client/src/components/run/__tests__/RunConsole.test.tsx`: given envelopes from a mocked `useRunStream`, the assistant text renders; typing `/co` shows the matching slash command and picking one fills the prompt; the token meter shows the computed totals; `onSubmit` fires with the prompt text; `onStop` fires from the stop control.
|
||||
- [ ] **Step 2: run, confirm it fails.**
|
||||
- [ ] **Step 3: move the code.** Pure move plus the props boundary. Do not redesign the editor.
|
||||
- [ ] **Step 4:** `npm run test:client` (pre-existing Run tests green with only import changes), `npm run build`, snapshot unchanged.
|
||||
- [ ] **Step 5:** header audit, commit — `refactor(run): extract RunConsole from the Run page`.
|
||||
|
||||
---
|
||||
|
||||
## Task 3 (A3): extract `RunSetup` and `RunHistory`, leave `Run.tsx` thin
|
||||
|
||||
**Files:** Create `client/src/components/run/RunSetup.tsx`, `client/src/components/run/RunHistory.tsx`; modify `client/src/pages/Run.tsx`.
|
||||
|
||||
**Produces:** `<RunSetup>` owning mode / model / permission-mode / effort / cwd / resume-session pickers, the binary-status check and `LimitationsBanner`; `<RunHistory>` owning past runs, live runs and attach. After this task `Run.tsx` holds only page-level state and composition — report its final line count in your report.
|
||||
|
||||
- [ ] **Step 1: write the failing tests** for both components: `RunSetup` reports each selection through its callbacks and surfaces a missing-binary state; `RunHistory` lists history, marks a live run, and fires attach with the right run id.
|
||||
- [ ] **Step 2: run, confirm they fail.**
|
||||
- [ ] **Step 3: move the code.**
|
||||
- [ ] **Step 4:** `npm run test:client`, `npm run build`, snapshot unchanged.
|
||||
- [ ] **Step 5:** header audit, commit — `refactor(run): extract RunSetup and RunHistory, thin the Run page`.
|
||||
|
||||
---
|
||||
|
||||
## Task 4 (A4): the server glue
|
||||
|
||||
**Files:** Modify `server/db.js`, `server/lib/lanes.js`, `server/routes/lanes.js`, `server/lib/run-spawner.js`, `server/__tests__/lane-lifecycle.test.js`; docs as listed in the constraints.
|
||||
|
||||
**Produces:**
|
||||
- `POST /api/lanes/ensure` `{cwd, title?}` → `{lane, created: boolean}`. Returns the lane that owns `cwd` (exact match or the longest path-boundary parent, reusing `resolveLaneByCwd`), else creates an `adopted` lane. Behind the same-origin guard. Concurrent calls for the same path must yield ONE lane — rely on the `cwd` UNIQUE constraint and treat the constraint violation as "someone else created it, re-read and return it".
|
||||
- `mode` accepted by the lane `start` action and passed through to `spawnRun`, so a headless one-shot is reachable through a lane.
|
||||
- `dashboard_runs.lane_id`, one additive probe, written when a run starts through a lane; `GET /api/run/history` accepts an optional `laneId` filter.
|
||||
- **A finished run releases its lane.** When the run-spawner observes a child's real exit, clear `run_id` and set `status: "idle"` on the lane holding that `run_id`, and broadcast `lane_update`. Do this without creating a require cycle (`run-spawner` must not import a route module — read how `broadcastLane` is exported and pick the clean direction, or invert it with a callback registered at boot). State in your report which direction you chose and why.
|
||||
|
||||
- [ ] **Step 1: write the failing tests:** `ensure` returns the existing lane for an exact path, for a nested path, and creates one otherwise; two concurrent `ensure` calls for the same path create exactly one lane; a lane-started run records `lane_id` in `dashboard_runs` and `GET /api/run/history?laneId=` filters by it; **when a run ends on its own, the lane's `run_id` becomes null and its status returns to `idle`**; a cross-origin `POST /api/lanes/ensure` is refused.
|
||||
- [ ] **Step 2: run, confirm they fail.**
|
||||
- [ ] **Step 3: implement.**
|
||||
- [ ] **Step 4:** `npm run test:server`; regenerate `openapi.yaml` and confirm `git diff openapi.yaml` is empty.
|
||||
- [ ] **Step 5:** header audit, commit — `feat(lanes): ensure endpoint, run history per lane, release the lane when a run ends`.
|
||||
|
||||
---
|
||||
|
||||
## Task 5 (A5): compose the Workspace page
|
||||
|
||||
**Files:** Create `client/src/pages/Workspace.tsx`; modify `client/src/App.tsx`, `client/src/components/Sidebar.tsx`, `client/src/lib/api.ts`, `client/src/i18n/locales/*/lanes.json`; modify `client/src/pages/__tests__/screens.snapshot.test.tsx`.
|
||||
|
||||
**Produces:** the merged page at `/run`; `/lanes` redirects to it (`<Navigate to="/run" replace />`); one sidebar entry. Layout top to bottom: lane strip (horizontal scroll, counters, Add) → `PipelineMap` for the selected lane → `RunSetup` → `RunConsole` → `RunHistory` filtered to the lane. Selecting a lane switches pipeline, console and history together. Starting a run goes through `POST /api/lanes/:id/start`; choosing a cwd that no lane owns calls `POST /api/lanes/ensure` first. `api.lanes` gains `ensure`.
|
||||
|
||||
- [ ] **Step 1: write the failing tests** — `client/src/pages/__tests__/Workspace.test.tsx`: the strip lists lanes and the counters match; selecting a lane switches the pipeline and the console's run id; starting a run posts to the LANE start endpoint (assert the URL, not just that something was called); picking an unowned cwd calls `ensure` before `start`; **after a full start-and-message cycle the lane's stage is never posted to** (assert no call to any `/stage` URL).
|
||||
- [ ] **Step 2: run, confirm they fail.**
|
||||
- [ ] **Step 3: implement.** Keep `Run.tsx`'s remaining shell only if something still needs it; if the Workspace page fully replaces it, delete it and say so.
|
||||
- [ ] **Step 4:** `npm run test:client`, `npm run build`. The screens snapshot WILL change here — read the diff, confirm it is only the merged layout, then regenerate.
|
||||
- [ ] **Step 5:** header audit, commit — `feat(lanes): merge the Lanes and Run pages into one Workspace`.
|
||||
|
||||
---
|
||||
|
||||
## Task 6 (A6): docs and the seams
|
||||
|
||||
**Files:** Modify `docs/LANES.md`, `docs/API.md`, `server/README.md`, `ARCHITECTURE.md`, `README.md`, `server/openapi-extra/lanes.js` (+ regenerate `openapi.yaml`), `CLAUDE.md`.
|
||||
|
||||
**Produces:** documentation of the merged page and the new seams: that `/lanes` redirects to `/run`; that the UI starts runs through the lane while `POST /api/run` remains for the CLI; the `ensure` endpoint and when the UI calls it; `dashboard_runs.lane_id`; that a finished run releases its lane. `CLAUDE.md` gains the rule: **the console never writes a lane's stage — declared comes from `ccam stage`, inferred from detection.** Every path and command you print must exist; verify each.
|
||||
|
||||
- [ ] **Step 1:** write the docs.
|
||||
- [ ] **Step 2:** verify every referenced file, route and command exists (`ls`, `grep`, or run it).
|
||||
- [ ] **Step 3:** `npm run test:server`, `npm run test:client`, `node scripts/generate-openapi-yaml.js` then `git diff openapi.yaml` empty.
|
||||
- [ ] **Step 4:** header audit, commit — `docs(lanes): document the merged Workspace page and its seams`.
|
||||
|
||||
---
|
||||
|
||||
## Out of scope
|
||||
|
||||
- Any redesign of the prompt editor, the envelope renderer, or the token meter. A1-A3 move them unchanged.
|
||||
- Multiple concurrent runs per lane. `start` already 409s when one is live.
|
||||
- Per-lane dependency bootstrap for a fresh worktree (still sub-project D if ever wanted).
|
||||
- Stage inference — that is sub-project B, and the console must not do it either way.
|
||||
@@ -0,0 +1,580 @@
|
||||
# Worktree-backed Lanes Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Let a lane own a git worktree that CCAM creates, resets, removes and purges, with every destructive action gated on counted facts and on three independent safety checks.
|
||||
|
||||
**Architecture:** Design doc: `docs/superpowers/specs/2026-07-28-worktree-lanes-design.md` — read it once before Task 1. All git work goes through one module (`server/lib/worktree.js`) that shells out with `execFile` and an argv array, never a shell string, and re-verifies its own safety preconditions. Lanes gain a `kind` of `adopted` (pointer at a directory the user already had — never destroyable) or `managed` (a worktree CCAM created — destroyable). Destructive actions are serialised per lane and preceded by a preflight endpoint that returns counts, which the confirmation UI renders and the server re-checks before acting.
|
||||
|
||||
**Tech Stack:** Node 18+, Express, better-sqlite3, `node:child_process.execFile`, real `git` against temp-directory fixtures, `node:test` (server), React 18 + TypeScript + Vitest (client).
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- Branch: create `feat/worktree-lanes` off the current head of `feat/lanes-pipeline`. Never work on `master`.
|
||||
- Every `.js/.ts/.tsx` file created or modified MUST start with a file overview comment plus the exact line `@author Nguyễn Ngọc Trí Vĩ <vinnt@smartgift.vn>`. Verify with `bash .claude/skills/file-headers/scripts/check-headers.sh` (must exit 0).
|
||||
- Schema changes are additive only and migration-safe on an existing database: `try { SELECT col } catch { ALTER TABLE … ADD COLUMN }`, the pattern at `server/db.js:412-418`. Existing rows must migrate to `kind='adopted'`.
|
||||
- **Never `rm -rf` a lane directory.** Removal goes through `git worktree remove`; if git refuses, surface git's error unchanged.
|
||||
- **Never build a shell command string.** `execFile("git", [...args])` only. No `shell: true`, no template-literal commands.
|
||||
- Destructive routes stay behind the existing same-origin guard exported from `server/routes/run.js`.
|
||||
- Preserve existing behavior: no existing route, response shape, WebSocket type, or CLI command changes meaning. `lane_update` stays the only lane WS type.
|
||||
- Server is CommonJS. No new npm dependencies. Server tests use `node:test` + `node:assert/strict`; client tests use Vitest + Testing Library.
|
||||
- The pre-commit hook runs Prettier and the full server suite; a commit takes minutes. Do not disable it.
|
||||
- Baseline before this plan: 790 server tests, 279 client tests, all passing.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
**Create**
|
||||
- `server/lib/worktree.js` — every git invocation, plus the three-check safety guard. No Express, no DB.
|
||||
- `server/lib/lane-preflight.js` — counts for `reset` / `remove` / `purge`. Reads git and the DB; mutates nothing.
|
||||
- `server/lib/lane-lock.js` — per-lane async mutex.
|
||||
- `server/__tests__/worktree.test.js` — git behaviour against a real temp repo.
|
||||
- `server/__tests__/lane-lifecycle.test.js` — HTTP: add / preflight / reset / remove / purge.
|
||||
- `client/src/components/lanes/DestructiveLaneModal.tsx` — preflight table inside the existing `ConfirmModal`.
|
||||
|
||||
**Modify**
|
||||
- `server/db.js` — four additive columns.
|
||||
- `server/lib/lanes.js` — `kind`/`source_repo`/`base_branch`/`slug` in create/patch/payload; `purgeLaneSessions`.
|
||||
- `server/routes/lanes.js` — `POST /worktree`, `GET /:id/preflight`, `reset` + `purge` actions, lock usage.
|
||||
- `bin/ccam.js` — `ccam lanes add --repo`, `ccam lanes reset|remove|purge`.
|
||||
- `client/src/lib/api.ts`, `client/src/lib/types.ts` — preflight + worktree types and calls.
|
||||
- `client/src/components/lanes/LaneCard.tsx` — kind badge; destructive buttons only for `managed`.
|
||||
- `docs/LANES.md`, `CLAUDE.md` — the new lifecycle.
|
||||
|
||||
---
|
||||
|
||||
## Task 1: `server/lib/worktree.js` — git plumbing and the safety guard
|
||||
|
||||
**Files:**
|
||||
- Create: `server/lib/worktree.js`
|
||||
- Test: `server/__tests__/worktree.test.js`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: nothing from earlier tasks.
|
||||
- Produces:
|
||||
- `LANES_ROOT` — `process.env.LANES_ROOT || path.join(os.homedir(), ".claude", "ccam-lanes")`
|
||||
- `git(cwd, args): Promise<{stdout, stderr}>` — rejects with `err.git = {args, code, stderr}` on non-zero
|
||||
- `isGitRepo(dir): Promise<boolean>`
|
||||
- `resolveBase(sourceRepo, wanted): Promise<string>` — `origin/<wanted>` → `<wanted>` → current HEAD
|
||||
- `slugify(text): string` — lowercase, non-alphanumerics to `-`, collapsed, trimmed, max 40 chars
|
||||
- `listWorktrees(sourceRepo): Promise<Array<{path, branch, locked}>>` — parses `--porcelain`
|
||||
- `branchCheckedOutAt(sourceRepo, branch): Promise<string|null>`
|
||||
- `addWorktree({sourceRepo, dir, branch, base}): Promise<{dir, branch, created: boolean}>`
|
||||
- `assertDestroyable(lane): Promise<void>` — the three checks; throws `err.code = "ENOTMANAGED" | "EOUTSIDEROOT" | "ENOTWORKTREE"`
|
||||
- `resetWorktree(lane): Promise<void>`
|
||||
- `removeWorktree(lane): Promise<void>`
|
||||
- `statusCounts(dir): Promise<{dirty, untracked, head}>`
|
||||
- `unpushedCount(dir): Promise<number>`
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
Create `server/__tests__/worktree.test.js`. It builds a real repository in a temp directory — mocks would test nothing that matters here.
|
||||
|
||||
```js
|
||||
/**
|
||||
* @file Tests for server/lib/worktree.js against a REAL git repository created
|
||||
* in a temp directory. Every behaviour worth testing here is git's own — branch
|
||||
* collisions, what `clean -fd` spares, what `worktree list` reports — so mocking
|
||||
* git would only test our idea of git.
|
||||
* @author Nguyễn Ngọc Trí Vĩ <vinnt@smartgift.vn>
|
||||
*/
|
||||
|
||||
const { describe, it, before, after } = require("node:test");
|
||||
const assert = require("node:assert/strict");
|
||||
const fs = require("node:fs");
|
||||
const os = require("node:os");
|
||||
const path = require("node:path");
|
||||
const { execFileSync } = require("node:child_process");
|
||||
|
||||
const ROOT = fs.mkdtempSync(path.join(os.tmpdir(), "ccam-wt-"));
|
||||
process.env.LANES_ROOT = path.join(ROOT, "lanes");
|
||||
|
||||
const wt = require("../lib/worktree");
|
||||
|
||||
const SRC = path.join(ROOT, "src-repo");
|
||||
const g = (cwd, ...args) => execFileSync("git", args, { cwd, encoding: "utf8" });
|
||||
|
||||
before(() => {
|
||||
fs.mkdirSync(SRC, { recursive: true });
|
||||
g(SRC, "init", "-b", "main");
|
||||
g(SRC, "config", "user.email", "t@example.com");
|
||||
g(SRC, "config", "user.name", "Test");
|
||||
fs.writeFileSync(path.join(SRC, "README.md"), "hello\n");
|
||||
fs.writeFileSync(path.join(SRC, ".gitignore"), "node_modules/\n.env\n");
|
||||
g(SRC, "add", "-A");
|
||||
g(SRC, "commit", "-m", "init");
|
||||
});
|
||||
|
||||
after(() => fs.rmSync(ROOT, { recursive: true, force: true }));
|
||||
|
||||
function laneFor(dir, branch, over = {}) {
|
||||
return { id: 1, kind: "managed", cwd: dir, branch, source_repo: SRC, base_branch: "main", ...over };
|
||||
}
|
||||
|
||||
describe("worktree", () => {
|
||||
it("slugifies a title into a safe single segment", () => {
|
||||
assert.equal(wt.slugify("Rename Metric → Rule!"), "rename-metric-rule");
|
||||
assert.equal(wt.slugify(" a//b "), "a-b");
|
||||
assert.ok(wt.slugify("x".repeat(80)).length <= 40);
|
||||
});
|
||||
|
||||
it("resolves the base branch, falling back when origin has none", async () => {
|
||||
assert.equal(await wt.resolveBase(SRC, "main"), "main");
|
||||
assert.equal(await wt.resolveBase(SRC, "does-not-exist"), "main");
|
||||
});
|
||||
|
||||
it("creates a worktree on a new branch and lists it", async () => {
|
||||
const dir = path.join(process.env.LANES_ROOT, "src-repo__alpha");
|
||||
const r = await wt.addWorktree({ sourceRepo: SRC, dir, branch: "feat/alpha", base: "main" });
|
||||
assert.equal(r.created, true);
|
||||
assert.ok(fs.existsSync(path.join(dir, "README.md")));
|
||||
const list = await wt.listWorktrees(SRC);
|
||||
assert.ok(list.some((w) => w.path === dir && w.branch === "feat/alpha"));
|
||||
});
|
||||
|
||||
it("refuses a branch already checked out in another worktree", async () => {
|
||||
const dir2 = path.join(process.env.LANES_ROOT, "src-repo__alpha2");
|
||||
await assert.rejects(
|
||||
() => wt.addWorktree({ sourceRepo: SRC, dir: dir2, branch: "feat/alpha", base: "main" }),
|
||||
(e) => e.code === "EBRANCHBUSY" && typeof e.checkedOutAt === "string",
|
||||
);
|
||||
});
|
||||
|
||||
it("counts dirty, untracked and unpushed work", async () => {
|
||||
const dir = path.join(process.env.LANES_ROOT, "src-repo__alpha");
|
||||
fs.appendFileSync(path.join(dir, "README.md"), "edit\n");
|
||||
fs.writeFileSync(path.join(dir, "scratch.txt"), "untracked\n");
|
||||
fs.mkdirSync(path.join(dir, "node_modules"), { recursive: true });
|
||||
fs.writeFileSync(path.join(dir, "node_modules", "dep.js"), "x\n");
|
||||
const s = await wt.statusCounts(dir);
|
||||
assert.equal(s.dirty, 1);
|
||||
assert.equal(s.untracked, 1); // node_modules is ignored, so it does not count
|
||||
assert.match(s.head, /^[0-9a-f]{7,40}$/);
|
||||
assert.equal(await wt.unpushedCount(dir), 0); // no upstream yet
|
||||
});
|
||||
|
||||
it("reset restores base, drops untracked files, and spares gitignored ones", async () => {
|
||||
const dir = path.join(process.env.LANES_ROOT, "src-repo__alpha");
|
||||
await wt.resetWorktree(laneFor(dir, "feat/alpha"));
|
||||
assert.equal(fs.readFileSync(path.join(dir, "README.md"), "utf8"), "hello\n");
|
||||
assert.equal(fs.existsSync(path.join(dir, "scratch.txt")), false);
|
||||
assert.equal(fs.existsSync(path.join(dir, "node_modules", "dep.js")), true);
|
||||
const s = await wt.statusCounts(dir);
|
||||
assert.equal(s.dirty, 0);
|
||||
});
|
||||
|
||||
it("refuses to destroy an adopted lane, a path outside LANES_ROOT, or a non-worktree", async () => {
|
||||
const dir = path.join(process.env.LANES_ROOT, "src-repo__alpha");
|
||||
await assert.rejects(
|
||||
() => wt.assertDestroyable(laneFor(dir, "feat/alpha", { kind: "adopted" })),
|
||||
(e) => e.code === "ENOTMANAGED",
|
||||
);
|
||||
await assert.rejects(
|
||||
() => wt.assertDestroyable(laneFor("/tmp", "feat/alpha")),
|
||||
(e) => e.code === "EOUTSIDEROOT",
|
||||
);
|
||||
const ghost = path.join(process.env.LANES_ROOT, "src-repo__ghost");
|
||||
fs.mkdirSync(ghost, { recursive: true });
|
||||
await assert.rejects(
|
||||
() => wt.assertDestroyable(laneFor(ghost, "feat/ghost")),
|
||||
(e) => e.code === "ENOTWORKTREE",
|
||||
);
|
||||
});
|
||||
|
||||
it("removes the worktree and its branch, leaving git's list clean", async () => {
|
||||
const dir = path.join(process.env.LANES_ROOT, "src-repo__alpha");
|
||||
await wt.removeWorktree(laneFor(dir, "feat/alpha"));
|
||||
assert.equal(fs.existsSync(dir), false);
|
||||
const list = await wt.listWorktrees(SRC);
|
||||
assert.equal(list.some((w) => w.path === dir), false);
|
||||
const branches = g(SRC, "branch", "--list", "feat/alpha").trim();
|
||||
assert.equal(branches, "");
|
||||
});
|
||||
|
||||
it("never deletes the base branch even if a lane claims it", async () => {
|
||||
const dir = path.join(process.env.LANES_ROOT, "src-repo__beta");
|
||||
await wt.addWorktree({ sourceRepo: SRC, dir, branch: "feat/beta", base: "main" });
|
||||
await wt.removeWorktree(laneFor(dir, "main")); // lane lies about its branch
|
||||
assert.match(g(SRC, "branch", "--list", "main"), /main/);
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `node --test server/__tests__/worktree.test.js`
|
||||
Expected: FAIL — `Cannot find module '../lib/worktree'`.
|
||||
|
||||
- [ ] **Step 3: Implement the module**
|
||||
|
||||
Create `server/lib/worktree.js`. Key requirements the tests pin, restated so nothing is inferred:
|
||||
|
||||
- `git(cwd, args)` wraps `execFile("git", args, {cwd, maxBuffer: 8 * 1024 * 1024})` promisified. On failure throw an `Error` carrying `err.git = { args, code, stderr }`.
|
||||
- `slugify` — `text.toLowerCase().normalize("NFKD").replace(/[^a-z0-9]+/g, "-").replace(/^-+|-+$/g, "").slice(0, 40)`, and if the result is empty throw `err.code = "EBADSLUG"`.
|
||||
- `resolveBase(sourceRepo, wanted)` — try `git rev-parse --verify --quiet origin/<wanted>`, then `<wanted>`, then `git rev-parse --abbrev-ref HEAD`. Return the first that resolves.
|
||||
- `listWorktrees` — parse `git worktree list --porcelain`: records separated by blank lines, `worktree <path>`, `branch refs/heads/<name>`, bare `locked` line. Return `{path, branch, locked}` with `branch` null for a detached worktree.
|
||||
- `branchCheckedOutAt(sourceRepo, branch)` — the `path` from `listWorktrees` whose branch matches, else null.
|
||||
- `addWorktree({sourceRepo, dir, branch, base})`:
|
||||
- if `branchCheckedOutAt` returns a path, throw `err.code = "EBRANCHBUSY"`, `err.checkedOutAt = thatPath`
|
||||
- `fs.mkdirSync(path.dirname(dir), {recursive: true})`
|
||||
- if `git rev-parse --verify --quiet <branch>` succeeds, run `worktree add <dir> <branch>` and return `{created: false}`; otherwise `worktree add -b <branch> <dir> <base>` and return `{created: true}`
|
||||
- `assertDestroyable(lane)` — in order: `kind !== "managed"` → `ENOTMANAGED`; `fs.realpathSync(lane.cwd)` not inside `fs.realpathSync(LANES_ROOT)` on a path boundary → `EOUTSIDEROOT` (a non-existent path fails this check too, which is correct — it cannot be a live worktree); not present in `listWorktrees(lane.source_repo)` → `ENOTWORKTREE`.
|
||||
- `PROTECTED_BRANCHES = new Set(["main", "master"])`, plus the lane's own `base_branch`: `deleteBranchSafely(sourceRepo, branch, baseBranch)` returns without acting when the branch is protected or falsy.
|
||||
- `resetWorktree(lane)` — `assertDestroyable` first, then, all in `lane.cwd`: `fetch origin --prune` (tolerate failure when there is no remote), `checkout <base>` (creating it from `origin/<base>` if absent), `reset --hard <base>`, `clean -fd` (**never** `-x`), then in `source_repo` `deleteBranchSafely(lane.branch)`, then back in the worktree `checkout -b <lane.branch> <base>`.
|
||||
- `removeWorktree(lane)` — `assertDestroyable`, then in `source_repo`: `worktree unlock <dir>` (ignore failure), `worktree remove --force <dir>`, `worktree prune`, `deleteBranchSafely(lane.branch, lane.base_branch)`. If `worktree remove` fails, rethrow git's error untouched — do not fall back to filesystem deletion.
|
||||
- `statusCounts(dir)` — parse `git status --porcelain=v1 --untracked-files=normal`: lines starting `??` are untracked, others dirty. `head` from `git rev-parse --short HEAD`.
|
||||
- `unpushedCount(dir)` — `git rev-list --count @{u}..HEAD`; when there is no upstream, git exits non-zero — return 0.
|
||||
|
||||
- [ ] **Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `node --test server/__tests__/worktree.test.js`
|
||||
Expected: PASS, 8 tests.
|
||||
|
||||
- [ ] **Step 5: Header audit and commit**
|
||||
|
||||
Run: `bash .claude/skills/file-headers/scripts/check-headers.sh`
|
||||
|
||||
```bash
|
||||
git add server/lib/worktree.js server/__tests__/worktree.test.js
|
||||
git commit -m "feat(lanes): git worktree plumbing with a three-check destroy guard"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Schema, lane fields, and per-lane locking
|
||||
|
||||
**Files:**
|
||||
- Modify: `server/db.js` (the `lanes` block)
|
||||
- Modify: `server/lib/lanes.js`
|
||||
- Create: `server/lib/lane-lock.js`
|
||||
- Test: `server/__tests__/lanes-lib.test.js` (append a `describe`)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: nothing from Task 1 (kept independent so both can be reviewed alone).
|
||||
- Produces:
|
||||
- four columns on `lanes`: `kind` (`NOT NULL DEFAULT 'adopted'`), `source_repo`, `base_branch`, `slug`
|
||||
- `createLane` accepts and stores `kind`, `source_repo`, `base_branch`, `slug`; unknown values of `kind` are rejected with `err.code = "EBADKIND"`
|
||||
- `PATCHABLE` gains `kind`, `source_repo`, `base_branch`, `slug`
|
||||
- `purgeLaneSessions(id): {sessions, events, tokenRows}` — deletes the lane's sessions (never the one in `lanes.session_id`), their events, and `token_usage` rows left orphaned
|
||||
- `server/lib/lane-lock.js`: `withLaneLock(id, fn): Promise<any>` — serialises per lane id, releases on throw
|
||||
|
||||
- [ ] **Step 1: Write the failing test** (append to `server/__tests__/lanes-lib.test.js`)
|
||||
|
||||
```js
|
||||
const { withLaneLock } = require("../lib/lane-lock");
|
||||
|
||||
describe("lane kind, worktree fields and purge", () => {
|
||||
it("defaults to adopted and stores worktree fields when given", () => {
|
||||
const a = lanes.createLane({ cwd: "/tmp/wt-kind-a" });
|
||||
assert.equal(a.kind, "adopted");
|
||||
const m = lanes.createLane({
|
||||
cwd: "/tmp/wt-kind-b", kind: "managed",
|
||||
source_repo: "/tmp/src", base_branch: "main", slug: "b",
|
||||
});
|
||||
assert.equal(m.kind, "managed");
|
||||
assert.equal(m.source_repo, "/tmp/src");
|
||||
assert.equal(m.base_branch, "main");
|
||||
assert.equal(m.slug, "b");
|
||||
lanes.deleteLane(a.id);
|
||||
lanes.deleteLane(m.id);
|
||||
});
|
||||
|
||||
it("rejects an unknown kind", () => {
|
||||
assert.throws(() => lanes.createLane({ cwd: "/tmp/wt-kind-c", kind: "gremlin" }),
|
||||
(e) => e.code === "EBADKIND");
|
||||
});
|
||||
|
||||
it("purges a lane's sessions, their events and orphaned token rows, sparing the live one", () => {
|
||||
const l = lanes.createLane({ cwd: "/tmp/wt-purge" });
|
||||
const { db } = require("../db");
|
||||
db.prepare("INSERT INTO sessions (id, cwd, status) VALUES (?, ?, 'completed')").run("purge-1", "/tmp/wt-purge/sub");
|
||||
db.prepare("INSERT INTO sessions (id, cwd, status) VALUES (?, ?, 'active')").run("purge-live", "/tmp/wt-purge");
|
||||
db.prepare("INSERT INTO events (session_id, event_type) VALUES (?, 'PostToolUse')").run("purge-1");
|
||||
db.prepare("INSERT INTO token_usage (session_id, model, input_tokens) VALUES (?, 'm', 5)").run("purge-1");
|
||||
lanes.updateLane(l.id, { session_id: "purge-live" });
|
||||
|
||||
const counts = lanes.purgeLaneSessions(l.id);
|
||||
assert.equal(counts.sessions, 1);
|
||||
assert.equal(counts.events, 1);
|
||||
assert.equal(counts.tokenRows, 1);
|
||||
assert.equal(db.prepare("SELECT COUNT(*) c FROM sessions WHERE id='purge-live'").get().c, 1);
|
||||
assert.equal(db.prepare("SELECT COUNT(*) c FROM events WHERE session_id='purge-1'").get().c, 0);
|
||||
assert.equal(db.prepare("SELECT COUNT(*) c FROM token_usage WHERE session_id='purge-1'").get().c, 0);
|
||||
lanes.deleteLane(l.id);
|
||||
});
|
||||
|
||||
it("serialises work per lane and releases the lock when the body throws", async () => {
|
||||
const order = [];
|
||||
const slow = withLaneLock(7, async () => { order.push("a-start"); await new Promise((r) => setTimeout(r, 50)); order.push("a-end"); });
|
||||
const fast = withLaneLock(7, async () => { order.push("b"); });
|
||||
await Promise.all([slow, fast]);
|
||||
assert.deepEqual(order, ["a-start", "a-end", "b"]);
|
||||
await assert.rejects(() => withLaneLock(7, async () => { throw new Error("boom"); }));
|
||||
await withLaneLock(7, async () => order.push("c"));
|
||||
assert.equal(order[order.length - 1], "c");
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `node --test server/__tests__/lanes-lib.test.js`
|
||||
Expected: FAIL — `Cannot find module '../lib/lane-lock'`.
|
||||
|
||||
- [ ] **Step 3: Add the columns**
|
||||
|
||||
In `server/db.js`, after the `lanes` table and its index, following the probe pattern at `server/db.js:412-418`:
|
||||
|
||||
```js
|
||||
// Managed lanes own a git worktree CCAM created and may be destroyed; adopted
|
||||
// lanes merely point at a directory the user already had and never may be.
|
||||
// Existing rows default to 'adopted', so no lane gains a destructive path by
|
||||
// upgrading.
|
||||
try {
|
||||
db.prepare("SELECT kind FROM lanes LIMIT 1").get();
|
||||
} catch {
|
||||
db.prepare("ALTER TABLE lanes ADD COLUMN kind TEXT NOT NULL DEFAULT 'adopted'").run();
|
||||
db.prepare("ALTER TABLE lanes ADD COLUMN source_repo TEXT").run();
|
||||
db.prepare("ALTER TABLE lanes ADD COLUMN base_branch TEXT").run();
|
||||
db.prepare("ALTER TABLE lanes ADD COLUMN slug TEXT").run();
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Extend `server/lib/lanes.js` and write the lock**
|
||||
|
||||
`createLane` gains the four fields (validating `kind` against `new Set(["adopted", "managed"])`), `PATCHABLE` gains them, and `purgeLaneSessions(id)` runs inside one `db.transaction`:
|
||||
|
||||
- select the lane's sessions: `WHERE (cwd = ? OR cwd LIKE ? || '/%')` against `lane.cwd`, excluding `lanes.session_id` and any session whose `status = 'active'`
|
||||
- count and delete their `events`, then their `token_usage`, then the sessions themselves
|
||||
- return `{sessions, events, tokenRows}`
|
||||
- run `db.pragma("optimize")` after the transaction commits — never `VACUUM`, which locks the whole database
|
||||
|
||||
Create `server/lib/lane-lock.js` — a `Map<laneId, Promise>` chain:
|
||||
|
||||
```js
|
||||
const chains = new Map();
|
||||
|
||||
function withLaneLock(id, fn) {
|
||||
const key = String(id);
|
||||
const prev = chains.get(key) || Promise.resolve();
|
||||
const run = prev.then(fn, fn); // run regardless of how the previous holder settled
|
||||
// Keep the chain alive but never let a rejection poison the next waiter.
|
||||
chains.set(key, run.then(() => {}, () => {}));
|
||||
return run;
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Run tests to verify they pass**
|
||||
|
||||
Run: `node --test server/__tests__/lanes-lib.test.js`
|
||||
Expected: PASS — the four new tests plus every earlier one.
|
||||
|
||||
- [ ] **Step 6: Full suite and commit**
|
||||
|
||||
Run: `npm run test:server`
|
||||
|
||||
```bash
|
||||
git add server/db.js server/lib/lanes.js server/lib/lane-lock.js server/__tests__/lanes-lib.test.js
|
||||
git commit -m "feat(lanes): managed/adopted kinds, worktree fields, session purge, per-lane lock"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Preflight — counted facts before anything destructive
|
||||
|
||||
**Files:**
|
||||
- Create: `server/lib/lane-preflight.js`
|
||||
- Test: `server/__tests__/lane-lifecycle.test.js` (new file; later tasks append to it)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `statusCounts`, `unpushedCount`, `listWorktrees` (Task 1); `getLane` (Task 2).
|
||||
- Produces: `preflight(lane, action): Promise<object>` where `action ∈ "reset" | "remove" | "purge"`.
|
||||
- `reset` / `remove` → `{action, lane, kind, branch, dirty, untracked, unpushed, head, blocked: string[], warnings: string[]}`
|
||||
- `purge` → `{action, lane, sessions, events, tokenRows, bytesEstimate, activeSessionSkipped: boolean}`
|
||||
- `blocked` contains `"adopted"` when the lane is not managed, `"missing"` when the directory is gone, and `"unpushed-commits"` when `unpushed > 0`. It is advisory data, not an exception — the route decides.
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
Create `server/__tests__/lane-lifecycle.test.js` with the standard harness (temp `DASHBOARD_DB_PATH`, `DASHBOARD_REMOTE_SYNC_MS=0`, `DASHBOARD_LIVENESS_PROBE=0`, `LANES_ROOT` pointed at a temp dir, `startServer(createApp(), 0)`; copy the request helper from `server/__tests__/lanes-api.test.js`), plus a real git fixture repo as in Task 1. Tests:
|
||||
|
||||
```js
|
||||
it("preflight on an adopted lane blocks and counts nothing", async () => { /* create adopted lane, GET preflight?action=reset, expect blocked includes "adopted" */ });
|
||||
it("preflight counts dirty, untracked and unpushed for a managed lane", async () => { /* dirty the worktree, expect dirty:1 untracked:1 and a head sha */ });
|
||||
it("preflight for purge counts only this lane's non-live sessions", async () => { /* two sessions, one bound live, expect sessions:1 and activeSessionSkipped:true */ });
|
||||
it("preflight 404s for an unknown lane and 400s for an unknown action", async () => {});
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run to verify it fails**
|
||||
|
||||
Run: `node --test server/__tests__/lane-lifecycle.test.js`
|
||||
Expected: FAIL — the route does not exist yet (404 with an HTML body).
|
||||
|
||||
- [ ] **Step 3: Implement `server/lib/lane-preflight.js` and the route**
|
||||
|
||||
The module is read-only. `bytesEstimate` is `(events + tokenRows) * 512` — label it in `docs/LANES.md` as a rough estimate, because a real per-row size needs `dbstat`, which is not compiled in by default.
|
||||
|
||||
In `server/routes/lanes.js`, add **before** the `/:id/:action` route so it is not swallowed:
|
||||
|
||||
```js
|
||||
router.get("/:id/preflight", async (req, res) => {
|
||||
const lane = lanesLib.getLane(req.params.id);
|
||||
if (!lane) return res.status(404).json({ error: { code: "ENOLANE", message: "lane not found" } });
|
||||
const action = String(req.query.action || "");
|
||||
if (!["reset", "remove", "purge"].includes(action)) {
|
||||
return res.status(400).json({ error: { code: "EBADACTION", message: `unknown action ${action}` } });
|
||||
}
|
||||
try {
|
||||
res.json(await preflight(lane, action));
|
||||
} catch (err) {
|
||||
res.status(500).json({ error: { code: err.code, message: err.message } });
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run to verify it passes**
|
||||
|
||||
Run: `node --test server/__tests__/lane-lifecycle.test.js`
|
||||
Expected: PASS, 4 tests.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add server/lib/lane-preflight.js server/routes/lanes.js server/__tests__/lane-lifecycle.test.js
|
||||
git commit -m "feat(lanes): preflight counts for reset, remove and purge"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 4: `add` — provision a worktree in the background
|
||||
|
||||
**Files:**
|
||||
- Modify: `server/routes/lanes.js`
|
||||
- Test: `server/__tests__/lane-lifecycle.test.js` (append)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `addWorktree`, `resolveBase`, `slugify`, `LANES_ROOT` (Task 1); `createLane`, `updateLane` (Task 2); `broadcastLane`, `sameOriginGuard` (existing).
|
||||
- Produces: `POST /api/lanes/worktree` with body `{sourceRepo, title, base?, slug?}` → `202 {lane}` with `status: "provisioning"`, then a background `lane_update` when the worktree is ready or `status: "failed"` with the git error in `notes`.
|
||||
|
||||
- [ ] **Step 1: Write the failing tests** (append)
|
||||
|
||||
```js
|
||||
it("creates a managed lane, returns 202 provisioning, then flips to idle when the worktree lands", async () => {});
|
||||
it("rejects a sourceRepo that is not an absolute path or not a git repo", async () => {});
|
||||
it("suffixes the slug when the directory already exists", async () => {});
|
||||
it("marks the lane failed with git's message when provisioning fails", async () => {});
|
||||
```
|
||||
|
||||
Poll `GET /api/lanes/:id` until `status !== "provisioning"` with a bounded deadline (2 s, 50 ms interval) — never a bare sleep.
|
||||
|
||||
- [ ] **Step 2: Run to verify they fail**
|
||||
|
||||
Run: `node --test server/__tests__/lane-lifecycle.test.js`
|
||||
Expected: FAIL — `POST /api/lanes/worktree` 404s.
|
||||
|
||||
- [ ] **Step 3: Implement**
|
||||
|
||||
Registered before `/:id/:action`, behind `sameOriginGuard`. Validate: `sourceRepo` absolute, exists, `isGitRepo`. Compute `slug = slugify(req.body.slug || req.body.title)`, `dir = path.join(LANES_ROOT, `${path.basename(sourceRepo)}__${slug}`)`, suffixing `-2`, `-3`… while the directory exists. Create the lane row `kind: "managed", status: "provisioning"`, respond `202`, then in the background — wrapped in `withLaneLock(lane.id, …)` — resolve the base, `addWorktree`, and `updateLane` to `status: "idle"` (or `"failed"` with `notes` set to `err.git?.stderr || err.message`), broadcasting either way.
|
||||
|
||||
Provisioning must never leave a half-state: if `addWorktree` throws, the lane row stays with `kind: "managed"` and `status: "failed"` so the user can `remove` it, and no directory is left behind that git does not know about.
|
||||
|
||||
- [ ] **Step 4: Run to verify they pass** — Expected: PASS, 8 tests total in the file.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add server/routes/lanes.js server/__tests__/lane-lifecycle.test.js
|
||||
git commit -m "feat(lanes): provision a git worktree for a managed lane"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 5: `reset`, `remove`, `purge` actions
|
||||
|
||||
**Files:**
|
||||
- Modify: `server/routes/lanes.js`
|
||||
- Test: `server/__tests__/lane-lifecycle.test.js` (append)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: everything from Tasks 1-4.
|
||||
- Produces: `reset` and `purge` join the `ACTIONS` set; `remove` gains worktree teardown. All three require `{confirm: true}`; `reset` and `remove` additionally require `{force: true}` when preflight reports `unpushed > 0`, and accept `{expect: {head, dirty, untracked, unpushed}}` — a mismatch returns `409 ESTALE`.
|
||||
|
||||
- [ ] **Step 1: Write the failing tests** (append)
|
||||
|
||||
```js
|
||||
it("reset requires confirm, restores the branch from base and clears lane state", async () => {});
|
||||
it("reset refuses with 409 when the worktree has unpushed commits, and proceeds with force", async () => {});
|
||||
it("reset returns 409 ESTALE when the head moved since preflight", async () => {});
|
||||
it("remove tears down the worktree and the branch, and deletes the lane row", async () => {});
|
||||
it("reset and remove refuse an adopted lane with 400 ENOTMANAGED", async () => {});
|
||||
it("purge deletes the lane's sessions and reports the counts", async () => {});
|
||||
```
|
||||
|
||||
The adopted-lane refusal is the single most important test in this plan: it is what stands between a mis-click and a user's real project directory.
|
||||
|
||||
- [ ] **Step 2: Run to verify they fail** — Expected: FAIL, the actions are unknown or non-destructive.
|
||||
|
||||
- [ ] **Step 3: Implement**
|
||||
|
||||
Inside the existing `/:id/:action` handler, all three branches run within `withLaneLock(lane.id, async () => …)`, and each begins by killing the lane's run and awaiting its exit (poll `runs.getRun(lane.run_id)` until it is no longer `running`/`spawning`, bounded, then clear `run_id`).
|
||||
|
||||
Map the guard errors to HTTP: `ENOTMANAGED` / `EOUTSIDEROOT` / `ENOTWORKTREE` → `400` with the code intact; `ESTALE` → `409`; `EUNPUSHED` → `409`; git failures → `500` carrying `err.git.stderr`.
|
||||
|
||||
- [ ] **Step 4: Run to verify they pass** — Expected: PASS, 14 tests in the file.
|
||||
|
||||
- [ ] **Step 5: Full suite and commit**
|
||||
|
||||
Run: `npm run test:server`
|
||||
|
||||
```bash
|
||||
git add server/routes/lanes.js server/__tests__/lane-lifecycle.test.js
|
||||
git commit -m "feat(lanes): reset, remove and purge with preflight and stale-state guards"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 6: CLI
|
||||
|
||||
**Files:**
|
||||
- Modify: `bin/ccam.js`
|
||||
- Test: `server/__tests__/lanes-cli.test.js` (append)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: the routes from Tasks 3-5, via the existing `get` / `post` helpers (`bin/ccam.js:191-192`).
|
||||
- Produces: `ccam lanes add --repo <path> [--title <t>] [--base <branch>]` (worktree mode; the existing `--cwd` form still adopts); `ccam lanes reset|remove|purge <id> [--force]`, each printing the preflight table and refusing without `--yes`.
|
||||
|
||||
- [ ] **Step 1: Write the failing tests** (append) — worktree add via CLI lands a managed lane; `reset` without `--yes` exits non-zero and changes nothing; `--yes` performs it.
|
||||
- [ ] **Step 2: Run to verify they fail.**
|
||||
- [ ] **Step 3: Implement**, reusing the async `cli()` harness and the existing flag reader. Print the preflight counts as a small aligned table before asking for `--yes`, so the terminal path has the same "confirm against numbers" property as the UI.
|
||||
- [ ] **Step 4: Run to verify they pass.**
|
||||
- [ ] **Step 5: Commit** — `feat(lanes): ccam lanes add --repo, reset, remove, purge`
|
||||
|
||||
---
|
||||
|
||||
## Task 7: UI and docs
|
||||
|
||||
**Files:**
|
||||
- Create: `client/src/components/lanes/DestructiveLaneModal.tsx`
|
||||
- Modify: `client/src/components/lanes/LaneCard.tsx`, `client/src/lib/api.ts`, `client/src/lib/types.ts`, `client/src/i18n/locales/*/lanes.json`
|
||||
- Modify: `docs/LANES.md`, `CLAUDE.md`
|
||||
- Test: `client/src/components/lanes/__tests__/DestructiveLaneModal.test.tsx`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `api.lanes.preflight(id, action)` and `api.lanes.action(id, action, body)`.
|
||||
- Produces: `<DestructiveLaneModal lane action onClose onConfirm>` — fetches preflight on open, renders the counts, disables the confirm button while loading or when `blocked` contains anything other than `unpushed-commits`, and exposes a "Force" checkbox only for `unpushed-commits`.
|
||||
|
||||
- [ ] **Step 1: Write the failing test** — the modal renders the counts it was given; the confirm button is disabled for an `adopted` lane; ticking Force enables confirm when the only blocker is unpushed commits; confirming passes back the `expect` block it displayed.
|
||||
- [ ] **Step 2: Run to verify it fails.**
|
||||
- [ ] **Step 3: Implement**, wrapping the repo's existing `ConfirmModal`. `LaneCard` shows a `managed`/`adopted` badge and renders reset/remove/purge only for `managed` lanes. Every string goes through i18n in all four locales.
|
||||
- [ ] **Step 4: Run `npm run test:client` and `npm run build`.** Review the screens snapshot diff before accepting it.
|
||||
- [ ] **Step 5: Docs** — `docs/LANES.md` gains a Lifecycle section covering the two kinds, the three safety checks, each verb with what it destroys and what it spares (`clean -fd` keeps gitignored files), the preflight contract, the env vars, and the fresh-worktree-has-no-dependencies limitation. `CLAUDE.md`'s Lanes section gains the rule: **never `rm -rf` a lane; never build a git command as a shell string; adopted lanes are not destroyable.**
|
||||
- [ ] **Step 6: Header audit and commit** — `feat(lanes): destructive-action modal with preflight counts, lifecycle docs`
|
||||
|
||||
---
|
||||
|
||||
## Out of scope
|
||||
|
||||
- Dependency bootstrap for a fresh worktree (`node_modules`, `.env`) — Shipyard's profile-hook subsystem. A separate sub-project if wanted.
|
||||
- `VACUUM` as part of `purge` — it locks the whole database; if disk reclamation is wanted it becomes its own maintenance action.
|
||||
- Per-lane ports, databases, Docker services.
|
||||
- Stage auto-detection (sub-project B) and the merged Workspace page (sub-project A) — separate specs.
|
||||
@@ -0,0 +1,273 @@
|
||||
# Workspace UI Rebuild Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Rebuild the Workspace page to the reference screen's legibility — card grid, large pipeline, collapsible console — and fill the two data gaps that make lanes look emptier than they are (git facts, expiring detection).
|
||||
|
||||
**Architecture:** Design doc: `docs/superpowers/specs/2026-07-29-workspace-ui-design.md` — read it once before Task 1. Two server tasks land first because the client renders what they produce: detection expiry in `recordDetection`, and a read-only `GET /api/lanes/:id/git` reusing `worktree.js`'s existing `git()` and `statusCounts()`. Then the card is rebuilt, then the page shell around it.
|
||||
|
||||
**Tech Stack:** Node 18+, Express, better-sqlite3, `node:test` (server); React 18 + TypeScript + Vite + Tailwind, Vitest + Testing Library (client).
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- Branch: `feat/workspace-ui`, cut from the head of `feat/workspace-page`. Never work on `master`.
|
||||
- Every `.js/.ts/.tsx` created or modified MUST start with a file overview comment plus the exact line `@author Nguyễn Ngọc Trí Vĩ <vinnt@smartgift.vn>`. Verify with `bash .claude/skills/file-headers/scripts/check-headers.sh` (must exit 0).
|
||||
- **Detection never writes `lanes.stage`**, and **an inferred node never renders `done`.** Both are load-bearing invariants from sub-project B; a change that lets either slip is a failed task regardless of what else it achieves.
|
||||
- **No git command may be built as a shell string.** `execFile` with an argv array only, through the existing `git()` wrapper in `server/lib/worktree.js` — it scrubs the inherited `GIT_*` environment, and that scrub exists because a real bug was traced to it.
|
||||
- The destroy guard (`assertDestroyable`) and the preflight/`expect` echo are not touched by this plan.
|
||||
- `GET /api/lanes` stays free of git subprocesses. Git facts are their own endpoint.
|
||||
- Schema changes are additive with a per-column probe (`try { SELECT col } catch { ALTER }`).
|
||||
- Server CommonJS. No new npm dependencies. Server tests `node:test` + `node:assert/strict`; client tests Vitest + Testing Library. Exact-value assertions, no bare sleeps.
|
||||
- i18n strings in all four locales (`en`, `zh`, `vi`, `ko`), genuinely translated — no English copied into the other three.
|
||||
- **Node 24 is required to run the suites.** Node 25 breaks 20 client tests (global `localStorage`) and 6 server tests (better-sqlite3 ABI). Run with `PATH="$HOME/.nvm/versions/node/v24.14.1/bin:$PATH"`.
|
||||
- The pre-commit hook runs Prettier plus both suites and takes minutes. Let it finish. NEVER `--no-verify`.
|
||||
- Baseline at branch point: 906 server tests, 347 client tests, all passing. Each task leaves `git status --short` empty.
|
||||
|
||||
---
|
||||
|
||||
## Task 1 (D1): detection expires
|
||||
|
||||
**Files:** Modify `server/lib/lanes.js`, `server/__tests__/lanes-lib.test.js`.
|
||||
|
||||
**Produces:** `recordDetection` gains a staleness window. When the lane's
|
||||
`detected_at` is older than `DETECTION_TTL_MS` (read from `process.env`, default
|
||||
`1_800_000`), the forward-only comparison against `detected_stage` is skipped
|
||||
entirely and a fresh detection is accepted even if it sits behind. Inside the
|
||||
window, behaviour is byte-for-byte what it is today.
|
||||
|
||||
The declared-wins rule is NOT affected by the window: a lane whose declared
|
||||
stage leads still refuses the detection, stale or not. Only the
|
||||
detected-vs-detected comparison expires.
|
||||
|
||||
A lane with `detected_stage` set but `detected_at` NULL (rows written before
|
||||
this column was populated) is treated as stale — an unknown age cannot be
|
||||
proven fresh.
|
||||
|
||||
- [ ] **Step 1: write the failing tests** in `lanes-lib.test.js`: a backward detection inside the window still returns `behind-detected` and writes nothing; the same backward detection with `detected_at` set beyond the TTL is written and returns `{written: true}`; a stale detection that is behind the DECLARED stage still returns `behind-declared`; a lane with `detected_stage` set and `detected_at` NULL accepts a backward detection; the TTL reads from `DETECTION_TTL_MS`. Set `detected_at` by writing the column directly in the fixture — do not sleep.
|
||||
- [ ] **Step 2: run, confirm they fail** — `node --test server/__tests__/lanes-lib.test.js`.
|
||||
- [ ] **Step 3: implement.** One added branch in `recordDetection`. Do not touch `withDetected`, `clearLane`, or the payload shape.
|
||||
- [ ] **Step 4: run, confirm they pass;** then the full server suite.
|
||||
- [ ] **Step 5:** header audit, commit — `feat(lanes): expire a stale detection so a lane can move backwards between sessions`.
|
||||
|
||||
---
|
||||
|
||||
## Task 2 (D2): the detected signal says what matched
|
||||
|
||||
**Files:** Modify `server/lib/stage-detect.js`, `server/__tests__/stage-detect.test.js`.
|
||||
|
||||
**Produces:** `detect()` returns a `signal` built from the span the rule's regex
|
||||
actually matched plus surrounding context, instead of the whole flattened input.
|
||||
A rule with no `match` (it fired on the tool name alone) keeps today's behaviour:
|
||||
the flattened input, capped. The existing `capSignal` cap (120 chars, whitespace
|
||||
collapsed, ellipsis) still applies last.
|
||||
|
||||
Concretely: `Bash` with
|
||||
`cd /very/long/path && npm run test:server 2>&1 | tail -5` currently yields the
|
||||
whole string; it must yield a signal containing `npm run test:server` and not the
|
||||
`cd` prefix.
|
||||
|
||||
`detect()` must remain total — it never throws on any input.
|
||||
|
||||
- [ ] **Step 1: write the failing tests:** the Bash example above yields a signal containing `npm run test:server` and not `/very/long/path`; a rule with no `match` still yields the flattened input; a signal longer than the cap is still capped with the ellipsis; a matched span at the very start and at the very end of the input both survive; `detect` still returns null for an unmentioned tool.
|
||||
- [ ] **Step 2: run, confirm they fail.**
|
||||
- [ ] **Step 3: implement.**
|
||||
- [ ] **Step 4: run, confirm they pass;** then the full server suite.
|
||||
- [ ] **Step 5:** header audit, commit — `feat(lanes): report the matched span as the detection signal`.
|
||||
|
||||
---
|
||||
|
||||
## Task 3 (D3): `gitFacts()` in the worktree library
|
||||
|
||||
**Files:** Modify `server/lib/worktree.js`; modify `server/__tests__/worktree.test.js`.
|
||||
|
||||
**Produces:** `gitFacts(dir)` returning `{branch, head, subject, dirty, untracked}`.
|
||||
It reuses the EXISTING `git()` wrapper and `statusCounts(dir)` in the same file —
|
||||
do NOT add a second subprocess helper and do NOT build any command as a shell
|
||||
string. `branch` comes from `rev-parse --abbrev-ref HEAD`, `subject` from
|
||||
`log -1 --format=%s`, and `head`/`dirty`/`untracked` come from `statusCounts`.
|
||||
It throws nothing the caller must catch beyond what `git()` already throws; the
|
||||
route in D4 decides what a failure means.
|
||||
|
||||
No route, no HTTP, no OpenAPI in this task.
|
||||
|
||||
- [ ] **Step 1: write the failing tests** against a real temporary git repo fixture (`git init`, one commit, then one modified tracked file and one untracked file): the branch name, the short head matching `rev-parse --short HEAD`, the exact commit subject, `dirty: 1`, `untracked: 1`. Also: a detached HEAD yields a `branch` of `HEAD` (assert the exact value the command returns, do not invent one); a repo whose only commit has a subject containing spaces returns it whole.
|
||||
- [ ] **Step 2: run, confirm they fail.**
|
||||
- [ ] **Step 3: implement.**
|
||||
- [ ] **Step 4:** full server suite.
|
||||
- [ ] **Step 5:** header audit, commit — `feat(lanes): read branch, head, subject and working-tree counts from a worktree`.
|
||||
|
||||
---
|
||||
|
||||
## Task 4 (D4): the `GET /api/lanes/:id/git` route
|
||||
|
||||
**Files:** Modify `server/routes/lanes.js`, `server/openapi-extra/lanes.js` (+ regenerate `openapi.yaml`); modify `server/__tests__/lanes-api.test.js`.
|
||||
|
||||
**Produces:** `GET /api/lanes/:id/git` → `200 {available: true, ...facts}` for a git
|
||||
worktree; `200 {available: false}` when the lane's `cwd` is missing, is not a git
|
||||
repo, or git fails for any reason. A missing lane is `404`. It is a READ endpoint:
|
||||
no same-origin guard (that guard is for the destructive actions), and it must
|
||||
never mutate a lane.
|
||||
|
||||
Register the route **before** the `/:id/:action` catch-all, the same way
|
||||
`/ensure` had to be — otherwise `git` is swallowed as an action name. State in
|
||||
your report that you checked the ordering and how.
|
||||
|
||||
- [ ] **Step 1: write the failing tests:** a lane pointing at a real temporary git repo returns the branch, short head, subject, `dirty` and `untracked`; a lane whose `cwd` is a plain directory returns `{available: false}` with HTTP 200; a lane whose `cwd` does not exist returns `{available: false}`; an unknown lane id returns 404; the route is NOT shadowed by `/:id/:action` — assert the response body shape, not merely the status.
|
||||
- [ ] **Step 2: run, confirm they fail.**
|
||||
- [ ] **Step 3: implement.**
|
||||
- [ ] **Step 4:** full server suite; `node scripts/generate-openapi-yaml.js` then confirm `git diff openapi.yaml` shows only the added path.
|
||||
- [ ] **Step 5:** header audit, commit — `feat(lanes): expose a lane's git facts over the API`.
|
||||
|
||||
---
|
||||
|
||||
## Task 5 (D5): client API + types for git facts
|
||||
|
||||
**Files:** Modify `client/src/lib/api.ts`, `client/src/lib/types.ts`; modify the matching api test if one exists, else add the assertion to `client/src/lib/__tests__/`.
|
||||
|
||||
**Produces:** `api.lanes.git(id)` calling `GET /api/lanes/:id/git`, and a
|
||||
`LaneGitFacts` type (`{available: true, branch, head, subject, dirty, untracked} | {available: false}`)
|
||||
exported from `client/src/lib/types.ts`. Nothing renders it yet.
|
||||
|
||||
Keep the discriminated union — a caller must be forced to check `available`
|
||||
before reading `branch`. Do not make the fields optional on one flat type.
|
||||
|
||||
- [ ] **Step 1: write the failing test:** `api.lanes.git(3)` requests exactly `/api/lanes/3/git` with method GET, and returns the parsed body.
|
||||
- [ ] **Step 2: run, confirm it fails.**
|
||||
- [ ] **Step 3: implement.**
|
||||
- [ ] **Step 4:** `npm run test:client`, `npm run build`.
|
||||
- [ ] **Step 5:** header audit, commit — `feat(lanes): client binding for a lane's git facts`.
|
||||
|
||||
---
|
||||
|
||||
## Task 6 (D6): rebuild the lane card's own fields
|
||||
|
||||
**Files:** Modify `client/src/components/lanes/LaneCard.tsx`, `client/src/i18n/locales/{en,zh,vi,ko}/lanes.json`; create `client/src/components/lanes/__tests__/LaneCard.test.tsx`.
|
||||
|
||||
**Produces:** the card laid out per the design doc's table, using only fields the
|
||||
lane payload ALREADY carries: header row (`LANE <id>`, liveness dot, status),
|
||||
title, declared-stage chip with progress bar / `%` / time-on-stage, the
|
||||
dashed-amber `auto: <stage>` chip carrying `detected_signal` as its tooltip, the
|
||||
kind and CI tags, the needs-you banner, and the action row.
|
||||
|
||||
**No git block in this task** — that is D7. Do not call `api.lanes.git` here.
|
||||
|
||||
The existing action wiring and `DestructiveLaneModal` usage are preserved
|
||||
exactly: `reset` and `remove` keep their preflight and `expect` echo. Every
|
||||
string goes through i18n in all four locales, genuinely translated.
|
||||
|
||||
The card shows a chip, never a node state — it must not render a detected stage
|
||||
as done.
|
||||
|
||||
- [ ] **Step 1: write the failing tests** in `LaneCard.test.tsx`: every field of a fully-populated fixture lane renders with its exact value; the `auto` chip appears only when the detected stage leads the declared one, and its `title` contains the signal; a lane whose detected stage equals or trails the declared one shows NO auto chip; clicking `reset` opens the destructive modal rather than firing the action directly; the plain action callbacks fire with the right action name.
|
||||
- [ ] **Step 2: run, confirm they fail** — `cd client && npx vitest run src/components/lanes/__tests__/LaneCard.test.tsx`.
|
||||
- [ ] **Step 3: implement.**
|
||||
- [ ] **Step 4:** `npm run test:client`, `npm run build`.
|
||||
- [ ] **Step 5:** header audit, commit — `feat(lanes): rebuild the lane card for legibility`.
|
||||
|
||||
---
|
||||
|
||||
## Task 7 (D7): the card's git block
|
||||
|
||||
**Files:** Modify `client/src/components/lanes/LaneCard.tsx`, `client/src/i18n/locales/{en,zh,vi,ko}/lanes.json`; modify `client/src/components/lanes/__tests__/LaneCard.test.tsx`.
|
||||
|
||||
**Produces:** the card fetches its own facts through `api.lanes.git(lane.id)` on
|
||||
mount and every 30s, and renders a git row — branch, short head, commit subject,
|
||||
and the dirty/untracked counts. It renders the rest of the card unchanged while
|
||||
the facts are still loading and whenever `available` is false. A failed request
|
||||
is silent: no error banner, no retry storm.
|
||||
|
||||
The interval must be cleared on unmount.
|
||||
|
||||
- [ ] **Step 1: write the failing tests:** the git row renders each fact from a mocked `available: true` response; an `available: false` response renders the card with NO git row and no error; a rejected request renders the card with no git row and no error; unmounting clears the interval (assert the timer count, or that no further request is made after unmount).
|
||||
- [ ] **Step 2: run, confirm they fail.**
|
||||
- [ ] **Step 3: implement.**
|
||||
- [ ] **Step 4:** `npm run test:client`, `npm run build`.
|
||||
- [ ] **Step 5:** header audit, commit — `feat(lanes): show a lane's branch and working-tree state on its card`.
|
||||
|
||||
---
|
||||
|
||||
## Task 8 (D8): the page header and the card grid
|
||||
|
||||
**Files:** Modify `client/src/pages/Workspace.tsx`, `client/src/i18n/locales/{en,zh,vi,ko}/lanes.json`; modify `client/src/pages/__tests__/Workspace.test.tsx`.
|
||||
|
||||
**Produces:** the header bar — page title, the four counters (`lanes`, `running`,
|
||||
`needs you`, `dead`) from the API's `counts`, and the Add-lane control — and the
|
||||
responsive card grid (1 column, 2 at `md`, 3 at `xl`) replacing today's
|
||||
horizontal lane strip. Selecting a card still drives the same `selectedLaneId`
|
||||
state it does now.
|
||||
|
||||
Do NOT touch the console or the pipeline panel in this task; leave them exactly
|
||||
where they are, below the grid, however they currently render.
|
||||
|
||||
- [ ] **Step 1: write the failing tests:** the four counters render the exact values from the API's `counts`; every lane in the response gets a card; clicking a card sets it selected (assert an observable consequence, e.g. the pipeline panel's lane, not an internal state variable).
|
||||
- [ ] **Step 2: run, confirm they fail.**
|
||||
- [ ] **Step 3: implement.**
|
||||
- [ ] **Step 4:** `npm run test:client`, `npm run build`. The screens snapshot WILL change — read the diff, confirm it is only the header and grid, then regenerate.
|
||||
- [ ] **Step 5:** header audit, commit — `feat(lanes): lane grid and counters in the Workspace header`.
|
||||
|
||||
---
|
||||
|
||||
## Task 9 (D9): the selected-lane detail panel
|
||||
|
||||
**Files:** Modify `client/src/pages/Workspace.tsx`, `client/src/i18n/locales/{en,zh,vi,ko}/lanes.json`; modify `client/src/pages/__tests__/Workspace.test.tsx`.
|
||||
|
||||
**Produces:** the detail panel between the header and the grid: the selected
|
||||
lane's title, its declared stage and — when detection leads — the inferred one,
|
||||
a large `PipelineMap`, and the legend naming the five node states plus the
|
||||
dashed-amber inferred treatment.
|
||||
|
||||
**Do not change `PipelineMap` itself** — not its node-state logic, not its props.
|
||||
This task places and sizes it.
|
||||
|
||||
- [ ] **Step 1: write the failing tests:** the panel shows the selected lane's title and declared stage; selecting a different card switches the panel's pipeline; **no node rendered in the panel carries both `data-detected="true"` and `data-state="done"`** (the sub-project B premise guard, re-asserted at the new layout); the legend names each of the five states.
|
||||
- [ ] **Step 2: run, confirm they fail.**
|
||||
- [ ] **Step 3: implement.**
|
||||
- [ ] **Step 4:** `npm run test:client`, `npm run build`, snapshot diff read.
|
||||
- [ ] **Step 5:** header audit, commit — `feat(lanes): a full-width pipeline panel for the selected lane`.
|
||||
|
||||
---
|
||||
|
||||
## Task 10 (D10): collapse the console
|
||||
|
||||
**Files:** Modify `client/src/pages/Workspace.tsx`, `client/src/i18n/locales/{en,zh,vi,ko}/run.json`; modify `client/src/pages/__tests__/Workspace.test.tsx`.
|
||||
|
||||
**Produces:** `RunSetup` + `RunConsole` + `RunHistory` wrapped in a disclosure
|
||||
that starts collapsed and expands on click, with the console section moved above
|
||||
the card grid so an expanded console sits beside the lane it belongs to.
|
||||
|
||||
**The subscription must stay mounted while collapsed.** Collapse the visual
|
||||
container with CSS; do NOT conditionally unmount `RunConsole` — unmounting
|
||||
disposes `useRunStream`'s subscription and a live run's envelopes are lost.
|
||||
State in your report which mechanism you used and how you proved the
|
||||
subscription survived.
|
||||
|
||||
No prop of `RunSetup`, `RunConsole` or `RunHistory` changes. The console still
|
||||
never posts a stage.
|
||||
|
||||
- [ ] **Step 1: write the failing tests:** the console is collapsed on first render and expands on click; **an envelope delivered through the mocked event bus while the console is collapsed is present in the DOM once it is expanded**; after a full start-and-message cycle no request is made to any `/stage` URL.
|
||||
- [ ] **Step 2: run, confirm they fail.**
|
||||
- [ ] **Step 3: implement.**
|
||||
- [ ] **Step 4:** `npm run test:client`, `npm run build`, snapshot diff read then regenerated.
|
||||
- [ ] **Step 5:** header audit, commit — `feat(lanes): collapse the console without dropping its stream`.
|
||||
|
||||
---
|
||||
|
||||
## Task 11 (D11): docs
|
||||
|
||||
**Files:** Modify `docs/LANES.md`, `docs/API.md`, `README.md`, `ARCHITECTURE.md`, `CLAUDE.md`.
|
||||
|
||||
**Produces:** the new layout described where the old one was; `GET /api/lanes/:id/git` documented with its `available: false` contract and the reason it is not folded into `GET /api/lanes`; the detection TTL documented with `DETECTION_TTL_MS`, its default, and the explicit note that expiry does NOT weaken declared-wins or let inference render `done`. Every path, route and command printed must exist — verify each.
|
||||
|
||||
- [ ] **Step 1:** write the docs.
|
||||
- [ ] **Step 2:** verify every referenced file, route and command exists (`ls`, `grep`, or run it).
|
||||
- [ ] **Step 3:** full server suite, full client suite, `node scripts/generate-openapi-yaml.js` then `git diff openapi.yaml` empty.
|
||||
- [ ] **Step 4:** header audit, commit — `docs(lanes): document the rebuilt Workspace, git facts and detection expiry`.
|
||||
|
||||
---
|
||||
|
||||
## Out of scope
|
||||
|
||||
- Tickets, preview-port links, per-lane credentials, and the `agents`/`creds` buttons from the reference screen — CCAM has no data behind any of them.
|
||||
- Any change to `PipelineMap`'s node-state logic, the destroy guard, or the preflight contract.
|
||||
- Inferring `done` or a gate outcome. Still forbidden, TTL or not.
|
||||
- Re-styling any page other than `/run`.
|
||||
@@ -0,0 +1,419 @@
|
||||
# Agent Conversation Viewer Design
|
||||
|
||||
## Overview
|
||||
|
||||
Add a conversation viewer to the SessionDetail page, enabling visual inspection of Main Agent and sub-agent interactions (message content and tool call details), with data sourced from real-time JSONL transcript files.
|
||||
|
||||
## Problem
|
||||
|
||||
The current dashboard tracks agent sessions, events, and tool usage at a summary level, but does not expose the actual conversation content — user messages, assistant replies, tool call parameters, and tool results. Users cannot see what each agent actually did or said, limiting debugging and audit capabilities.
|
||||
|
||||
### v2 Additional Problems: Poor Pagination UX + No Real-time Updates
|
||||
|
||||
After v1 implementation, two core UX issues emerged:
|
||||
|
||||
1. **Pagination doesn't match conversation intuition** — v1 uses offset-based pagination starting from the beginning, so users see the oldest messages first and must page through to reach recent interactions, which doesn't align with chat product conventions.
|
||||
2. **No real-time updates** — v1 doesn't subscribe to WebSocket events, so users must manually refresh to see new messages, making it impossible to follow active sessions in real time.
|
||||
3. **Sub-agent selection uses database IDs** — v1's `agent_id` parameter relies on database agent IDs, but JSONL files are named with short IDs (e.g. `ad18a79192af10ed1`), causing a mismatch that prevents sub-agent transcripts from loading.
|
||||
|
||||
## Design Decisions
|
||||
|
||||
| Decision | Choice | Rationale |
|
||||
|----------|--------|-----------|
|
||||
| Data source | Real-time JSONL reads | Data is always current, no extra storage needed |
|
||||
| UI location | Conversation tab within SessionDetail | User-requested; keeps agent tree in the same context |
|
||||
| Claude home path | Configurable via `CLAUDE_HOME` env var | Supports non-default paths like `~/.codefuse/engine/cc/` |
|
||||
| Message rendering | Collapsible tool calls and thinking blocks | Keeps the view scannable; expand for details |
|
||||
| Load strategy (v2) | Chat-flow: load latest N by default, scroll up for history | Matches chat product intuition; users care most about recent interactions |
|
||||
| Real-time updates (v2) | WebSocket `new_event` triggers incremental load | Active sessions don't need manual refresh |
|
||||
| Agent selection (v2) | Filesystem scan + dropdown | Bypasses database ID mismatch by using file short IDs directly |
|
||||
|
||||
## Architecture
|
||||
|
||||
### Data Flow
|
||||
|
||||
**v1 (deprecated):**
|
||||
```
|
||||
User clicks "Conversation" tab
|
||||
→ Frontend calls GET /api/sessions/:id/transcript[?agent_id=xxx&limit=50&offset=0]
|
||||
→ Server resolves JSONL path via claude-home.js
|
||||
→ Server reads and parses JSONL file
|
||||
→ Server returns structured message list
|
||||
→ Frontend renders MessageList (with collapsible blocks)
|
||||
```
|
||||
|
||||
**v2 Chat-flow (current implementation):**
|
||||
```
|
||||
Initial load:
|
||||
User opens Conversation tab
|
||||
→ GET /api/sessions/:id/transcripts ← fetch available transcript list
|
||||
→ GET /api/sessions/:id/transcript?limit=50 ← default returns latest 50 messages
|
||||
→ Frontend renders message list + auto-scrolls to bottom
|
||||
|
||||
Real-time updates:
|
||||
CLI Hook → POST /api/hooks/event → processEvent()
|
||||
→ broadcast("new_event", {session_id, ...})
|
||||
→ WebSocket → ConversationView
|
||||
→ GET /api/sessions/:id/transcript?after=N ← incremental load
|
||||
→ Append to bottom + auto-scroll (if user is at bottom)
|
||||
|
||||
History load:
|
||||
User scrolls to top
|
||||
→ GET /api/sessions/:id/transcript?before=M&limit=50 ← load older messages
|
||||
→ Prepend to top + preserve scroll position (no jump)
|
||||
```
|
||||
|
||||
### Configurable Claude Home Directory
|
||||
|
||||
New module `server/lib/claude-home.js` centralizes all Claude directory path logic:
|
||||
|
||||
```
|
||||
CLAUDE_HOME env var (default: ~/.claude)
|
||||
├── projects/<encoded-cwd>/<session-id>.jsonl ← main session transcript
|
||||
│ (encoding rule: all non-alphanumeric chars → "-", e.g. "/Users/txj/.codefuse" → "-Users-txj--codefuse")
|
||||
├── projects/<encoded-cwd>/<session-id>/subagents/agent-<id>.jsonl ← sub-agent transcript
|
||||
│ (sub-agent ID format: ad18a79192af10ed1, acompact-f8427be966459435)
|
||||
└── settings.json ← hooks configuration
|
||||
```
|
||||
|
||||
Existing hardcoded paths in `import-history.js`, `install-hooks.js`, and `settings.js` are migrated to use this module.
|
||||
|
||||
---
|
||||
|
||||
## API
|
||||
|
||||
### GET /api/sessions/:id/transcripts (v2 new)
|
||||
|
||||
List available transcript files for a session (main + sub-agents), scanned directly from the filesystem.
|
||||
|
||||
**Response (200):**
|
||||
|
||||
```json
|
||||
{
|
||||
"transcripts": [
|
||||
{ "id": "main", "name": "Main Agent", "type": "main", "has_transcript": true },
|
||||
{ "id": "ad18a79192af10ed1", "name": "code-reviewer", "type": "subagent", "subagent_type": "code-reviewer", "has_transcript": true },
|
||||
{ "id": "acompact-f8427be966459435", "name": "Context Compaction", "type": "compaction", "has_transcript": true }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Design notes:**
|
||||
|
||||
- Bypasses database agent IDs; scans the filesystem directly for JSONL file short IDs
|
||||
- `id` field maps directly to the filename: `agent-<id>.jsonl`, used as the `agent_id` parameter for the `transcript` API
|
||||
- Compaction file name format: `agent-acompact-<hex>.jsonl`, id is `acompact-<hex>`
|
||||
- Attempts to read `.meta.json` in the same directory for agent type description
|
||||
- Falls back to scanning all `projects/` subdirectories when the exact encoded path doesn't exist
|
||||
|
||||
### GET /api/sessions/:id/transcript
|
||||
|
||||
Read a session's JSONL transcript file and return a structured message list.
|
||||
|
||||
**Query Parameters:**
|
||||
|
||||
| Parameter | Type | Default | Description |
|
||||
|-----------|------|---------|-------------|
|
||||
| `agent_id` | string | null | Transcript short ID (from `transcripts` endpoint); omit for main session |
|
||||
| `limit` | number | 50 | Max messages to return (max 200) |
|
||||
| `after` | number | null | Incremental mode: only return messages with JSONL line > after (v2 new) |
|
||||
| `before` | number | null | History mode: only return the latest N messages with JSONL line < before (v2 new) |
|
||||
| `offset` | number | 0 | Legacy pagination offset (compatible, mutually exclusive with after/before) |
|
||||
|
||||
**Response (200):**
|
||||
|
||||
```json
|
||||
{
|
||||
"messages": [
|
||||
{
|
||||
"type": "user",
|
||||
"timestamp": "2026-04-24T10:23:45Z",
|
||||
"content": [
|
||||
{ "type": "text", "text": "Please implement the login feature" }
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "assistant",
|
||||
"timestamp": "2026-04-24T10:23:52Z",
|
||||
"model": "claude-sonnet-4-6",
|
||||
"usage": { "input_tokens": 1500, "output_tokens": 800 },
|
||||
"content": [
|
||||
{ "type": "text", "text": "I'll help you implement the login feature." },
|
||||
{ "type": "thinking", "text": "Let me analyze the codebase..." },
|
||||
{
|
||||
"type": "tool_use",
|
||||
"name": "Read",
|
||||
"id": "toolu_abc123",
|
||||
"input": { "file_path": "/src/auth.ts" }
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"total": 120,
|
||||
"has_more": true,
|
||||
"last_line": 523,
|
||||
"first_line": 474
|
||||
}
|
||||
```
|
||||
|
||||
**v2 New Response Fields:**
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `last_line` | number | JSONL line number of the last message in the current response; used as the `after` parameter for incremental requests |
|
||||
| `first_line` | number | JSONL line number of the first message in the current response; used as the `before` parameter for history loading |
|
||||
|
||||
**Loading Modes:**
|
||||
|
||||
| Mode | Parameters | Behavior | Use Case |
|
||||
|------|-----------|----------|----------|
|
||||
| Default | No after/before/offset | Return the latest N messages | Initial load |
|
||||
| Incremental | `after=N` | Return messages with line > N (up to limit) | WebSocket-triggered new message loading |
|
||||
| History | `before=M` | Return the latest N messages with line < M | Scroll-up to load older messages |
|
||||
| Compatible | `offset=K` | Skip first K, return next N | Legacy pagination (kept for compatibility) |
|
||||
|
||||
**Error Responses:**
|
||||
|
||||
| Status | Condition |
|
||||
|--------|-----------|
|
||||
| 200 | When JSONL file doesn't exist, returns empty `{ messages: [], total: 0, has_more: false, last_line: 0, first_line: 0 }` |
|
||||
| 404 | Session ID not found in database |
|
||||
|
||||
**Implementation Rules:**
|
||||
|
||||
- Only extract entries with `type: "user"` or `type: "assistant"`; skip system/progress entries
|
||||
- Match `tool_use` and `tool_result` via `id` field; unpaired tool_use shows no result section
|
||||
- Truncate individual content exceeding 10KB, appending `[truncated]`
|
||||
- Re-read the file on every request (no server-side caching) to ensure real-time freshness
|
||||
- When `cwd` is null, scan all `projects/` subdirectories to find the JSONL for the sessionId
|
||||
- Internally use JSONL line numbers as cursors; remove the `line` field from responses, expose `first_line` / `last_line` to the client
|
||||
|
||||
---
|
||||
|
||||
## Frontend
|
||||
|
||||
### SessionDetail Page Changes
|
||||
|
||||
Replace the current flat layout with a **tabbed interface**:
|
||||
|
||||
```
|
||||
[Agents] [Conversation] [Timeline]
|
||||
```
|
||||
|
||||
- **Agents tab** — existing agent hierarchy tree (active by default)
|
||||
- **Conversation tab** — new conversation viewer
|
||||
- **Timeline tab** — existing event timeline
|
||||
|
||||
### Conversation Tab Components
|
||||
|
||||
**v2 Chat-flow architecture:**
|
||||
|
||||
```
|
||||
ConversationView.tsx
|
||||
├── TranscriptSelector — dropdown selector (v2 replaces AgentFilter)
|
||||
├── ScrollContainer — scrollable message container
|
||||
│ ├── HistoryLoader — scroll-up history loading indicator
|
||||
│ └── MessageList.tsx
|
||||
│ ├── UserMessage — user message
|
||||
│ └── AssistantMessage
|
||||
│ ├── TextBlock — plain text content
|
||||
│ ├── ThinkingBlock — collapsible thinking content
|
||||
│ └── ToolCallBlock — collapsible tool call + result
|
||||
│ ├── ToolUse — tool name + parameters
|
||||
│ └── ToolResult — execution result / error
|
||||
└── NewMsgButton — "New messages" floating button (v2 new)
|
||||
```
|
||||
|
||||
### TranscriptSelector (v2 replaces AgentFilter)
|
||||
|
||||
- Top dropdown selector: `[Main Agent ▾]` or `[Context Compaction ▾]`
|
||||
- Data source: `GET /api/sessions/:id/transcripts` (filesystem scan, not database)
|
||||
- Reloads the corresponding transcript on switch
|
||||
- Only shown when transcripts > 1
|
||||
- Message count displayed alongside: `518 messages`
|
||||
|
||||
### Chat-flow Behavior (v2 new)
|
||||
|
||||
**Initial load:**
|
||||
- Call `transcript?limit=50` to get the latest 50 messages
|
||||
- Auto-scroll to bottom after rendering
|
||||
- Track `last_line` and `first_line` for subsequent requests
|
||||
|
||||
**Real-time updates (WebSocket-driven):**
|
||||
- Subscribe to `eventBus` `new_event` events
|
||||
- Only process events where `session_id` matches the current session
|
||||
- On event, call `transcript?after=last_line&limit=50` for incremental loading
|
||||
- If user is at bottom (< 100px from bottom), auto-scroll to latest message
|
||||
- If user has scrolled up, show "New messages" floating button; click to scroll to bottom
|
||||
|
||||
**Scroll-up history loading:**
|
||||
- Listen for scroll events; trigger when `scrollTop < 50` and `has_more` is true
|
||||
- Call `transcript?before=first_line&limit=50` to fetch older messages
|
||||
- Prepend to top of list; preserve scroll position via `scrollHeight` delta
|
||||
- Show spinner while loading; show "↑ Scroll up for older messages" hint at top
|
||||
|
||||
**Key Refs:**
|
||||
- `lastLineRef` — tracks the JSONL line number of the newest message, used for incremental requests
|
||||
- `firstLineRef` — tracks the JSONL line number of the oldest loaded message, used for history loading
|
||||
- `scrollContainerRef` — scroll container DOM reference
|
||||
- `isAtBottomRef` — boolean flag tracking whether user is at the bottom
|
||||
|
||||
### Message Rendering
|
||||
|
||||
- **User messages**: right-aligned, blue background, display text content
|
||||
- **Assistant messages**: left-aligned, default background, including:
|
||||
- Model name and token usage as faded metadata
|
||||
- Text blocks rendered inline
|
||||
- Thinking blocks: collapsed by default, click to expand (dimmed style)
|
||||
- Tool calls: collapsed by default showing only tool name, click to expand:
|
||||
- Tool name as header with icon
|
||||
- Input parameters formatted as JSON (collapsible)
|
||||
- Tool result with success/error indicator
|
||||
|
||||
### Interaction Details
|
||||
|
||||
- **Long text truncation**: content over 500 characters is truncated by default, with an "expand" link
|
||||
- **Lazy loading (v2)**: initial load of latest 50 messages; scroll-up auto-loads older 50; WebSocket-driven incremental append
|
||||
- **Real-time updates (v2)**: on WebSocket `new_event` with matching `session_id`, incrementally load new messages
|
||||
- **Auto-scroll (v2)**: auto-scroll to latest when user is at bottom; show floating "New messages" button when user has scrolled up
|
||||
- **Empty state**: when JSONL is missing or empty, show "No conversation records found."
|
||||
|
||||
---
|
||||
|
||||
## Server Module: claude-home.js
|
||||
|
||||
```js
|
||||
// Centralized Claude home directory path management
|
||||
function getClaudeHome() {
|
||||
return process.env.CLAUDE_HOME || path.join(os.homedir(), ".claude");
|
||||
}
|
||||
|
||||
function getProjectsDir() {
|
||||
return path.join(getClaudeHome(), "projects");
|
||||
}
|
||||
|
||||
function getSettingsPath() {
|
||||
return path.join(getClaudeHome(), "settings.json");
|
||||
}
|
||||
|
||||
// Encoding rule: all non-alphanumeric characters replaced with "-"
|
||||
// Example: "/Users/txj/.codefuse" → "-Users-txj--codefuse"
|
||||
function encodeCwd(cwd) {
|
||||
return cwd.replace(/[^a-zA-Z0-9]/g, "-");
|
||||
}
|
||||
|
||||
function getTranscriptPath(sessionId, cwd) {
|
||||
if (!cwd) return null;
|
||||
const encoded = encodeCwd(cwd);
|
||||
const candidate = path.join(getProjectsDir(), encoded, `${sessionId}.jsonl`);
|
||||
if (fs.existsSync(candidate)) return candidate;
|
||||
// Fallback: scan projects/ subdirectories
|
||||
return findTranscriptPath(sessionId);
|
||||
}
|
||||
|
||||
function getSubagentTranscriptPath(sessionId, cwd, agentId) {
|
||||
if (!cwd) return null;
|
||||
const encoded = encodeCwd(cwd);
|
||||
const candidate = path.join(getProjectsDir(), encoded, sessionId, "subagents", `agent-${agentId}.jsonl`);
|
||||
if (fs.existsSync(candidate)) return candidate;
|
||||
// Fallback: scan all project directories
|
||||
return findSubagentTranscriptPath(sessionId, agentId);
|
||||
}
|
||||
|
||||
function findTranscriptPath(sessionId) {
|
||||
// Fallback: when cwd is unknown, scan projects/ subdirectories
|
||||
const projectsDir = getProjectsDir();
|
||||
if (!fs.existsSync(projectsDir)) return null;
|
||||
const dirs = fs.readdirSync(projectsDir, { withFileTypes: true });
|
||||
for (const d of dirs) {
|
||||
if (!d.isDirectory()) continue;
|
||||
const candidate = path.join(projectsDir, d.name, `${sessionId}.jsonl`);
|
||||
if (fs.existsSync(candidate)) return candidate;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
// v2 new: support prefix fuzzy matching for compaction type
|
||||
function findSubagentTranscriptPath(sessionId, agentId) {
|
||||
const projectsDir = getProjectsDir();
|
||||
if (!fs.existsSync(projectsDir)) return null;
|
||||
const dirs = fs.readdirSync(projectsDir, { withFileTypes: true });
|
||||
for (const d of dirs) {
|
||||
if (!d.isDirectory()) continue;
|
||||
const subagentsDir = path.join(projectsDir, d.name, sessionId, "subagents");
|
||||
if (!fs.existsSync(subagentsDir)) continue;
|
||||
// Exact match
|
||||
const exact = path.join(subagentsDir, `agent-${agentId}.jsonl`);
|
||||
if (fs.existsSync(exact)) return exact;
|
||||
// Prefix fuzzy match (compaction type: agentId starts with "acompact-")
|
||||
if (agentId.startsWith("acompact-")) {
|
||||
const files = fs.readdirSync(subagentsDir);
|
||||
const match = files.find(f => f.startsWith("agent-acompact-") && f.endsWith(".jsonl"));
|
||||
if (match) return path.join(subagentsDir, match);
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## File Changes
|
||||
|
||||
| File | Action | Description |
|
||||
|------|--------|-------------|
|
||||
| `server/lib/claude-home.js` | **New** | Claude home directory path management; v2 adds `findSubagentTranscriptPath` prefix fuzzy matching |
|
||||
| `server/routes/sessions.js` | Modified | v1: add `GET /sessions/:id/transcript`; v2: add `GET /sessions/:id/transcripts`, transcript endpoint gains `after`/`before` params and `first_line`/`last_line` response |
|
||||
| `scripts/import-history.js` | Modified | Use `getClaudeHome()` instead of hardcoded path |
|
||||
| `scripts/install-hooks.js` | Modified | Use `getSettingsPath()` instead of hardcoded path |
|
||||
| `server/routes/settings.js` | Modified | Use `getClaudeHome()` for hooks detection |
|
||||
| `client/src/lib/types.ts` | Modified | v1: add `TranscriptMessage`, `TranscriptContent`; v2: add `TranscriptInfo`, `TranscriptListResult`, `TranscriptResult` gains `last_line`/`first_line` |
|
||||
| `client/src/lib/api.ts` | Modified | v1: add `sessions.transcript()`; v2: add `sessions.transcripts()`, `transcript()` gains `after`/`before` params |
|
||||
| `client/src/pages/SessionDetail.tsx` | Modified | Add tab switching and Conversation tab; v2: remove `agents` prop from ConversationView |
|
||||
| `client/src/components/conversation/ConversationView.tsx` | **New** → v2 rewrite | v1: basic pagination; v2: chat-flow mode (WebSocket incremental + scroll-up history + auto-scroll) |
|
||||
| `client/src/components/conversation/MessageList.tsx` | **New** | Message list (with collapsible blocks, command formatting, skill content folding, task notification folding) |
|
||||
| `client/src/components/conversation/ToolCallBlock.tsx` | **New** | Collapsible tool call display |
|
||||
|
||||
---
|
||||
|
||||
## Error Handling
|
||||
|
||||
| Scenario | Handling |
|
||||
|----------|----------|
|
||||
| JSONL file doesn't exist | Return `{ messages: [], total: 0, has_more: false, last_line: 0, first_line: 0 }`; UI shows "No conversation records found." |
|
||||
| JSONL line parse failure | Skip the line, continue processing remaining lines |
|
||||
| Single content exceeds 10KB | Truncate and append `[truncated]` marker |
|
||||
| Sub-agent JSONL doesn't exist | Same as main file — return empty list |
|
||||
| Session cwd is null | Use `findTranscriptPath()` to scan project directories |
|
||||
| CLAUDE_HOME path invalid | Log warning, return empty list |
|
||||
| Incremental load returns no new messages (v2) | `after` request returns empty array, frontend silently ignores |
|
||||
| History load failure (v2) | Silent failure, doesn't interrupt user experience |
|
||||
| WebSocket disconnection (v2) | Doesn't affect loaded messages; next event after reconnect triggers incremental load |
|
||||
|
||||
## Edge Cases
|
||||
|
||||
- **Compaction**: After `/compact`, older messages are lost from the JSONL. The viewer only shows what's currently in the file — this is expected behavior. Compact transcripts appear as separate entries in the transcript selector.
|
||||
- **Active sessions**: JSONL may be actively written to. Every request re-reads the file for real-time freshness. WebSocket events trigger incremental loading — no polling needed.
|
||||
- **Unpaired tool_use/tool_result**: Display the tool call without the result section; no error.
|
||||
- **Message order**: JSONL is ordered chronologically; responses preserve the same order (oldest first).
|
||||
- **Database ID vs file ID mismatch (v2)**: Database agent IDs use format `<sessionId>-jsonl-<shortId>`, but JSONL filenames use `agent-<shortId>.jsonl`. v2 bypasses database IDs entirely via the `transcripts` endpoint, which scans the filesystem and uses file short IDs.
|
||||
- **Compaction filename format (v2)**: In the database, compaction agent IDs use format `<sessionId>-compact-<uuid>`, but filenames use `agent-acompact-<hex>.jsonl`. `findSubagentTranscriptPath` supports prefix fuzzy matching for `agent-acompact-*.jsonl`.
|
||||
- **Scroll position preservation (v2)**: When loading history, the scroll position is preserved by computing the `scrollHeight` delta, ensuring the viewport content doesn't jump.
|
||||
- **Duplicate events (v2)**: WebSocket may send multiple `new_event` messages; incremental loading uses `after` line number for deduplication, preventing duplicate appends.
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
| Layer | Test Content |
|
||||
|-------|-------------|
|
||||
| API unit tests | `GET /sessions/:id/transcript` — normal response, file not found, invalid session, pagination params, agent_id filtering |
|
||||
| API unit tests | `GET /sessions/:id/transcript` — v2: `after` incremental loading, `before` history loading, `first_line`/`last_line` response |
|
||||
| API unit tests | `GET /sessions/:id/transcripts` — v2: file scanning, compaction type, meta.json reading |
|
||||
| API unit tests | `claude-home.js` — path inference logic, env var override, fallback scanning, compaction prefix fuzzy matching |
|
||||
| Frontend component tests | `MessageList` rendering, `ToolCallBlock` collapse/expand, command formatting, skill content folding |
|
||||
| Frontend component tests | `ConversationView` — v2: initial load, incremental append, history load, scroll detection, new messages indicator |
|
||||
|
||||
## Environment Variables
|
||||
|
||||
| Variable | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `CLAUDE_HOME` | `~/.claude` | Claude Code home directory (e.g. `~/.codefuse/engine/cc/`) |
|
||||
@@ -0,0 +1,220 @@
|
||||
# Design: Fix Agent-Monitor server memory leak
|
||||
|
||||
- **Date**: 2026-05-22
|
||||
- **Author**: zhihua + Claude (brainstorming collaboration)
|
||||
- **Status**: Design Approved, pending implementation plan
|
||||
|
||||
## Background
|
||||
|
||||
After running `npm start` locally, the server process memory grows continuously over time and eventually exhausts host memory when combined with Claude / IDE / browser. The initial proposal was to deploy Agent-Monitor on a remote server and access it via the local browser, but investigation showed this only relocates the problem — the root cause is in the server itself, and a long-running remote instance will also OOM.
|
||||
|
||||
This design focuses on **root-cause remediation**, not remote deployment. Once memory is stable post-fix, we can revisit whether remote deployment is still desirable.
|
||||
|
||||
## Current diagnosis (with code evidence)
|
||||
|
||||
Measured locally:
|
||||
|
||||
| Metric | Value |
|
||||
|---|---|
|
||||
| `data/dashboard.db` | 192 MB |
|
||||
| `events` row count | 251,244 |
|
||||
| `sessions` count | 1130 (completed 1015 + abandoned 110 + active 5) |
|
||||
| Largest single event size | 369 KB |
|
||||
| `~/.claude/projects` | 58 MB |
|
||||
|
||||
Three leak / performance sources were identified:
|
||||
|
||||
### Leak #1: TranscriptCache entry has no per-entry size cap
|
||||
|
||||
In `server/lib/transcript-cache.js`, every cache entry holds three push-only arrays:
|
||||
|
||||
- `state.turnDurations.push(...)` (l.327)
|
||||
- `state.errors.push(...)` (l.332, 343)
|
||||
- `state.compaction.entries.push(...)` (l.315)
|
||||
|
||||
`_merge()` incremental merging (l.456, 462, 468) likewise only pushes and never trims.
|
||||
|
||||
`MAX_CACHE_ENTRIES = 200` bounds the number of entries, but **each entry is unbounded in size**. A long session emits one turnDuration per turn (~50 bytes), so a few thousand turns = MB-scale per entry; 200 entries × tens of MB = **multiple GB**.
|
||||
|
||||
### Leak #2: `_set()` stores everything twice (per-entry memory doubled)
|
||||
|
||||
`server/lib/transcript-cache.js:51-58`:
|
||||
|
||||
```js
|
||||
this._set(key, {
|
||||
errors: result?.errors ? [...result.errors] : null, // top-level shallow copy
|
||||
turnDurations: result?.turnDurations ? [...result.turnDurations] : null,
|
||||
compaction: this._cloneCompaction(result.compaction),
|
||||
...
|
||||
result, // contains references to the same fields
|
||||
});
|
||||
```
|
||||
|
||||
The top-level fields are shallow-copied (`[...result.errors]`) new array objects that do not share references with `result.errors`. **Each array exists twice on the heap per cache entry.**
|
||||
|
||||
### Performance issue: the sweep does a full scan over events
|
||||
|
||||
`server/index.js:329` runs every 60-300s:
|
||||
|
||||
```sql
|
||||
SELECT DISTINCT e.session_id, json_extract(e.data,'$.transcript_path') AS tp
|
||||
FROM events e JOIN sessions s ON s.id=e.session_id
|
||||
WHERE s.status='active' AND json_extract(e.data,'$.transcript_path') IS NOT NULL
|
||||
GROUP BY e.session_id ORDER BY MAX(e.id) DESC
|
||||
```
|
||||
|
||||
Doing `json_extract` + DISTINCT + ORDER BY across 250k events rows produces large temporary SQLite memory spikes and is slow.
|
||||
|
||||
## Goals and constraints
|
||||
|
||||
**Goals**:
|
||||
|
||||
1. Server process RSS stays stable over long runs (< 300 MB)
|
||||
2. No Agent log loss (events table remains complete; no retention)
|
||||
3. Reversible changes confined to `server/`; no changes to hook-handler / UI / WebSocket protocol
|
||||
|
||||
**Non-goals** (explicitly out of scope):
|
||||
|
||||
- Remote deployment
|
||||
- Events table retention / archival
|
||||
- DB engine swap / compression / sharding
|
||||
- UI / frontend / CLI changes
|
||||
|
||||
## Design
|
||||
|
||||
### Change A: TranscriptCache per-entry sliding window
|
||||
|
||||
Add a configurable cap:
|
||||
|
||||
```js
|
||||
const MAX_ARRAY_LEN = parseInt(process.env.TRANSCRIPT_CACHE_MAX_ARRAY_LEN, 10) || 1000;
|
||||
```
|
||||
|
||||
After each push in `_streamRange` parsing (l.315/327/332/343 etc.) and in `_merge` incremental merging (l.456/462/468), trim immediately:
|
||||
|
||||
```js
|
||||
if (arr.length > MAX_ARRAY_LEN) arr.splice(0, arr.length - MAX_ARRAY_LEN);
|
||||
```
|
||||
|
||||
Applies to `turnDurations`, `errors`, `compaction.entries`, and `usageExtras.{service_tiers, speeds, inference_geos}` (these Set→Array conversions can also accumulate).
|
||||
|
||||
**Why no data loss**:
|
||||
|
||||
`routes/hooks.js:583, 633` already inserts `result.errors` / `result.turnDurations` into the events table on every hook trigger, with dedup (`SELECT 1 ... WHERE summary=?` / `WHERE created_at=?`). After cache truncation, the next hook re-reads the transcript file → dedup skips existing rows → only new rows are inserted. The events table stays complete.
|
||||
|
||||
**Capacity estimate**:
|
||||
- 1 turn ≈ 50 bytes
|
||||
- 1000 turns = 50 KB / cache entry
|
||||
- 200 entries full ≈ 10 MB
|
||||
|
||||
### Change B: Eliminate `_set()` double storage
|
||||
|
||||
Simplify the cache entry shape:
|
||||
|
||||
```js
|
||||
this._cache.set(key, { mtimeMs, size, bytesRead, result });
|
||||
```
|
||||
|
||||
Drop all top-level `errors` / `turnDurations` / `compaction` / `usageExtras` / `tokensByModel` / `thinkingBlockCount` / `latestModel` fields. `_merge` computes via local variables and writes back only into `result`.
|
||||
|
||||
**Expected effect**: ~50% memory reduction per entry.
|
||||
|
||||
### Change C: Stop sweeping events for transcript_path
|
||||
|
||||
**Schema migration** (`server/db.js`):
|
||||
|
||||
```sql
|
||||
-- Add column (idempotent)
|
||||
ALTER TABLE sessions ADD COLUMN transcript_path TEXT;
|
||||
|
||||
-- One-time backfill (runs once at startup, gated by a .migrations marker file to prevent reruns)
|
||||
UPDATE sessions SET transcript_path = (
|
||||
SELECT json_extract(data,'$.transcript_path') FROM events
|
||||
WHERE events.session_id=sessions.id
|
||||
AND json_extract(data,'$.transcript_path') IS NOT NULL
|
||||
LIMIT 1
|
||||
) WHERE transcript_path IS NULL;
|
||||
```
|
||||
|
||||
Follow the idempotent migration pattern at `server/db.js:284` (the `agents_new` rebuild).
|
||||
|
||||
**Write path** (`server/routes/hooks.js` `ensureSession`):
|
||||
|
||||
When `transcript_path` is first seen, run `UPDATE sessions SET transcript_path=? WHERE id=? AND transcript_path IS NULL`.
|
||||
|
||||
**Sweep query rewrite** (`server/index.js:329`):
|
||||
|
||||
```sql
|
||||
SELECT id, transcript_path FROM sessions
|
||||
WHERE status='active' AND transcript_path IS NOT NULL
|
||||
```
|
||||
|
||||
The query at `server/index.js:309` that fetches `transcript_path` on abandonment is also rewritten to read from the sessions table.
|
||||
|
||||
**Complexity**: drops from O(total events rows) to O(active sessions ≈ single digits). **Not a single events row is removed.**
|
||||
|
||||
## Verification strategy
|
||||
|
||||
### Unit tests (new `server/__tests__/transcript-cache-bounded.test.js`)
|
||||
|
||||
1. With `MAX_ARRAY_LEN=100`, feed 500 turns → `result.turnDurations.length === 100`, tail retained
|
||||
2. After cache truncation, re-extracting → events table dedup skips existing rows, insert count == 0
|
||||
3. Coarse memory assertion: 200 entries × 1000 turns, `process.memoryUsage().heapUsed` delta < 30 MB
|
||||
|
||||
### Integration tests
|
||||
|
||||
- `npm run test:server` green
|
||||
- `npm run test:client` green
|
||||
- `npm run mcp:typecheck` passes
|
||||
|
||||
### Measurement script (one-off)
|
||||
|
||||
New `scripts/memory-soak-test.js`:
|
||||
|
||||
- Generate fake transcript jsonl with 10000 turns
|
||||
- Start the server, simulate 10 concurrent active sessions, fire a hook every 1s
|
||||
- Run for 30 minutes, log `process.memoryUsage().rss` per minute
|
||||
- Assert: RSS growth at minute 30 < 50 MB
|
||||
|
||||
### Verification checklist (pre-merge)
|
||||
|
||||
- [ ] Unit + integration tests green
|
||||
- [ ] `npm run mcp:typecheck` passes
|
||||
- [ ] Local `npm start` for 1h, `ps -o rss=` monitoring shows a flat curve
|
||||
- [ ] DB migration idempotent: two consecutive `npm start` runs without errors
|
||||
- [ ] Old DB (no `transcript_path` column) → migrate + backfill → sweep works
|
||||
- [ ] After cache truncation, the UI events list still shows all old turns/errors
|
||||
|
||||
## Risks and rollback
|
||||
|
||||
### Risk matrix
|
||||
|
||||
| Risk | Likelihood | Impact | Mitigation |
|
||||
|---|---|---|---|
|
||||
| `MAX_ARRAY_LEN=1000` too small for ultra-long sessions | Low | Medium | Env var tunable to 5000-10000; events table is always complete, UI can still query |
|
||||
| Extra dedup SELECTs after cache truncation | Medium | Low | Sweep runs every 60-300s; an extra 100-1000 primary-key lookups per run is acceptable |
|
||||
| ALTER TABLE fails on old DB | Very low | High | Use the migration pattern at `db.js:284` — try-catch + column-existence check |
|
||||
| transcript_path backfill is slow due to events scan | Low | Low | One-time migration takes ~1s; use EXISTS subquery instead of join |
|
||||
| `_set()` shape change breaks other readers | Low | Medium | Grep the repo to confirm all external consumers of `extract()` only read `result.*` |
|
||||
|
||||
### Rollback
|
||||
|
||||
- All changes are confined to `server/`; **hook-handler / UI / WebSocket protocol untouched**
|
||||
- Rollback at any phase = `git revert` of the matching commit
|
||||
- DB schema: `ALTER TABLE ... ADD COLUMN` is not reversible, but an unread/unwritten new column is harmless; once code is reverted, sessions just has an extra empty column
|
||||
|
||||
## Optional follow-ups (out of scope here)
|
||||
|
||||
- Add a `(session_id, event_type, created_at)` composite index on events (UI query performance)
|
||||
- Add a `lastProcessedTurnTimestamp` cursor to the cache so `extract` only returns new turns (eliminates dedup SELECTs entirely)
|
||||
- `/api/internal/memory` diagnostic endpoint returning `cache.stats()` + `process.memoryUsage()`
|
||||
|
||||
## Decision record
|
||||
|
||||
| Option | Choice | Rationale |
|
||||
|---|---|---|
|
||||
| Remote deployment vs fix leak | Fix leak | Remote deployment relocates the problem; the leak hits remote too |
|
||||
| Permanent events retention vs retention policy | Permanent | Hard user constraint: guarantee Agent log integrity |
|
||||
| Truncate cache vs not truncate | Truncate to MAX_ARRAY_LEN | Events table already persists raw data; the cache is a derived view |
|
||||
| Delete events vs rewrite the sweep query | Rewrite the query | Satisfies the "no log loss" constraint |
|
||||
| Introduce LRU byte-budget instead of entry count | No | Entry-count cap + per-entry cap is already enough; byte accounting adds complexity |
|
||||
@@ -0,0 +1,256 @@
|
||||
# Tabby — Floating Companion (Design Spec)
|
||||
|
||||
**Date:** 2026-05-28
|
||||
**Status:** Approved (design) — pending spec review before planning
|
||||
**Owner:** Nguyễn Ngọc Trí Vĩ (David)
|
||||
**Topic:** A cute-but-functional cat companion that lives in the dashboard's bottom corner, reacts to live session events, and expands into a panel for status, quick actions, and asking questions.
|
||||
|
||||
---
|
||||
|
||||
## 1. Summary
|
||||
|
||||
**Tabby** is a floating cat avatar pinned to the bottom-right corner of the Agent Dashboard on every route. It is two things at once:
|
||||
|
||||
1. **A reactive mascot** — an SVG cat whose face, ears, eyes, and posture react in real time to what the monitored Claude Code sessions are doing (a session finishes → tail-up, eyes `^^`; an error/hook fails → arch + ears-back; idle → curls up asleep). Eyes track the cursor when alert.
|
||||
2. **An assistant** — click the avatar (or press `⌘B` / `Ctrl+B`) to expand a panel with a live status line, quick navigation actions, and an **Ask** box that answers simple questions from cached dashboard data, with a handoff to the existing **Run** page to ask Claude for real.
|
||||
|
||||
The "do the job" path reuses what already exists: `POST /api/run` spawns a real `claude` subprocess and streams over WebSocket. Tabby does **not** introduce any new LLM backend, API key, or server route in P1/P2. P3 adds a single client-only deep-link prefill.
|
||||
|
||||
Name **Tabby** matches the app's identity: this is a **Monitor** ("watching your agents"), and Tabby is the alert watcher curled in the corner.
|
||||
|
||||
---
|
||||
|
||||
## 2. Goals / Non-Goals
|
||||
|
||||
### Goals
|
||||
- Delightful, on-theme personality layer over live session data — "cute but does the job."
|
||||
- Always-present, low-footprint corner avatar that auto-surfaces notable events as transient speech bubbles, then settles.
|
||||
- One-keystroke (`⌘B`) expand to a functional panel: status, quick actions, local Ask.
|
||||
- Reuse the existing event stream (`eventBus`) and Run flow — no new backend in P1/P2.
|
||||
- Fully consistent with the existing dark Tailwind theme (`surface-*`, `accent`, `border`).
|
||||
- Accessible: keyboard-operable, `aria-live` bubbles, honors `prefers-reduced-motion`.
|
||||
- Degrades safe: if WebSocket is down/delayed, Tabby shows a calm/disconnected state — never errors, never blocks the page.
|
||||
|
||||
### Non-Goals (YAGNI)
|
||||
- No drag-to-reposition (fixed bottom-right).
|
||||
- No sound effects.
|
||||
- No new LLM/chat backend or API key (Ask is rule-based locally; real Claude = handoff to Run).
|
||||
- No server-side persistence (preferences in `localStorage` only).
|
||||
- No multi-avatar / skins / customization.
|
||||
- No changes to existing pages beyond the minimal mount + the P3 Run prefill.
|
||||
|
||||
---
|
||||
|
||||
## 3. Where it lives (architecture)
|
||||
|
||||
```
|
||||
App.tsx
|
||||
└─ useWebSocket(onMessage = eventBus.publish) // single shared socket, already exists
|
||||
└─ Layout.tsx
|
||||
├─ UpdateNotifier (existing global floater)
|
||||
├─ Tabby ◀── NEW: mounted here, sibling of UpdateNotifier
|
||||
└─ <Outlet/> (page routes)
|
||||
```
|
||||
|
||||
- **Mount point:** `client/src/components/Layout.tsx`, right next to `<UpdateNotifier/>`. This guarantees Tabby persists across every route and shares the one WebSocket connection.
|
||||
- **Data source:** the existing `eventBus` (`client/src/lib/eventBus.ts`).
|
||||
- `eventBus.subscribe(handler)` → every `WSMessage`.
|
||||
- `eventBus.onConnection(handler)` + `eventBus.connected` → WS up/down.
|
||||
- No prop drilling, no new context provider. The brain hook subscribes directly.
|
||||
- **Navigation:** quick actions use `react-router` (`useNavigate`) to jump to existing routes (`/sessions`, `/sessions/:id`, `/activity`, `/run`).
|
||||
|
||||
### Component layout (new, isolated directory)
|
||||
|
||||
```
|
||||
client/src/components/Tabby/
|
||||
Tabby.tsx # Container. Owns open/collapsed/muted state, ⌘B + Esc handlers,
|
||||
# localStorage persistence. Composes the three presentational parts.
|
||||
CatAvatar.tsx # Pure presentational SVG cat. Props: { mood, eyeTarget, reducedMotion }.
|
||||
# No data access — fully testable in isolation.
|
||||
SpeechBubble.tsx # Transient bubble. Props: { text, onDismiss }. aria-live="polite",
|
||||
# auto-dismiss ~4.5s. No data access.
|
||||
TabbyPanel.tsx # Expanded panel: status header + quick actions + Ask box.
|
||||
# Receives status summary + handlers as props.
|
||||
useTabbyBrain.ts # The brain. Subscribes eventBus → derives { mood, statusSummary,
|
||||
# bubbleQueue }. Owns all timers (idle/sleep/stuck). The only unit
|
||||
# that touches eventBus.
|
||||
intents.ts # Local Ask: maps a free-text question → templated answer from cached
|
||||
# status, or a { runHandoff: prompt } signal. Pure function.
|
||||
quips.ts # mood/event → randomized phrase pool. The personality. Pure data + picker.
|
||||
tabby.css # Keyframes (breathe/blink/ear-twitch/arch/tail-flick), translucency,
|
||||
# prefers-reduced-motion overrides.
|
||||
```
|
||||
|
||||
**Boundaries / contracts:**
|
||||
- `useTabbyBrain` is the *only* unit that subscribes to `eventBus`. Everything else receives plain props. This keeps the live-data surface in one place and the rest trivially testable.
|
||||
- `CatAvatar`, `SpeechBubble`, `TabbyPanel` are pure presentational components — given props, render UI. No side effects.
|
||||
- `intents.ts` and `quips.ts` are pure functions over inputs — unit-testable with no DOM.
|
||||
|
||||
---
|
||||
|
||||
## 4. Data flow
|
||||
|
||||
```
|
||||
server broadcast ──► useWebSocket ──► eventBus.publish ──► useTabbyBrain subscriber
|
||||
│
|
||||
(reduce WSMessage + timers into state)│
|
||||
▼
|
||||
{ mood, statusSummary, bubbleQueue }
|
||||
│
|
||||
┌──────────────────────────────┬──────────────┴───────────────┐
|
||||
▼ ▼ ▼
|
||||
CatAvatar(mood) SpeechBubble(next bubble) TabbyPanel(statusSummary)
|
||||
│
|
||||
quick action │ Ask
|
||||
▼
|
||||
useNavigate(route) | intents() → answer
|
||||
| or → /run?prompt=
|
||||
```
|
||||
|
||||
`useTabbyBrain` maintains a small in-memory model derived from the stream (it does not refetch):
|
||||
- `liveCount` — active sessions/agents currently working.
|
||||
- `errorCount` — sessions/agents in error since last clear.
|
||||
- `lastEventAt` — timestamp of most recent `new_event`/update (drives `stuck`/`sleeping`).
|
||||
- `connected` — from `eventBus.onConnection`.
|
||||
- `recentDone` — transient flag set on a `session_updated` → status `completed`, cleared after the happy animation.
|
||||
|
||||
The exact `WSMessage.type` union the brain switches on (from `client/src/lib/types.ts`):
|
||||
`session_created`, `session_updated`, `agent_created`, `agent_updated`, `new_event`,
|
||||
`import.progress`, `update_status`, `run_stream`, `run_status`, `run_input_ack`, `cc_config_changed`.
|
||||
Tabby only cares about: `session_created`/`session_updated`/`agent_created`/`agent_updated` (mood + counts),
|
||||
`new_event` (activity heartbeat → `lastEventAt`, and hook-failure detection via the event payload),
|
||||
`run_status` (run finished → `happy`). The rest are ignored.
|
||||
|
||||
These feed both the avatar mood and the panel's status line. Counts are best-effort from the stream; the panel may also read a one-shot from existing stats endpoints if needed for an accurate initial number (open item — see §10).
|
||||
|
||||
---
|
||||
|
||||
## 5. Mood state machine (rule-based brain)
|
||||
|
||||
Mood is a pure function of `(streamModel, timers)`, evaluated on every relevant event and on timer ticks. **Highest-priority matching state wins:**
|
||||
|
||||
| Priority | Mood | Trigger | Cat expression |
|
||||
|---------:|------|---------|----------------|
|
||||
| 1 | `disconnected` | WS down (`eventBus.connected === false`) | faded/desaturated, flat ears, still |
|
||||
| 2 | `worried` | `session_updated`/`agent_updated` with status `error`, or a hook-failure `new_event` | arch + puff, ears back, brow down, brief shake |
|
||||
| 3 | `stuck` | ≥1 live session AND `now - lastEventAt > STUCK_MS` | ears-up alert stare, `!` |
|
||||
| 4 | `happy` | `session_updated` → `completed`, or `run_status` finished (transient, ~4s) | tail-up, eyes `^^`, head-bob |
|
||||
| 5 | `thinking` | Ask in flight (panel) | head-tilt, `…` |
|
||||
| 6 | `watching` | ≥1 live session, recent activity | eyes track cursor, ears up, tail flick |
|
||||
| 7 | `sleeping` | no activity AND idle `> SLEEP_MS` | curled, eyes shut, `zzz` |
|
||||
| 8 | `idle` | default / fallback | slow blink, gentle breathe |
|
||||
|
||||
Constants (tunable, defined in `useTabbyBrain`): `STUCK_MS` (~10 min), `SLEEP_MS` (~3 min). All timers cleared on unmount.
|
||||
|
||||
**Event → mood mapping (concrete):**
|
||||
- `onConnection(true)` → recompute (leaves `disconnected`).
|
||||
- `onConnection(false)` → `disconnected`.
|
||||
- `session_updated` data.status `error` → `worried` (+ increment `errorCount`).
|
||||
- `session_updated` data.status `completed` → `happy` (transient) + decrement `liveCount`.
|
||||
- `session_created` / `session_updated` data.status `active` → `watching`, recompute `liveCount`.
|
||||
- `agent_updated` status `error` → `worried`.
|
||||
- `new_event` → refresh `lastEventAt`; hook-failure event types (confirm in build, see §10) → `worried`.
|
||||
- `run_status` finished → `happy` (transient).
|
||||
- (timers) inactivity → `stuck` (if live) or `sleeping` (if not).
|
||||
|
||||
---
|
||||
|
||||
## 6. Eyes & motion
|
||||
|
||||
- **Eye tracking (`watching`/`idle`):** pupils follow the mouse, clamped inside the eye socket via a small vector-normalize + clamp. Throttled (rAF or ~30ms) to stay cheap.
|
||||
- **On event:** eyes glance toward the bubble, then relax back to tracking.
|
||||
- **Ears/tail/body:** CSS keyframe animations in `tabby.css`, swapped by a `data-mood` attribute on the avatar root.
|
||||
- **`prefers-reduced-motion`:** static eyes (centered), no breathe/shake/arch — mood still conveyed via static pose + face. Detected via `matchMedia`, passed as `reducedMotion` prop.
|
||||
|
||||
---
|
||||
|
||||
## 7. Auto-surface (speech bubbles)
|
||||
|
||||
- Pipeline: event → `quips.pick(mood/event)` → enqueue bubble → show ~4.5s → dismiss → settle.
|
||||
- **Rate limit:** at most one bubble every few seconds; coalesce bursts ("3 sessions finished" instead of three bubbles).
|
||||
- **Mute toggle:** persisted in `localStorage`. Muted = no bubbles, but faces/animations still react. Toggle lives in the panel.
|
||||
- **Accessibility:** bubble container is `aria-live="polite"` so screen readers announce notable events without stealing focus.
|
||||
|
||||
Example quips (from `quips.ts`, randomized):
|
||||
- happy: "session wrapped 🐾", "nice, that one's done", "4m12s — clean run"
|
||||
- worried: "ow, an error", "a hook tripped — peek?"
|
||||
- stuck: "this one's been quiet a while…", "still chewing on something?"
|
||||
- sleeping: "zzz", "wake me if something happens"
|
||||
|
||||
---
|
||||
|
||||
## 8. Panel (click / ⌘B)
|
||||
|
||||
Opens as a small card anchored above the avatar. Themed with `surface-3`/`border`/`accent`.
|
||||
|
||||
**Status header:** `🐾 N live · M errored · ●connected` (from brain's `statusSummary`; `●` reflects WS state, colored by health).
|
||||
|
||||
**Quick actions** (each = `useNavigate` to an existing route, or a local toggle):
|
||||
- Jump to errored session → `/sessions/:id` (most recent error) or `/sessions?status=error`.
|
||||
- Active sessions → `/sessions` (or `/activity`).
|
||||
- **Run Claude** → `/run`.
|
||||
- Activity feed → `/activity`.
|
||||
- Mute / unmute bubbles (local toggle, persisted).
|
||||
- Clear alerts (reset `errorCount`).
|
||||
|
||||
**Ask box:**
|
||||
- P1/P2: `intents()` matches the query against a small set of local intents over cached status — e.g. *what's running*, *any errors*, *how many today*, *slowest* — and returns a templated answer rendered in the panel.
|
||||
- Unmatched query → offer: "Ask Claude directly?" → opens `/run?prompt=<query>` (P3).
|
||||
|
||||
**Dismiss:** `Esc`, click-outside, or re-press `⌘B`.
|
||||
|
||||
---
|
||||
|
||||
## 9. Phasing
|
||||
|
||||
### P1 — Mascot (delight, zero backend)
|
||||
- `CatAvatar.tsx` (full SVG + all moods + eye tracking + reduced-motion).
|
||||
- `useTabbyBrain.ts` (eventBus subscription, mood machine, timers, bubble queue).
|
||||
- `SpeechBubble.tsx`, `quips.ts`, `tabby.css`.
|
||||
- `Tabby.tsx` container mounting avatar + bubble; `⌘B` reserved but panel stubbed.
|
||||
- Mounted in `Layout.tsx`.
|
||||
- **Outcome:** living, reacting cat in the corner with auto-bubbles. No panel yet.
|
||||
|
||||
### P2 — Panel (functional)
|
||||
- `TabbyPanel.tsx`: status header + quick actions (router nav) + local Ask.
|
||||
- `intents.ts` local intent matching.
|
||||
- `localStorage` for `collapsed` + `muted`; mute/clear in panel.
|
||||
- `Settings.tsx`: a single on/off toggle for Tabby (persisted), read by `Tabby.tsx`.
|
||||
- **Outcome:** click/⌘B opens a useful panel; Ask answers from local data.
|
||||
|
||||
### P3 — "Do the job" handoff
|
||||
- `Run.tsx`: read `?prompt=` search param → `setPrompt(prefill)` on mount (mirrors the existing `?session=` pattern). Client-only, no server change.
|
||||
- Wire Ask's unmatched-query path → `/run?prompt=<query>`.
|
||||
- **Outcome:** Tabby can hand a real question to a real `claude` subprocess via the existing Run flow.
|
||||
|
||||
---
|
||||
|
||||
## 10. Open items (resolve during planning/build)
|
||||
1. **Accurate initial counts:** the stream gives deltas; on first mount counts are unknown until events arrive. Decide: (a) start at 0 and let the stream fill in (simplest), or (b) one-shot read from the existing stats endpoint for an accurate seed. Leaning (a) for P1, optional (b) in P2 panel.
|
||||
2. **Hook-failure detection:** confirm which `event` `eventType` values represent hook failures vs. normal lifecycle, so `worried` only fires on real problems. Verify against `server/routes/hooks.js` + DB event types during build.
|
||||
3. **Errored-session deep link:** confirm `/sessions` supports a `status=error` query or whether to navigate to the specific `/sessions/:id`.
|
||||
|
||||
---
|
||||
|
||||
## 11. Theme & accessibility notes
|
||||
- Colors strictly from existing tokens: `surface-0..5`, `border`/`border-light`, `accent`/`accent-hover`. Cat palette: warm accent-tinted body that reads on the dark `surface-0` background; soft glow via `accent-muted`.
|
||||
- Fonts inherit (`Inter` / `JetBrains Mono`) — bubble/status text uses existing classes.
|
||||
- Keyboard: `⌘B`/`Ctrl+B` toggle, `Esc` close, panel actions tab-focusable.
|
||||
- `prefers-reduced-motion`: disables continuous animation.
|
||||
- z-index above content, below modals; never traps focus when collapsed.
|
||||
|
||||
---
|
||||
|
||||
## 12. Verification (per CLAUDE.md)
|
||||
- **Frontend:** `npm run test:client`.
|
||||
- Unit tests for `useTabbyBrain` mood transitions (each event → expected mood, priority ordering, timer-driven `stuck`/`sleeping`).
|
||||
- Unit tests for `intents()` (known queries → templated answers; unknown → runHandoff).
|
||||
- Unit test for `quips.pick` (returns a string for every mood).
|
||||
- **No server change in P1/P2** → `npm run test:server` not required for those phases. P3 touches only `Run.tsx` (client) → still client-only; run `test:client`.
|
||||
- Manual: load dashboard, trigger a run, observe mood/bubble transitions; toggle reduced-motion; toggle mute; ⌘B/Esc.
|
||||
|
||||
---
|
||||
|
||||
## 13. File change summary
|
||||
**New:** `client/src/components/Tabby/{Tabby,CatAvatar,SpeechBubble,TabbyPanel}.tsx`, `client/src/components/Tabby/{useTabbyBrain.ts,intents.ts,quips.ts,tabby.css}`, plus `__tests__` for brain/intents/quips.
|
||||
**Edited:** `client/src/components/Layout.tsx` (mount, P1) · `client/src/pages/Settings.tsx` (on/off toggle, P2) · `client/src/pages/Run.tsx` (`?prompt=` prefill, P3) · i18n files (`tabby:*` keys, as strings are added).
|
||||
@@ -0,0 +1,74 @@
|
||||
# Stage auto-detection — design
|
||||
|
||||
**Status:** approved 2026-07-28. Sub-project B of three (C = worktree lanes, shipped on `feat/worktree-lanes`; A = merged Workspace page, next). Built after C because C settled the lane data model.
|
||||
|
||||
## Problem
|
||||
|
||||
A lane's stage only moves when the driving agent calls `ccam stage <name>`. Every un-instrumented session — which is most of them — sits at `idle` forever while its agent works, so the pipeline map shows nothing. The dashboard already receives every tool call the agent makes; it just never reads them.
|
||||
|
||||
## Goal
|
||||
|
||||
Infer a lane's stage from the hook stream it already ingests, and show it **without ever claiming it as evidence**.
|
||||
|
||||
## The rule that shapes everything
|
||||
|
||||
**Inference never renders green.** A node the dashboard inferred reaches `passed-no-evidence` (amber) at most; `done` requires a declared stage carrying evidence via `ccam stage --evidence`. If inference could paint a node green, the amber/green distinction — the reason this feature exists — would be worthless.
|
||||
|
||||
Declared always outranks detected. A lane that has declared `review` ignores a detection for `implement`.
|
||||
|
||||
## Signals that actually exist here
|
||||
|
||||
Verified against a real 121 MB install before designing:
|
||||
|
||||
| Source | Rows | Notes |
|
||||
|---|---|---|
|
||||
| `events.tool_name` | Bash 29 470, Read 20 236, Edit 4 678, Write 1 330, Agent 1 188, TaskUpdate 908, Skill 127 | the bulk of the signal |
|
||||
| `events.data.tool_input` | present on every `PostToolUse` | full Bash command strings, Edit/Write paths, Skill names |
|
||||
| `TodoWrite` | **0** | this Claude Code build uses `TaskCreate`/`TaskUpdate` instead — do not design around TodoWrite |
|
||||
|
||||
So the signal is `tool_name` plus a regex over `tool_input`. No model call, no extra query.
|
||||
|
||||
## Where the rules live
|
||||
|
||||
In the pipeline template, not in code. `server/data/pipelines/default.json` gains an optional `detect` array per node:
|
||||
|
||||
```json
|
||||
{ "id": "tests", "detect": [{ "tool": "Bash", "match": "\\b(npm (run )?test|pytest|vitest|jest|go test|cargo test)\\b" }] }
|
||||
```
|
||||
|
||||
A rule is `{tool, match?}`: `tool` matches `events.tool_name` exactly; `match` is a regex tested against a flattened string of the tool's input. A rule with no `match` fires on the tool alone. A custom template supplied through `DASHBOARD_PIPELINES_DIR` may define its own rules, so a team can teach the dashboard their own conventions without touching the code.
|
||||
|
||||
## Where it runs
|
||||
|
||||
`server/lib/stage-detect.js` exposes one pure function, `detect(pipeline, event) → {nodeId, signal} | null`. It is called from the existing fail-safe block in `touchLaneFromHook` (`server/routes/hooks.js`) that already resolves the lane — no new pass over the hook path, no new query, and the same swallow-everything guarantee, because a hook must never fail on account of bookkeeping.
|
||||
|
||||
## Anti-flapping
|
||||
|
||||
- **Forward only.** A detection whose node index is not greater than the current detected index is dropped. Reading a file after editing it must not pull a lane back to `plan`.
|
||||
- **Write only on change.** Bash alone accounts for 29 470 rows in a real install; the lane row is written only when the detected node actually advances.
|
||||
- **Declared wins.** If the lane's declared stage sits at or beyond the detection, nothing is written.
|
||||
|
||||
## Data model
|
||||
|
||||
Three additive columns on `lanes`, each behind its own `try { SELECT col } catch { ALTER }` probe so a partial migration self-heals: `detected_stage`, `detected_signal`, `detected_at`. `stage` keeps its exact current meaning — the declared stage.
|
||||
|
||||
`lanePayload` gains `detected_stage`, `detected_signal`, and a per-node `detected: boolean` inside `pipeline_nodes`.
|
||||
|
||||
## What the user sees
|
||||
|
||||
A detected node renders amber with a **dashed** border, distinguishing it from an amber solid node (declared without evidence). Its tooltip names the signal: `tests ← npm run test:server`. The lane card shows `auto: tests` when the detection is ahead of the declaration. The header line still shows the declared stage, because that is what the agent asserted.
|
||||
|
||||
## Deliberately not in scope
|
||||
|
||||
- No inference of `done`, and no inference of gate results. A gate is a judgement; only an agent may claim one.
|
||||
- No back-filling of history. Detection starts when this ships; existing lanes gain nothing retroactively.
|
||||
- No inference from `workflows.phases`. It exists and would work, but it covers only Workflow-tool runs and would need its own reconciliation with the declared stage — a separate feature if wanted.
|
||||
- No writing to `stage`. Ever. Detection lives in its own columns so that turning the feature off loses nothing.
|
||||
|
||||
## Testing
|
||||
|
||||
- Rule matcher: one test per shipped rule, plus a rule with no `match`, an invalid regex in a template (must be skipped, not crash the hook), and a tool the rules do not mention.
|
||||
- Monotonic guard: an out-of-order detection is dropped; a same-node detection writes nothing.
|
||||
- Declared precedence: a lane declared at `review` ignores an `implement` detection.
|
||||
- Fail-safety: a malformed event cannot throw out of the hook path.
|
||||
- **Inference never renders green:** given a lane with only detections and no declarations, no node in `pipeline_nodes` may have state `done`. This is the test that guards the feature's whole premise.
|
||||
@@ -0,0 +1,59 @@
|
||||
# Merged Workspace page — design
|
||||
|
||||
**Status:** approved 2026-07-28. Sub-project A of three (C = worktree lanes, shipped; B = stage detection, planned). Built last because it consumes both.
|
||||
|
||||
## Problem
|
||||
|
||||
Lanes and Run are two pages that describe the same activity. `Run` spawns a `claude` process in a directory and streams its output; `Lanes` shows what a lane is doing and where it is in its pipeline. A user watching an agent work has to hold both in their head, and the lane card cannot even send a prompt — `start` opens a promptless conversation run and `message` has no input field.
|
||||
|
||||
## Goal
|
||||
|
||||
One page. A lane strip across the top, the selected lane's pipeline beneath it, and that lane's Claude console below — with every capability the Run page has today.
|
||||
|
||||
## Decisions already taken
|
||||
|
||||
- **The merged page lives at `/run`.** `/lanes` redirects there. One sidebar entry.
|
||||
- **Every run belongs to a lane.** Choosing a working directory that no lane owns creates one (`kind='adopted'`) rather than running loose. A lane is, after all, just a working directory the dashboard is watching.
|
||||
- **Everything from Run survives:** slash-command autocomplete in the prompt editor, model / permission-mode / effort selectors, the token meter with cost, run history, and attach-to-a-live-run.
|
||||
- **Layout:** lane strip (horizontal, scrollable, with the counters and Add) → pipeline map of the selected lane → console. Selecting a lane switches both the pipeline and the console.
|
||||
|
||||
## Architecture
|
||||
|
||||
`client/src/pages/Run.tsx` is 3658 lines holding an envelope model, a merge/typewriter engine, a slash-autocomplete prompt editor, a token meter, cwd suggestions, run history and the page shell. It is extracted into pieces that the new page composes:
|
||||
|
||||
| Unit | Responsibility |
|
||||
|---|---|
|
||||
| `client/src/hooks/useRunStream.ts` | envelope state for one run id: subscribe `run_stream` / `run_status` / `run_input_ack`, merge envelopes, typewriter |
|
||||
| `client/src/components/run/RunConsole.tsx` | render the envelope stream, the prompt editor with slash autocomplete, the token meter, stop/clear |
|
||||
| `client/src/components/run/RunSetup.tsx` | mode / model / permission / effort / cwd / resume pickers, binary status, the limitations banner |
|
||||
| `client/src/components/run/RunHistory.tsx` | past runs, live runs, attach |
|
||||
| `client/src/pages/Workspace.tsx` | lane strip + `PipelineMap` + the three above |
|
||||
|
||||
**The extraction is mechanical and must not change behaviour.** Each unit moves in its own commit with the existing Run tests passing untouched except for import paths. Only once `Run.tsx` is a thin composition does the new page get built. Extraction and composition never share a commit — that is the difference between a reviewable refactor and an unreviewable rewrite.
|
||||
|
||||
## Server glue
|
||||
|
||||
Four small pieces, each independently useful:
|
||||
|
||||
1. **Runs start through the lane.** The UI always calls `POST /api/lanes/:id/start`, which already exists, sits behind the same-origin guard, and records `run_id` on the lane. `POST /api/run` stays for the CLI and other callers; the UI simply stops using it. Lane `start` gains `mode` so a headless one-shot is still possible.
|
||||
2. **`POST /api/lanes/ensure`** — `{cwd, title?}` returns the lane owning that path or creates an `adopted` one. Avoids the UI having to catch a 409 and re-read, and keeps the create-then-start pair from racing.
|
||||
3. **`dashboard_runs.lane_id`** — one additive column, set when a run is started through a lane, so history can be filtered per lane instead of guessed at by `cwd`.
|
||||
4. **A finished run releases its lane.** Today nothing clears `lanes.run_id` when a run ends on its own: the lane reads `running` forever and `message` keeps targeting a dead run. The run-spawner already knows the moment of exit (`actualExitedAt`, added on the worktree branch); on that event, clear the owning lane's `run_id` and set its status back to `idle`. This is a bug the merge exposes rather than causes.
|
||||
|
||||
## What the console must not do
|
||||
|
||||
**It never touches the lane's stage.** Typing `/code-review` in the UI does not move the lane to `review`; only `ccam stage` declares, and only detection (sub-project B) infers. The console is a window onto a process, not a driver of the pipeline. Keeping that boundary is what stops the pipeline from becoming a lie.
|
||||
|
||||
## Risks and how they are contained
|
||||
|
||||
- **The extraction is the whole risk.** 3658 lines, one of them the typewriter engine, with a screens snapshot over the page. Containment: one unit per commit, tests untouched but for imports, snapshot diffs read rather than regenerated, and the composition deferred until the last extraction is green.
|
||||
- **Two consoles for one lane.** Only one run is live per lane (`start` 409s when one exists), so the console shows exactly one stream.
|
||||
- **A lane created just to try a command** leaves an `adopted` lane behind. Acceptable: `adopted` lanes are never destroyable, forgetting one is a click, and the alternative — runs that belong to nothing — is what this design set out to remove.
|
||||
|
||||
## Testing
|
||||
|
||||
- Each extraction: the existing Run tests pass with only import changes, and the screens snapshot for `/run` is unchanged until the page itself changes.
|
||||
- `useRunStream`: envelopes merge in order; a `run_status` terminal event stops the stream; the subscription is disposed on unmount.
|
||||
- `POST /api/lanes/ensure`: returns the existing lane for a path already owned, for a path nested inside one, and creates exactly one lane under concurrent calls.
|
||||
- Run-exit releases the lane: after a run ends by itself, the lane's `run_id` is null and its status is `idle`.
|
||||
- The console does not move the stage: after a full run through the console, the lane's `stage` is what it was.
|
||||
@@ -0,0 +1,133 @@
|
||||
# Worktree-backed lanes + Shipyard-style lifecycle — design
|
||||
|
||||
**Status:** approved 2026-07-28. Sub-project C of three (B = stage auto-detection, A = merged Workspace page) — each gets its own spec, plan and execution cycle. C is being built first because it fixes the lane data model that the other two build on.
|
||||
|
||||
## Problem
|
||||
|
||||
A lane today is a pointer at a directory that already exists. Two agents working in parallel therefore work in the *same* checkout and collide — the exact failure Shipyard solves by giving every lane its own clone. CCAM has no provisioning at all: no way to create a lane's working copy, no way to reset it between features, no way to remove it, and no way to reclaim the database a finished lane leaves behind (a real install reached 121 MB).
|
||||
|
||||
## Goals
|
||||
|
||||
- A lane can own a **git worktree** that CCAM creates, resets and removes.
|
||||
- The lifecycle verbs mirror Shipyard's, because that vocabulary is proven: `add`, `clear`, `reset`, `remove`, plus `purge` as CCAM's analogue of Shipyard's per-lane `dropdb`.
|
||||
- Every destructive action is confirmed **against counted facts**, not adjectives.
|
||||
- Directories the user already had must be impossible for CCAM to destroy.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- No dependency bootstrap. A fresh worktree has no `node_modules`, no `.env`, no `.claude/settings.local.json` — all gitignored. Shipyard solves this with per-project `bootstrap`/`migrate`/`seed` hooks, which is a whole subsystem. Out of scope; documented as a limitation.
|
||||
- No per-lane ports, databases, or Docker services.
|
||||
- No orchestration. Unchanged from the existing feature: the driving Claude session declares its own stage.
|
||||
- `VACUUM` is not part of `purge` (see Database reclamation).
|
||||
|
||||
## Data model
|
||||
|
||||
Additive columns on `lanes`, each guarded by the repo's `try { SELECT col } catch { ALTER }` probe:
|
||||
|
||||
| column | meaning |
|
||||
|---|---|
|
||||
| `kind` | `adopted` \| `managed`. Default `adopted`, so every pre-existing row migrates into the safe class. |
|
||||
| `source_repo` | absolute path of the checkout a managed worktree was created from |
|
||||
| `base_branch` | the branch the worktree was cut from, e.g. `development` |
|
||||
| `slug` | sanitised from the title; used for both the directory and the branch name |
|
||||
|
||||
`branch`, `cwd`, `stage`, `stages` and the rest keep their current meaning. `cwd` stays `UNIQUE`.
|
||||
|
||||
Environment: `LANES_ROOT` (default `~/.claude/ccam-lanes`), `LANE_BASE_BRANCH`, `LANE_BRANCH_PREFIX` (default `feat/`).
|
||||
|
||||
Layout: worktree at `$LANES_ROOT/<repo-basename>__<slug>`, branch `<prefix><slug>`. Numbered `lane1..lane9` slots were considered and rejected — CCAM is multi-repo, and slot numbers carry no meaning without Shipyard's per-lane ports.
|
||||
|
||||
## Safety model
|
||||
|
||||
`adopted` lanes expose no destructive verb. No reset, no remove-with-worktree, no branch deletion. The UI hides those controls; the API refuses them.
|
||||
|
||||
A `managed` lane may be destroyed only when **all three** independent checks pass:
|
||||
|
||||
1. `kind === 'managed'`
|
||||
2. the lane's `cwd`, fully resolved (symlinks included), lies inside `LANES_ROOT`
|
||||
3. `git worktree list --porcelain` run in `source_repo` actually lists that path
|
||||
|
||||
Shipyard gets away with one check (`case "$DIR" in */lane$N`) because its directory names are fixed. Dropping numbered slots costs that guarantee, so three cheaper checks replace it. Every destructive function in `server/lib/worktree.js` re-runs the three checks itself rather than trusting its caller.
|
||||
|
||||
CCAM never runs `rm -rf` on a lane. Removal goes through `git worktree remove`; if git refuses, the error surfaces unchanged.
|
||||
|
||||
## Verbs
|
||||
|
||||
| verb | steps | destructive |
|
||||
|---|---|---|
|
||||
| `add` | resolve base → `git worktree add -b <prefix><slug> <dir> <base>` → insert lane row `kind=managed` | no |
|
||||
| `adopt` | today's `POST /api/lanes` — point a lane at an existing directory, `kind=adopted` | no |
|
||||
| `clear` | reset stage/status fields only (already implemented) | no |
|
||||
| `reset` | kill and await the run → `git fetch origin --prune` → checkout base → `reset --hard <base>` → `clean -fd` → delete the feature branch → recreate it from base → clear lane state | **yes** |
|
||||
| `remove` | kill and await the run → unlock if locked → `git worktree remove --force` → `git worktree prune` → delete the branch → delete the lane row | **yes** |
|
||||
| `purge` | delete the lane's sessions and their events plus the orphan `token_usage` rows | **yes** |
|
||||
|
||||
`clean -fd` deliberately omits `-x`, exactly as Shipyard does: gitignored files (`node_modules`, `.env`) survive a reset, untracked-but-not-ignored files do not.
|
||||
|
||||
Branch deletion never touches `main`, `master`, or the lane's `base_branch`, and only runs after the worktree holding that branch is gone.
|
||||
|
||||
## Preflight
|
||||
|
||||
`GET /api/lanes/:id/preflight?action=reset|remove|purge` returns counted facts, never prose:
|
||||
|
||||
```json
|
||||
{
|
||||
"action": "reset",
|
||||
"lane": 3, "branch": "feat/criteria-form", "kind": "managed",
|
||||
"dirty": 4, "untracked": 11, "unpushed": 2,
|
||||
"head": "9b3e74a",
|
||||
"blocked": ["unpushed-commits"],
|
||||
"warnings": ["no-remote"]
|
||||
}
|
||||
```
|
||||
|
||||
For `purge`: `sessions`, `events`, `tokenRows`, `bytesEstimate`, and `activeSessionSkipped`.
|
||||
|
||||
The confirmation modal renders those numbers. The action then re-verifies: the client echoes back the `head` and counts it was shown, and the server returns `409` if they moved. `unpushed > 0` blocks `reset`, and `remove` when a managed worktree is actually at risk, unless the request carries `{force: true}`. `unpushed` counts what the action would really discard — with no remote that is `<base_branch>..HEAD`, this lane's own work, not the repository's whole history.
|
||||
|
||||
## Concurrency
|
||||
|
||||
One mutex per lane serialises destructive actions, mirroring `repo_lock()` in the AutomaticWorkflow bot — where concurrent git operations on a shared checkout produced a real `git checkout` exit 128, not a theoretical one. A destructive action first kills the lane's run and **awaits its exit** before touching git.
|
||||
|
||||
`add` can take seconds on a large repo, so it returns `202` with `status=provisioning` and finishes in the background, broadcasting `lane_update` on completion — the same pattern Shipyard uses to drive its spinner.
|
||||
|
||||
## Edge cases and their resolutions
|
||||
|
||||
- **Branch already exists:** if unused, `git worktree add` without `-b`; if checked out elsewhere, refuse and name the other path. Slug collisions get a `-2` suffix.
|
||||
- **Base branch missing on origin:** resolve `origin/<base>` → local `<base>` → the source repo's current HEAD.
|
||||
- **Source repo is itself a worktree, or bare:** works; git resolves through `--git-common-dir`.
|
||||
- **Repo with no commits:** `worktree add` fails and the lane lands `failed` with git's stderr in `notes`. Preflight does not pre-empt it — the lane is created first, then provisioning reports the git failure, and the row is forgotten with `DELETE /api/lanes/:id`.
|
||||
- **Worktree directory deleted by hand:** the lane reports `missing` and only `remove` is offered, taking the prune path.
|
||||
- **`cwd` uniqueness:** `remove` deletes the row, so re-adding the same slug is clean; `reset` keeps the path.
|
||||
|
||||
## Database reclamation
|
||||
|
||||
`events` cascades from `sessions`, but `token_usage` has no foreign key — `purge` must delete those rows explicitly or leave orphans. The currently-live session is never purged.
|
||||
|
||||
SQLite does not shrink on `DELETE`. `purge` runs `DELETE` plus `PRAGMA optimize` and reports the reclaimable size; `VACUUM` is a separate, explicitly-labelled maintenance action because it locks the whole database for seconds. Hiding a database-wide lock inside a button labelled "clean up" would be a trap.
|
||||
|
||||
## Surfaces
|
||||
|
||||
**API:** `POST /api/lanes/worktree` (add), `GET /api/lanes/:id/preflight`, and `reset` / `purge` joining the existing `POST /api/lanes/:id/:action` set, all behind the existing same-origin guard.
|
||||
|
||||
**CLI:** `ccam lanes add --repo <path> [--title <t>] [--base <branch>]`, `ccam lanes reset|remove|purge <id> [--force]`.
|
||||
|
||||
**UI:** a `managed` / `adopted` badge on the lane card; destructive buttons rendered only for `managed`; the existing `ConfirmModal` showing the preflight table.
|
||||
|
||||
## Testing
|
||||
|
||||
Against a real git repository fixture created in a temp directory — no mocks, because every bug worth catching here lives in git's actual behaviour:
|
||||
|
||||
- worktree created, listed, removed, pruned clean
|
||||
- each of the three safety refusals: an `adopted` lane, a path outside `LANES_ROOT`, a path git does not list as a worktree
|
||||
- `reset` keeps gitignored files and removes untracked ones
|
||||
- the unpushed-commit guard blocks, and `force` overrides it
|
||||
- preflight's counts equal what the action actually changes
|
||||
- `purge` removes sessions, events and token rows, and skips the live session
|
||||
- `add` returns 202 and broadcasts `lane_update` when provisioning finishes
|
||||
|
||||
## Known limitations
|
||||
|
||||
- A fresh worktree has no installed dependencies or local env files (see Non-goals).
|
||||
- Nine worktrees of a large repository cost nine working trees of disk; the `add` preflight estimates the size first.
|
||||
- `--repo` may point anywhere the user can read. That is their own machine; validation is limited to "absolute, exists, is a git repo", and every route stays behind the loopback guard.
|
||||
@@ -0,0 +1,140 @@
|
||||
# Workspace UI rebuild — design
|
||||
|
||||
**Status:** approved 2026-07-29. Sub-project D, built on top of A (merged Workspace page).
|
||||
Reference: the Shipyard "Feature Harness" screen the user supplied.
|
||||
|
||||
## Problem
|
||||
|
||||
The merged Workspace page shipped with the right information and the wrong shape.
|
||||
Everything a lane knows — declared stage, inferred stage, progress, liveness,
|
||||
needs-you — is already on the card (`client/src/components/lanes/LaneCard.tsx`)
|
||||
and already correct on the wire. None of it is legible: the lane strip is a
|
||||
horizontal scroller of cramped cards, the pipeline sits above a console that
|
||||
dominates the viewport, and the `auto: <stage>` chip that proves detection works
|
||||
is 10px of amber text nobody sees.
|
||||
|
||||
Measured on the live install while writing this: lane 5 carried
|
||||
`detected_stage: "tests"` with a real signal, and the user's report was
|
||||
"the lane does not auto-detect". Detection was never broken. The display was.
|
||||
|
||||
Two facts also make lanes look emptier than they are:
|
||||
|
||||
- `branch` and `ci_status` are columns nobody writes, so those rows are always
|
||||
blank even for a managed worktree sitting on a real branch.
|
||||
- Detection is forward-only with no expiry, so a lane parks at the highest stage
|
||||
it ever touched. Lane 5 reached `tests` and can never show `implement` again,
|
||||
even while the agent is editing code.
|
||||
|
||||
## Goal
|
||||
|
||||
The reference screen's legibility, on CCAM's real data: a lane's state readable
|
||||
from across the room, the pipeline large enough to trace, and the console present
|
||||
but out of the way until wanted.
|
||||
|
||||
## Decisions taken
|
||||
|
||||
- **Card grid, not a strip.** Responsive 1 / 2 / 3 columns.
|
||||
- **The console collapses.** It keeps every capability from A; it starts
|
||||
collapsed and opens for the selected lane. Watching lanes is the default
|
||||
posture, driving one is the exception.
|
||||
- **Only real data.** No placeholder tiles for facts CCAM does not have
|
||||
(tickets, preview ports, per-lane credentials). Branch/commit/CI are added
|
||||
because they can be read for real — see below.
|
||||
- **Detection expires.** A detection older than a TTL stops holding the floor.
|
||||
|
||||
## Layout
|
||||
|
||||
Top to bottom, one column:
|
||||
|
||||
```
|
||||
header: title · [N lanes][N running][N need you][N dead] · [+ Add lane]
|
||||
detail: selected lane · declared + inferred headline · large PipelineMap · legend
|
||||
console: collapsed by default; expands to RunSetup + RunConsole + RunHistory
|
||||
grid: lane cards, 1/2/3 columns
|
||||
```
|
||||
|
||||
Selecting a card switches the detail panel and the console together, exactly as
|
||||
A wired it. The console is unchanged behind its new disclosure — no prop of
|
||||
`RunConsole`, `RunSetup` or `RunHistory` moves.
|
||||
|
||||
## The card
|
||||
|
||||
Reference layout, CCAM's fields, nothing invented:
|
||||
|
||||
| Row | Content | Source |
|
||||
|---|---|---|
|
||||
| header | `LANE <id>` · liveness dot · status | `id`, `liveness`, `status` |
|
||||
| title | title, falling back to `cwd` | existing |
|
||||
| progress | declared stage chip · bar · `%` · time on stage | `stage`, `progress`, `stage_seconds` |
|
||||
| inferred | dashed amber `auto: <stage>` with the signal as tooltip | `detected_stage`, `detected_signal` |
|
||||
| tags | `kind` (adopted/managed), CI when known | `kind`, `ci_status` |
|
||||
| git | branch · short head · last commit subject · dirty/untracked counts | new, see below |
|
||||
| alert | needs-you banner | `needs_action` |
|
||||
| actions | start · stop · clear · reset · remove | existing lane actions |
|
||||
|
||||
`reset` and `remove` keep their preflight + `expect` echo through
|
||||
`DestructiveLaneModal`. This redesign does not touch the destroy guard.
|
||||
|
||||
## Git facts
|
||||
|
||||
A new read-only endpoint, `GET /api/lanes/:id/git`, returning
|
||||
`{branch, head, subject, dirty, untracked}` or `{available: false}` when the
|
||||
lane's `cwd` is not a git repo or is unreadable.
|
||||
|
||||
Deliberately **not** folded into `GET /api/lanes`: that payload is polled and
|
||||
broadcast, and shelling out to git once per lane on the hot path would put a
|
||||
subprocess burst behind every hook-driven `lane_update`. The card fetches its
|
||||
own facts when it mounts and on a slow interval, and renders without them until
|
||||
they arrive.
|
||||
|
||||
`server/lib/worktree.js` already has `statusCounts(dir)` returning
|
||||
`{dirty, untracked, head}` and a `git()` wrapper that scrubs the inherited
|
||||
`GIT_*` environment. Both are reused as-is; the endpoint adds only the branch
|
||||
name and the commit subject. No second git helper, no shell strings.
|
||||
|
||||
## Detection expiry
|
||||
|
||||
`recordDetection` gains one rule: a `detected_stage` whose `detected_at` is
|
||||
older than `DETECTION_TTL_MS` (default 30 minutes) no longer blocks a new
|
||||
detection — the forward-only comparison is skipped and the fresh signal wins.
|
||||
Within the window nothing changes: forward-only and declared-wins hold exactly
|
||||
as they do today.
|
||||
|
||||
This keeps the anti-flapping property that motivated forward-only (a `Read`
|
||||
right after an `Edit` must not drag the lane backwards) while admitting the
|
||||
thing it got wrong: a work session ends, and the next one starts somewhere else
|
||||
in the pipeline.
|
||||
|
||||
**Unchanged, and not negotiable:** detection still never writes `lanes.stage`,
|
||||
and an inferred node still never renders `done`.
|
||||
|
||||
## Signal legibility
|
||||
|
||||
`detected_signal` currently captures the whole flattened tool input, so the chip's
|
||||
tooltip reads `cd /very/long/path && npm run test:server 2>&1 | grep …`. The
|
||||
matcher already knows which regex fired; the signal becomes the matched span plus
|
||||
a little context rather than the entire command. Cosmetic, but it is the text the
|
||||
tooltip exists to show.
|
||||
|
||||
## Risks
|
||||
|
||||
- **The console's disclosure is the only structural risk.** Mounting it inside a
|
||||
collapsed container must not unmount `useRunStream` and lose a live stream.
|
||||
The subscription stays mounted; only the visual container collapses.
|
||||
- **Git calls per card.** Bounded by the number of lanes on screen and a slow
|
||||
refresh; failure is silent and the card renders without those rows.
|
||||
- **The screens snapshot over `/run` will change.** It is read, not regenerated
|
||||
blindly.
|
||||
|
||||
## Testing
|
||||
|
||||
- The card renders every field from a fixture lane, and renders without the git
|
||||
block when the endpoint reports `available: false`.
|
||||
- A detected node still never carries `data-state="done"` — the premise guard
|
||||
from sub-project B is re-asserted at the new layout.
|
||||
- Collapsing and expanding the console does not tear down the run subscription:
|
||||
a stream envelope delivered while collapsed is present when it re-expands.
|
||||
- `GET /api/lanes/:id/git` returns the facts for a real repo fixture and
|
||||
`available: false` for a plain directory, and never shells out through a shell.
|
||||
- A detection older than the TTL is accepted even when it is behind the current
|
||||
`detected_stage`; one inside the window is still refused.
|
||||
Reference in New Issue
Block a user