Smartgift-AI/Claude-Code-Monitor
3
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity
Files
39c6e1c92093b0907a27d9a12222f4dadabb3e47
Claude-Code-Monitor/docs/superpowers/plans/2026-08-03-shipyard-parity-lanes.md
T

21 KiB
Raw Blame History

Shipyard parity for CCAM lanes — full plan

Source of truth for "Shipyard": ~/MyDrive/Projects/ResearchAndDevelopment/AgentWorkflow/ (the "Parallel Feature Harness"). Every capability below traces to a real file there, so each port can be checked against the original rather than a memory of it.

Goal: bring the whole harness into CCAM — isolated per-lane runtime and data, per-feature state with archive, proof gallery, cross-lane locks, the ship-feature pipeline with its QC agents, and the optional integrations.

Non-goal (explicit): OS-level sandboxing. Shipyard does not containerize either. Its isolation is resource namespacing on the host: separate working copy, ports, database, Redis logical DB, .env, upload dir. All lanes run as the same user, share the network, and can read any file the user can. Anything stronger is a separate design (§ Future).

Status

Subsystem Depends on Status
A1 Slots, ports, profile hooks, detached lifecycle — ✅ done 2026-08-03
A2 Data isolation: .env, database, Redis index A1 ✅ done 2026-08-03
A3 Stack detection + profile scaffolding A2 ✅ done 2026-08-04
B Per-feature state + archive — ✅ done 2026-08-04
C Proof gallery B ✅ done 2026-08-04
D Cross-lane named locks — ✅ done 2026-08-04
E ship-feature skill + QC agents A2·B·C·D planned
F Integrations (tracker / dev-QC / CI) E planned

A1 spec: docs/superpowers/specs/2026-08-03-lane-runtime-isolation-design.md.

Detail below tapers on purpose. A2 is written at implementation grade because everything it needs is known. A3, B, C, D are at design grade — decisions named, shapes fixed, exact code left to their own spec. E and F are at scope grade: E's skill is a port of a 15-stage document whose every bin/… call must be rewritten against command surfaces that B–D have not built yet, and writing that in detail now would be writing against imagined APIs. Each subsystem still gets its own spec → plan → implement cycle; this document is the map, not a substitute.

Invariants every phase must preserve

Carried from CLAUDE.md and proven in A1. Re-check each at review time.

  1. CCAM does not orchestrate. It offers primitives; the session sequences them. No chaining, no queue, no retry, no gate evaluation.
  2. The runtime never writes stage, status or notes — only allocation facts. status=running means an agent is working, not that a server listens.
  3. Liveness is computed, never stored. Anything CCAM does not control (a process, a port, a deploy) is re-derived on read.
  4. Never build a shell command as a string. execFile/spawn with argv arrays. Hooks are the deliberate exception and are spawned with a fixed argv.
  5. Config is parsed, never sourced. Hooks execute on purpose; declarations are only read.
  6. Adopted lanes are never destroyed and never written into.
  7. Every destructive path re-runs its own guard, never trusts its caller.

A2 — Data isolation

Goal: a lane gets its own database, its own Redis logical DB, its own .env, and its own data directories, so two lanes can run their stack at once without sharing state. This is the half of Shipyard's isolation A1 did not cover.

Traces to: bin/lane-env-seed.sh, the createdb/dropdb blocks in bin/lane-bootstrap.sh / lane-up.sh / lane-reset.sh / lane-remove.sh, _common.sh:lane_db / lane_db_url / lane_redis_url / lane_upload_dir.

Decisions to settle first

Question Proposed answer Why
Who creates the database? A db-create.sh / db-drop.sh hook (allowlist already exists in A1) CCAM must not assume Postgres. createdb vs mysqladmin create vs touch foo.db genuinely differ; the template ships Shipyard's docker-compose Postgres commands so a Postgres user gets parity by copying.
Who rewrites .env? CCAM, not a hook Mechanical and identical across stacks: parse KEY=VALUE, replace a declared set, keep the rest. Pushing it to every repo would duplicate ~30 lines and re-lose the JWT_SECRET lesson below in each.
Where do database credentials live? ~/.ccam/secrets.env, mode 0600 Machine-level, not repo-level (Shipyard's config/secrets.env). Never returned by any route and never logged; GET /runtime may report key names present, never values.
Does a lane without a database break? No DB_PREFIX empty (the default) means "no per-lane database": no name allocated, no hook called, no dead code. Same for REDIS=0.

Profile additions

# .ccam/profile/profile.env  (all optional; empty = feature off)
DB_PREFIX="myapp_l"          # lane in slot 3 -> myapp_l3 ; empty = no per-lane DB
DB_KIND="postgres"           # informational for A3's scaffolder; A2 gates on DB_PREFIX
DB_URL_SCHEME="postgresql"   # DATABASE_URL scheme
REDIS=1                      # 1 = allocate logical index = slot
ENV_FILES="backend/.env"     # files to seed, relative to the lane
ENV_SOURCE="backend/.env"    # source path in the source repo (defaults to ENV_FILES)
ENV_REWRITE="DATABASE_URL REDIS_URL UPLOAD_DIR"   # keys CCAM overwrites per lane
ENV_PRESERVE="JWT_SECRET"    # keys kept from the lane's EXISTING file on a --force refresh
UPLOAD_SUBDIR="backend/data/uploads"   # exported as UPLOAD_DIR

DB_NAME, DATABASE_URL, REDIS_URL, UPLOAD_DIR, TEST_DATABASE_URL (<DB_NAME>_test, for ci-gate) join the hook environment contract.

Steps

  1. server/lib/secrets.js — read ~/.ccam/secrets.env with the A1 parser (lane-profile.js:parseEnvFile, reuse it). Warn once and continue when the file is absent; refuse to load it when its mode is group/world-readable.
  2. server/lib/lane-slots.js — extend slotFacts: dbName, databaseUrl, redisUrl, uploadDir, testDatabaseUrl. Same one place every slot-derived fact already comes from.
  3. server/lib/lane-env.js (new) — seedEnv(lane, profile, { force }). Copies <source_repo>/$ENV_SOURCE → <lane>/$ENV_FILES when missing (or on --force), then rewrites $ENV_REWRITE keys in the file, so the file is correct on its own rather than merely masked by runtime exports.
    • Port the two hard-won behaviours verbatim. A --force refresh preserves $ENV_PRESERVE from the lane's existing file: swapping in the source's JWT_SECRET 401s a running lane until reboot. And a missing source .env falls back to .env.example with a loud warning, not silent success.
    • Refuses on an adopted lane (assertManaged): that file is the user's real working config.
  4. server/lib/lane-services.js (new) — ensureDatabase / dropDatabase calling the db-create / db-drop hooks.
    • dropDatabase runs assertManaged and asserts DB_NAME === slotFacts(lane.slot).dbName before spawning anything. Only a name CCAM derived can ever be dropped; nothing from a request reaches it.
  5. Wire into the lifecycle (lane-runtime.js, routes/lanes.js):
    • provision → allocate slot → seedEnv → bootstrap → db-create → migrate → seed
    • up → seedEnv (repair) → ensure DB exists → migrate every boot (Shipyard's note is right: lanes drift while idle and a stale schema cascade-fails the whole e2e suite) → seed only if the DB was just created
    • reset → down → git reset (existing) → bootstrap → clear LANE_DIRS → drop + create + migrate + seed, unless --keep-db
    • remove → down → db-drop for <db> and <db>_test → existing teardown
  6. Preflight + confirmation — lane-preflight.js reports the database name a reset/remove will drop, and the confirm dialog echoes it back like the existing counted facts. Dropping a database is the most destructive thing this whole roadmap adds; it gets the same echo-or-refuse contract as the rest.
  7. CLI — ccam lanes reset --keep-db; ccam lanes runtime gains the DB name and Redis index rows.
  8. Profile template — server/data/profile-templates/postgres-compose/ with db-create.sh / db-drop.sh holding Shipyard's exact docker compose exec -T $DB_SERVICE createdb/dropdb commands.

Verify

  • Fixture repo with a SQLite db-create.sh (touch $DB_NAME.db) — no Docker needed in CI: provision → file exists; reset → file recreated; remove → gone.
  • .env seeding: keys in ENV_REWRITE replaced, every other line byte-identical; --force preserves ENV_PRESERVE from the existing file; missing source falls back to .env.example and warns.
  • dropDatabase with a tampered DB_NAME throws before spawning.
  • seedEnv and dropDatabase both throw ENOTMANAGED on an adopted lane.
  • Secrets: a 0644 ~/.ccam/secrets.env is refused; GET /runtime response contains no secret value (assert on the serialized JSON).
  • Two lanes up at once, each writing its own database, neither seeing the other's rows — the actual point of the phase.

Risks

  • Dropping the wrong database. Mitigated by derived-name-only, the managed guard, and the echoed preflight. Review this path twice.
  • Secrets leaking into a log. runHook streams hook output to the websocket; a hook that echoes $DATABASE_URL publishes a password to every browser tab. Add a redaction pass over hook output for values that came from secrets.env.

A3 — Detection and scaffolding

Goal: ccam lanes profile init <repo> writes a working .ccam/profile/, so adopting a repo is one command instead of nine hand-written hooks. This is the gap that left Shipyard's own profiles/ empty.

Design grade. The presets must be derived from the profiles actually written during A1/A2 — that is why this phase is last in A, and its spec should start by reading them.

  • Detect from file signals, never from guessing: package.json (scripts dev/build/start, vite/next/express deps), docker-compose.yml (services named postgres/mysql/redis), pyproject.toml / requirements.txt (django/fastapi), manage.py, alembic.ini, prisma/schema.prisma, go.mod, Gemfile.
  • Scaffold, don't interpret. Detection writes a concrete, readable .ccam/profile/ the user owns and edits. The runtime never re-detects at boot; a wrong guess is fixed by editing a file, not by changing CCAM.
  • Unknown values are written as explicit TODO: markers rather than plausible defaults — a wrong default that boots something is worse than a refusal.
  • ccam lanes profile check — the harness-doctor equivalent: profile parses, declared hooks exist and are executable, no TODO: left, declared ports are free, secrets.env has what the declared database needs. Must exit non-zero on any of these.
  • Templates in server/data/profile-templates/<preset>/.

Verify: a fixture repo per preset scaffolds, profile check passes, and ccam lanes up boots it — the same end-to-end proof A1 used, once per preset.

B — Per-feature state and archive

Goal: a lane's history survives switching features. Today clearLane erases; Shipyard archives into state/laneN/<slug>.json with an .active pointer and keeps every past feature browsable.

Independent of A — can be built in parallel.

  • Schema: lane_features (lane_id, slug, title, branch, stage, stage_since, status, gate_decision, ci_status, qc_dev, stages, links, notes, archived_at), unique on (lane_id, slug), plus lanes.active_feature_id. The lanes row stays the live view so nothing downstream breaks.
  • clearLane becomes: snapshot the row into lane_features with archived_at, then reset. Nothing is lost by tidying up.
  • Slug canonicalization is the load-bearing detail. The slug keys three things — the state row, the branch, and (in C) the proof directory. Port Shipyard's state.sh activate rule exactly: drop a feat/ prefix, / and spaces to -, keep [A-Za-z0-9._-], refuse empty, and echo the canonical slug back so callers store what the server stored. Reuse worktree.js:slugify only if its output is identical; otherwise a separate function with its own test. Do not let the two drift.
  • Routes GET/POST /api/lanes/:id/features…; CLI ccam feature list|activate|show.
  • UI: a feature picker that swaps the detail panel to an archived snapshot while the lane keeps running.

Verify: activate → clear → activate a second slug → the first is still browsable with its final stage intact and the live row is clean; a slug with slashes and spaces lands as one flat segment; DELETE /api/lanes/:id cascades.

C — Proof gallery

Goal: the screenshots QC agents capture are visible in the dashboard, grouped by feature and phase. Depends on B for the grouping key.

  • Store at <lane>/.playwright-mcp/proof/<slug>/{qc-local,qc-dev,ticket}/.
  • Port _common.sh:ensure_proof_link. Clone-root proof/ becomes a symlink into the canonical directory, because screenshots land there whenever an MCP's --output-dir is not yet in effect. It exists to fix a real class of stranded evidence; skipping it recreates the bug.
  • Path containment is the entire security surface. Resolve, realpath, and assert the result is inside the lane's proof root. No request path reaches fs unresolved. This is the one part of C worth reviewing carefully.
  • Routes: manifest, file, delete. UI: thumbnails grouped feature → phase, lightbox, +N overflow, the 🎫 ticket-report link.

Verify: traversal attempts (../, absolute, symlink out of the root) all rejected; a proof written to the clone root still appears in the manifest.

D — Cross-lane named locks

Goal: serialize the steps that thrash a shared machine — builds, e2e runs — across all lanes. Independent; the smallest phase; ship it whenever.

server/lib/lane-lock.js today is an in-process promise chain: it serializes work within one lane, within one process. This is the other axis and must be a separate module (named-lock.js); conflating them would be a subtle bug.

  • mkdir for atomicity, owner file lane<slot> <epoch> — port lane-lock.sh. No flock dependency, so Shipyard's macOS brew install flock prerequisite disappears.
  • Time-based staleness with a floor. A holder older than LOCK_MAX_HOLD (default 2700s) is broken on the next acquire, but the floor of 300s stays: it is what stops a caller from force-breaking a live lock via an env var.
  • acquire heartbeats the waiting lane (~60s) so waiting never reads as stalled.
  • Port the etiquette text into docs/LANES.md. Waiting is normal; never kill a holder, never delete the lock directory, never shrink LOCK_MAX_HOLD. Agents read docs, and this is the rule they break.
  • GET /api/locks, ccam lock status|acquire|release, and a lock indicator on the lane card so a human can see why a lane is sitting still.

Verify: second acquire blocks then succeeds after release; a holder backdated past LOCK_MAX_HOLD is broken; one backdated within the floor is not, even with LOCK_MAX_HOLD=1; release from a non-holder is refused and leaves the lock.

E — ship-feature and the QC agents

Goal: port Shipyard's driving skill and its five agents. Needs A2·B·C·D.

Progress: pipeline template + skill text (E1) done 2026-08-04 — see docs/superpowers/specs/2026-08-04-ship-feature-skill-design.md. sync-base (E2) done 2026-08-05 — see docs/superpowers/specs/2026-08-05-sync-base-design.md. Agents and F's integrations remain.

This is where "CCAM does not orchestrate" is preserved by construction: the skill runs in the session and calls ccam commands; the dashboard still only records.

  • Pipeline template server/data/pipelines/ship-feature.json — the stages with aliases and detect rules in the existing node format. default.json untouched; a lane opts in via pipeline.
  • Skill .claude/skills/ship-feature-lane/SKILL.md — every "$HARNESS/bin/…" call rewritten to its ccam equivalent (state.sh set → ccam stage; activate → ccam feature activate; lane-ci-gate.sh → ccam lanes hook ci-gate; lane-up.sh --qc → ccam lanes up --qc; the .harness-lane marker disappears because CCAM already resolves a lane from cwd).
  • Keep, do not trim, the hard-won rules. Each encodes a real failure: the turn-liveness rule (every turn between stage 1 and done leaves a re-invoker pending), backgrounded-sleep polling, the e2e active-poll (a hung suite never fires its completion re-invoke), no-retry-cap/phase-clock, one-driver-per-MCP, never-push-base, and the test-only / localized re-entry fast paths.
  • Drop what CCAM makes unnecessary: the @@HARNESS_ROOT@@ install-time placeholder and the zsh source _common.sh warning. Keep an integration-mismatch check even though its original cause is gone — silently skipping ticket/dev-QC while reporting success is the failure that matters.
  • Agents → .claude/agents/: qc-local, dev-qc, senior-gate-reviewer, ticketer, pr-reviewer. Shipyard generates these per lane with embedded credentials; port that as ccam lanes agents install, writing to <lane>/.claude/agents/, git-excluded, creds at mode 0600.
  • ccam lanes sync-base — port lane-sync-dev.sh. The valuable part is the migration-number collision pre-check (exit 5): another lane's migration landed on the base with the number yours uses. Generic form driven by MIGRATIONS_DIR + a filename-number pattern. Also port the keep-ours merge driver for GENERATED_MERGE_PATHS plus the post-merge regen hook — it kills the single most common cross-lane conflict.

Verify: the skill is prose, so verification is a real dry run on a scratch repo walking stages 0→8, confirming every ccam command it names exists and behaves as documented. Each of those commands gets a CLI test. sync-base --check gets a unit test with a fixture collision.

F — Integrations

Goal: tracker, dev-site QC, CI deploy-wait. All off by default, read from <repo>/.ccam/profile/integrations.env, exposed as ccam lanes integration <name> (exit 0/1) and in GET /runtime.

  • Tracker — file one ticket per feature, idempotent (update, never duplicate), writing proof/<slug>/ticket/REPORT.html.
  • Dev-site QC — post-merge browser QC against the deployed site with a per-lane account; owns the qc_dev field; checkpoints to RESULTS.partial.md so a died agent resumes instead of restarting.
  • CI deploy-wait — port ci-job.sh (status/failures/rerun/cancel) as ccam ci, which is what makes the skill's flake-triage rule executable.
  • ccam lanes mcp sync — port lane-mcp-sync.sh: writes the lane's .mcp.json with Playwright servers pinned to the lane's own --output-dir. This is what stops two lanes' browser QC from sharing a profile directory.

Verify: all toggles off → the runtime reports them off and the skill's stage-skip path runs; a toggle on in the file but off via the API → the mismatch check fires instead of silently degrading.

Order

A1 ✅ ──▶ A2 ✅ ──▶ A3 ✅
                     │
B ✅ ──▶ C ─────────────┼──▶ E ──▶ F
D ✅ ────────────────────┘

B next (it blocks E, and its presets must be derived from the profiles actually written during A1/A2). B, C are independent (D done) of the A chain and of each other except C→B — take D first if the pain right now is lanes thrashing the machine, B first if it is clear erasing history.

Per phase, every time: spec → plan → implement → npm run test:server + npm run test:client → docs (update-project-docs) → review. No phase is done without its own end-to-end manual run, the way A1 proved detachment by killing the server and watching the stack survive.

Future: container isolation

Real isolation beyond Shipyard — per-lane container with its own filesystem view, network namespace and resource limits.

The hard part is not the container. It is that Claude Code, its hooks and its MCP servers must run inside it, so the hook → API path crosses a container boundary, and the cwd a hook reports becomes a container path that no longer matches the lane's host cwd. That breaks resolveLaneByCwd — the binding between a session and a lane, and the assumption the entire lane model rests on. ~/.claude would also have to be mounted or synthesized per lane.

A design document of its own, not a phase of this one.

Reference in New Issue View Git Blame Copy Permalink