docs(spec): AI Tools part A1 — catalog, detection, enable/default, lanes for any tool (#40)

docs/superpowers/specs/2026-09-28-ai-tools-a1-design.md

@@ -0,0 +1,268 @@
+# AI Tools — part A1: catalog, detection, enable/default, lanes for any tool
+
+Issue: #40. Date: 2026-09-28. Status: design agreed in chat, awaiting spec review.
+
+Part of a four-part split. **A1** (this spec) makes AgentHub know about many AI
+coding CLIs and run a lane on any of them. **A2** adds Install / Update, remote
+catalog sync and favicons (decisions recorded at the end, own spec later).
+**B** moves per-tool settings into each tool's Settings tab. **C** brings Codex
+to parity (token usage + pricing, hooks). **D** does the same for Antigravity
+after a spike.
+
+References studied: Orca (`stablyai/orca`, MIT) and 9Router (npm `9router`
+0.5.91, MIT). Catalog data (commands, flags, homepages) is taken from Orca's
+`agent-catalog.tsx`, `tui-agent-config.ts` and `tui-agent-permissions.ts`,
+with attribution in `server/data/ai-tools/README.md`.
+
+## Goals
+
+- Settings → Dashboard gets an **AI Tools** section: **Installed (N detected)**
+  with **Refresh**, and **Available to install**. Each row: icon (letter avatar
+  in A1, favicon in A2), label, command, **Enabled / Disabled**,
+  **Set default / ✓ Default**, homepage link ↗.
+- A newly installed CLI shows up without editing any config: the catalog is
+  broad (~40 tools, Oh My Pi included), detection re-reads the user's login-shell
+  PATH, and (A2) the catalog updates itself.
+- A lane can be created for any installed, enabled tool; it runs the tool's CLI
+  in the lane's tmux pane.
+- A disabled tool is hidden from Add lane and its Settings tab, and the server
+  refuses a NEW lane for it. Existing lanes keep working.
+
+## Non-goals (A1)
+
+Installing or updating a tool, remote catalog sync, favicons (A2). Per-tool
+settings tabs, pricing and hooks for non-Claude tools (B, C, D). Per-tool
+command/args/env overrides in the UI (Orca has them; rejected — they let a
+browser choose an executable for the server to run; a custom JSON file in
+`DASHBOARD_AI_TOOLS_DIR` covers the need). Heuristic discovery of CLIs that no
+catalog lists.
+
+## 1. Catalog
+
+One JSON file per tool in `server/data/ai-tools/<id>.json`, plus any
+`*.json` in `DASHBOARD_AI_TOOLS_DIR` (same override rule as
+`DASHBOARD_PIPELINES_DIR`: a custom file with a built-in id replaces it).
+
+```json
+{
+  "id": "antigravity",
+  "label": "Antigravity",
+  "detect": ["agy"],
+  "homepageUrl": "https://antigravity.google/docs/cli/overview",
+  "faviconDomain": "antigravity.google",
+  "prompt": { "mode": "flag", "flag": "-i" },
+  "resume": ["--continue"],
+  "bypass": ["--dangerously-skip-permissions"]
+}
+```
+
+- `id` — `^[a-z0-9][a-z0-9-]{0,39}$`. It is what `lanes.tool` stores.
+- `detect` — bare command names, no `/`. The first one found on PATH is also
+  the command the lane runs (Orca's `detectCmd` + aliases).
+- `requires` (optional) — other commands that must also be on PATH.
+- `args` (optional) — fixed args right after the command, before anything
+  else (e.g. Hermes needs `["chat"]` before its `--query` prompt flag).
+- `prompt` — how the first prompt reaches the tool:
+  - `{ "mode": "argv", "separator": "--" }` — trailing positional, optional
+    option terminator before it;
+  - `{ "mode": "flag", "flag": "-i" }` — `[flag, prompt]`;
+  - `{ "mode": "after-start", "delayMs": 3000 }` — start the bare CLI, then
+    paste the prompt into the pane after `delayMs` (`ponytail:` fixed delay;
+    add readiness detection like Orca's if a tool loses prompts).
+- `resume` (optional) — args that continue the last session. Absent: no Resume
+  button.
+- `bypass` / `bypassEnv` (optional) — args / env that turn off the tool's
+  permission prompts. Absent: no bypass switch.
+- `launcher` (built-ins only) — `"claude"` or `"codex"`: the lane keeps using
+  the existing hand-written argv builders (model, effort, permission mode,
+  sandbox, approval, AI Router). A custom file may not set it.
+- `faviconDomain`, `install` — used by A2; ignored in A1.
+
+**Where the data comes from.** Orca ships and exercises this data for ~40
+tools, so it is the source: `detectCmd`+aliases → `detect`, `requires` from
+`detectRequiredCommands`, prompt modes mapped as `argv` → `argv` (+ its
+`argvPromptSeparator`), `flag-prompt` → flag `--prompt`,
+`flag-prompt-interactive` → flag `--prompt-interactive`, `flag-interactive` →
+flag `-i`, `stdin-after-start` → `after-start`, `hermes-query` → `args
+["chat"]` + flag `--query`; `YOLO_TUI_AGENT_ARGS` split into argv arrays →
+`bypass`, `YOLO_TUI_AGENT_ENV` → `bypassEnv`. `resume` is filled only where
+the tool's `--help` or official docs confirm a continue-last flag; anything
+unconfirmed is **omitted**, not guessed. Tools whose launch is not "their own
+command" (Orca's `claude-agent-teams`, which launches through Orca itself) are
+left out.
+
+**Validation** (`server/lib/ai-tools.js`): a file that fails any rule above is
+skipped with one log line; it never throws at startup. Duplicate ids: custom
+beats built-in, else first file wins. Antigravity moves from hard-coded
+`pty-run.js` code to its catalog file.
+
+## 2. Detection
+
+- `detectTools()` resolves each tool's `detect` names by walking the PATH
+  directories with `fs` and checking for an executable file — no process per
+  tool (Windows is out of scope; AgentHub runs on Linux/macOS/WSL). Cached
+  until the next refresh.
+- **PATH hydration.** The server's PATH is whatever the systemd unit /
+  LaunchAgent captured at install time, so a CLI installed later into a new
+  directory is invisible. At startup and on Refresh, run the user's `$SHELL`
+  (fallback `/bin/bash`) with `-ilc` and a command that prints `$PATH` between
+  two delimiters, with a 5 s timeout; the text between the delimiters is merged
+  into `process.env.PATH` (new segments appended). Interactive (`-i`) because
+  nvm and most installers add to PATH in `.bashrc`, which `-lc` skips; the
+  delimiters discard whatever the rc files print. stdin is `ignore`, so an rc
+  file that waits for input or auto-attaches tmux gets EOF instead of hanging
+  until the timeout. Failure keeps the existing PATH. It runs after the server
+  is listening, never before. Off with `AGENTHUB_SHELL_PATH=0` for users whose
+  rc files must not run from a service. Taken from Orca's
+  `hydrate-shell-path.ts`.
+- Result per tool: `installed`, `path` (resolved binary), `command` (the name
+  that matched).
+- **The pane gets the same PATH.** A lane pane's environment comes from the
+  tmux server, not from AgentHub's process, and an npm-installed CLI is a
+  `#!/usr/bin/env node` script that dies without `node` on PATH. So every spawn
+  passes the hydrated `PATH` through `tmux new-session -e` / `respawn-pane -e`
+  (the channel `AGENTHUB_LANE_ID` already uses), and a catalog tool is started
+  by its resolved absolute `path`.
+
+## 3. Enable / default
+
+Stored in `<getDataDir()>/ai-tools.json`:
+`{ "disabled": ["grok"], "default": "claude" }`.
+
+- Tools are enabled unless listed in `disabled` — a freshly detected or
+  installed tool is on.
+- `default`: null or an id. Effective default = `default` if it is installed
+  and enabled, else `claude` if installed and enabled, else the first installed
+  enabled tool in catalog order, else null.
+- Only ids present in the catalog are accepted. Disabling the default clears it.
+
+## 4. API
+
+All mutating routes sit behind the existing `sameOriginGuard`.
+
+- `GET /api/ai-tools` →
+  `{ tools: [{ id, label, command, homepageUrl, installed, path, enabled,
+  source: "builtin"|"custom", launcher, prompt: {mode}, resume: bool,
+  bypass: bool, routerCapable: bool }], default: id|null }`.
+  `routerCapable` is true for `claude` and `codex` only. Arg arrays are not
+  returned.
+- `POST /api/ai-tools/refresh` → hydrate PATH, re-detect, same body as GET.
+- `PUT /api/ai-tools/:id` with `{ enabled?: boolean, default?: true }` →
+  same body as GET. Unknown id 404 `ENOTOOL`; making a not-installed or
+  disabled tool the default 400 `EBADDEFAULT`.
+- `GET /api/run/binary?tool=<id>` keeps its shape (`{found, path}`) but
+  resolves through the catalog; an unknown id stays 400 `EBADTOOL`.
+
+## 5. Lanes on any tool
+
+- `lanes.tool` accepts any catalog id. `VALID_TOOLS` (in `lanes.js`,
+  `pty-run.js`, `routes/lanes.js`) is replaced by `aiTools.has(id)`. No schema
+  change: `lanes.tool` and `dashboard_runs.tool` are unconstrained TEXT, and
+  `idx_lanes_cwd_tool` still allows one lane per (cwd, tool).
+- Creating a lane (`POST /api/lanes`, `/ensure`, `/worktree`) for a disabled
+  tool → 400 `EDISABLED`; unknown →
+  400 `EBADTOOL`. A tool that is not installed may still get a lane (it can be
+  installed afterwards); Start then fails with the existing missing-binary
+  banner.
+- `agenthub lanes add` has no `--tool` flag today; it gains `--tool <id>`
+  (default `claude`), and the help strings saying `--tool claude|codex` become
+  `--tool <id>`.
+- **Spawn** (`pty-run.js`): `launcher: "claude"|"codex"` → the existing
+  builders. Any other tool → `[path, ...args, ...resume?, ...bypass?,
+  ...prompt]` per the prompt mode; `after-start` sets a named tmux buffer with
+  `set-buffer -b <name> -- <prompt>` and pastes it with `paste-buffer -p -d -b
+  <name>` + Enter after `delayMs` (argv only — the tmux wrapper has no stdin),
+  skipped if the pane is gone. `POST /api/run` and `POST /api/lanes/:id/start`
+  both forward `bypass`.
+  Args come only from the catalog; the prompt is always ONE argv element or a
+  paste buffer, never part of a shell string.
+- **Bypass switch.** The run request carries `bypass: true`; the server
+  appends the catalog's `bypass` args / `bypassEnv`. Codex gains it
+  (`--dangerously-bypass-approvals-and-sandbox`); Claude does not (its
+  `bypassPermissions` permission mode already exists). Default off, never
+  remembered between runs. This replaces CLAUDE.md's "bypass flags are never
+  emitted" rule with "only when the user ticks the switch for that run".
+- A lane whose tool left the catalog: label falls back to the id, Start →
+  400 `ENOTOOL`, the lane is not touched otherwise.
+- Non-Claude tools: no pipeline, no agents install (as codex/antigravity
+  today); running/idle comes from the exited-pane heal sweep.
+- Places that list tools by name today and must follow the catalog instead:
+  `dashboard-runs.js` (stores any id, not codex/antigravity-or-claude),
+  `router-apply.js` (the Antigravity special case becomes "a tool that is not
+  `routerCapable` accepts only `native`, else `EUNSUPPORTED`"), `routes/run.js`
+  (binary probe). Filters that mean exactly one tool stay (`codex-ingest.js`,
+  `cc-reload.js`, hook ingestion pinned to `claude`).
+
+## 6. UI
+
+- `client/src/hooks/useAiTools.ts` — fetches `/api/ai-tools`, refetches on
+  window focus and after a mutation. Replaces `LANE_TOOL_LABEL` and the
+  Workspace's per-tool binary probes; `LaneTool` becomes `string`.
+- `client/src/components/settings/AiToolsSettings.tsx` — the AI Tools section
+  of Settings → Dashboard (Settings.tsx is ~1,900 lines already), with its own
+  table-of-contents entry. Rows as in Goals; A1 shows a letter avatar and no
+  Install button. An "Available to install" row has only the homepage link —
+  no Enabled/Default controls for a tool that is not there.
+- **Add lane — Which CLI:** two buttons, **Default (<label>)** and
+  **Other… ▾** (a dropdown of the remaining installed, enabled tools). With no
+  installed, enabled tool the modal says so and links to Settings → AI Tools.
+- **Model / AI Router row:** shown for every tool. The native option reads
+  **Native (<tool label>)** (the built-in router profile's label becomes plain
+  "Native" in `server/lib/ai-routers.js`); the AI Router option is disabled with
+  a tooltip for a tool that is not `routerCapable`.
+- **Run form, catalog tools:** cwd, prompt ("Ask <label> anything…"), **Resume
+  last session** when `resume`, and a red **Bypass permissions** checkbox with
+  a warning when `bypass`. Claude and Codex keep their forms; Codex gains the
+  checkbox.
+- **Settings tabs:** the Claude Code / Codex tabs are hidden while that tool is
+  disabled.
+- i18n: every new string in `en` and `vi`.
+
+## 7. Testing
+
+Server (`npm run test:server`):
+- catalog validation: bad id, `/` in a command, non-array args, `launcher` in
+  a custom file, duplicate ids, custom override;
+- detection over a temp PATH of fake executables (and a non-executable file),
+  and a stubbed shell for PATH hydration (delimiters, noisy rc output, timeout,
+  spawn error, `AGENTHUB_SHELL_PATH=0`);
+- the pane env carries the hydrated PATH; a catalog tool starts by absolute
+  path; `args` precede the prompt;
+- enable/default rules and `EBADDEFAULT`/`ENOTOOL`;
+- lane creation: `EDISABLED`, `EBADTOOL`, not-installed allowed;
+- argv for each prompt mode, resume, bypass on/off, Codex bypass, Claude
+  ignoring `bypass`, `after-start` paste scheduling;
+- `/api/run/binary` via the catalog.
+
+Client (`npm run test:client`): AI Tools section (rows, toggle, set default,
+refresh), Add lane Default/Other, disabled router option, catalog run form
+(resume, bypass warning), hidden tabs; snapshot baselines reviewed, not blindly
+updated.
+
+## 8. Docs
+
+CLAUDE.md (the lane-tool rule, the bypass rule, the router native label, the
+marker), AGENTS.md, docs/LANES.md, docs/API.md, docs/CLI.md (`lanes add
+--tool`), server/README.md, the OpenAPI source for the three new routes in
+`server/openapi-extra/` plus a regenerated `openapi.yaml`, and the
+`update-project-docs` pass.
+
+## A2 decisions (own spec later)
+
+- **Install:** a button per "Available to install" row; one install at a time,
+  live log under the row, re-detect and auto-enable when done. Commands come
+  only from catalog files shipped in a release (never from a request, never
+  from `DASHBOARD_AI_TOOLS_DIR`), as argv arrays. Vendor `curl … | bash`
+  scripts are allowed: the URL is fixed in the catalog, must be `https` on the
+  same registrable domain as the tool's homepage, is shown in a confirmation
+  dialog, is downloaded by the server to a temp file and run as `bash <file>`.
+- **Update:** per-row **Update** and **Update all**. npm-distributed tools
+  compare `npm view <pkg> version` with `<cmd> --version` (5 s timeout,
+  parallel, cached a day) and show a "new version" badge; script/self-updating
+  tools (Antigravity, Claude) only offer a re-run of their updater.
+- **Remote catalog:** fetched daily from the newest AgentHub release on the
+  forge (not `main`), validated like the local catalog, cached in the runtime
+  dir, off with `AGENTHUB_AI_TOOLS_SYNC=0`.
+- **Favicons:** fetched by the server once per `faviconDomain`, cached in the
+  runtime dir, served from `/api/ai-tools/:id/icon`; the browser never calls a
+  third-party service; letter avatar on failure.