feat(ai-tools): install/update AI CLIs, remote catalog sync, cached favicons (#40 A2)
This commit is contained in:
@@ -0,0 +1,213 @@
|
||||
# AI Tools — part A2: install, update, remote catalog, favicons
|
||||
|
||||
Issue: #40. Date: 2026-09-28. Status: design agreed in chat, self-reviewed,
|
||||
awaiting spec review. Builds on A1
|
||||
(`2026-09-28-ai-tools-a1-design.md`, whose § "A2 decisions" this refines).
|
||||
|
||||
## Goals
|
||||
|
||||
- An **Install** button on "Available to install" rows for the tools whose
|
||||
install command is known; a confirm dialog shows exactly what will run; a
|
||||
live log appears under the row; the tool is re-detected (and enabled) when
|
||||
the job ends.
|
||||
- **Update** on installed rows, a "new version" badge for npm-distributed
|
||||
tools, and **Update tools** in the section header (named so, not "Update
|
||||
all", because Settings already has an **Update all** for plugins).
|
||||
- The catalog refreshes itself daily from the newest AgentHub release, so a
|
||||
new tool or a fixed flag reaches an install without a plugin update.
|
||||
- Real icons, fetched by the server, never by the browser.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Install data for every tool. A2 ships it for a verified few (§ 1); the
|
||||
remote catalog can add more later.
|
||||
- `sudo`, Homebrew, `uv`, `pipx`. A system-wide Node's `EACCES` is reported,
|
||||
not worked around.
|
||||
- Uninstall. A job surviving a server restart (it is a child of the server
|
||||
and dies with it; the log is lost, the tool is re-detected on next start).
|
||||
|
||||
## 1. Catalog fields
|
||||
|
||||
Two optional fields in a catalog file:
|
||||
|
||||
- `install`: `{ "kind": "npm", "package": "<name>" }` or
|
||||
`{ "kind": "script", "url": "https://…" }`.
|
||||
- `package` matches `^(@[a-z0-9][a-z0-9._-]*\/)?[a-z0-9][a-z0-9._-]*$`.
|
||||
- `url` is `https:` and its host shares its last two labels with the host of
|
||||
the file's `homepageUrl`. (`ponytail:` two-label compare, not a public
|
||||
suffix list — wrong for `co.uk`-style hosts, none in the catalog; swap in a
|
||||
PSL if one arrives.) Against a malicious remote catalog this is only a
|
||||
sanity check — that file controls `homepageUrl` too; the trust is the
|
||||
forge's (§ 3).
|
||||
- `update`: `{ "kind": "npm" }` (needs `install.kind === "npm"`, reuses its
|
||||
package), `{ "kind": "script" }` (re-runs `install.url`), or
|
||||
`{ "kind": "self", "args": ["update"] }` (the tool's own updater, run as
|
||||
`[<detected path>, ...args]`).
|
||||
|
||||
A file with a bad `install`/`update` is rejected whole, like any A1 rule. A
|
||||
file from `DASHBOARD_AI_TOOLS_DIR` may not carry either field (nor `launcher`,
|
||||
as in A1) — that directory is writable by anyone who can drop a file, and an
|
||||
Install click would run what it names.
|
||||
|
||||
First batch (npm owners checked with `npm view <pkg> maintainers` on
|
||||
2026-09-28; each package's `bin` matches the catalog's `detect`):
|
||||
|
||||
| id | install | update |
|
||||
|---|---|---|
|
||||
| `codex` | npm `@openai/codex` | npm |
|
||||
| `gemini` | npm `@google/gemini-cli` | npm |
|
||||
| `qwen-code` | npm `@qwen-code/qwen-code` | npm |
|
||||
| `opencode` | npm `opencode-ai` | npm |
|
||||
| `copilot` | npm `@github/copilot` | npm |
|
||||
| `amp` | npm `@sourcegraph/amp` | npm |
|
||||
| `cursor` | script `https://cursor.com/install` | script |
|
||||
| `claude` | — (always present: it runs AgentHub) | self `["update"]` |
|
||||
|
||||
Antigravity gets neither until its updater is known.
|
||||
|
||||
## 2. Jobs — `server/lib/ai-tool-jobs.js`
|
||||
|
||||
- ONE job at a time, server-wide. A second request answers `409 EBUSY`.
|
||||
- A job is `{ id, tool, action: "install"|"update", status:
|
||||
"running"|"ok"|"failed", startedAt, endedAt, exitCode, log }`, kept in
|
||||
memory; `log` is stdout+stderr, capped at 256 KB (the head is dropped).
|
||||
After it ends it stays readable until the next job starts.
|
||||
- The argv comes only from the catalog entry:
|
||||
- npm install/update: `[<npm>, "install", "-g", "<package>@latest"]`, where
|
||||
`<npm>` is resolved on the **login shell's own PATH, in the shell's
|
||||
order** — not on `process.env.PATH`, where `hydrateShellPath` APPENDS the
|
||||
shell's directories after the unit's, so the pinned nvm Node's npm would
|
||||
still win, and `-g` there lands where the user's PATH never looks (the
|
||||
tool would "install" and stay undetected). `shell-path.js` therefore keeps
|
||||
the last shell PATH it read (`shellPath()`, `null` when hydration is off or
|
||||
failed ⇒ fall back to `process.env.PATH`), and a job's env PATH is
|
||||
`shellPath()` followed by `process.env.PATH`, so npm's own
|
||||
`#!/usr/bin/env node` picks the matching Node.
|
||||
- script: the server downloads `url` (https, 15 s timeout, ≤ 1 MB, the final
|
||||
URL after redirects must still be https on the same two-label host) to a
|
||||
0600 temp file and runs `["bash", <file>]`; the file is deleted after.
|
||||
- self: `[<detected path>, ...args]`.
|
||||
- `execFile`/`spawn` with an argv, never a shell; stdin is `ignore`; env is the
|
||||
server's env with the hydrated `PATH`; each command runs in its own process
|
||||
group (`detached`), and a 10-minute timeout sends the whole group SIGTERM,
|
||||
then SIGKILL after 5 s (`failed`, `ETIMEDOUT` in the log). A step ends on
|
||||
`close`, or 500 ms after `exit` — a daemon an installer left behind can hold
|
||||
the pipes forever, and `close` alone would keep the job (and `EBUSY`) alive.
|
||||
- On an `EACCES` in an npm job's output, the log gets one hint line: Node is
|
||||
installed system-wide, use a user-level Node (nvm/fnm) or install by hand.
|
||||
- When a job ends: `shell-path` hydrate + `aiTools.refresh()`; a successful
|
||||
install also sets the tool `enabled: true`; the version cache is dropped
|
||||
(§ 3) — and only THEN does the status leave `running`, because the client
|
||||
refetches once, on the first poll that sees it finished. A failure in that
|
||||
bookkeeping is appended to the log, never thrown.
|
||||
|
||||
## 3. Versions and the remote catalog
|
||||
|
||||
**npm-managed.** An installed tool counts as npm-managed only when the
|
||||
realpath of its detected binary lies under `<npm prefix -g>` (read once per
|
||||
versions pass with the § 2 npm). A tool installed another way — this
|
||||
machine's `codex` is Codex's standalone build under
|
||||
`~/.codex/packages/standalone/` — gets NO npm Update and no badge: an npm
|
||||
update would install a second copy the PATH order shadows, and the badge
|
||||
would never clear. Its row says `Installed outside npm` instead.
|
||||
|
||||
**Versions** (`update.kind === "npm"`, npm-managed tools only): installed =
|
||||
first `\d+\.\d+\.\d+` in `<path> --version`; latest = `npm view <pkg>
|
||||
version`. All tools in parallel, 5 s timeout each, cached in memory for 24 h,
|
||||
computed lazily on the first `GET /api/ai-tools/versions` and on Refresh.
|
||||
`updateAvailable` = latest > installed by the same compare `update-check.js`
|
||||
uses. Offline or unparsable ⇒ no badge, no error.
|
||||
|
||||
**Remote catalog** (`server/lib/ai-tools-sync.js`):
|
||||
|
||||
- Daily (and once 60 s after start), off with `AGENTHUB_AI_TOOLS_SYNC=0`.
|
||||
- Resolves the repo URL exactly as `update-check.js` does — marketplace
|
||||
manifest for a plugin-cache install, the canonical git remote for a
|
||||
checkout — through a new export `repoUrlFor(root)` there (built from its
|
||||
existing `detectPluginInstall`/`pluginMarketplaceUrl`/`pickCanonicalRemote`),
|
||||
and reads the newest release through `fetchReleaseFromUrl`.
|
||||
- Only when that release is NEWER than the running build's `package.json`
|
||||
version — a dev checkout ahead of the newest release must never be rolled
|
||||
back to older data.
|
||||
- Lists `server/data/ai-tools/` at the release tag through the forge contents
|
||||
API (`/api/v1/repos/<o>/<r>/contents/server/data/ai-tools?ref=<tag>`, the
|
||||
Gitea/Forgejo shape `releaseApiUrl` already builds), downloads each `*.json`
|
||||
(≤ 64 KB each, ≤ 200 files), validates each with A1's `validate` at
|
||||
**release** trust (may carry `launcher`, `install`, `update`), and writes the
|
||||
set atomically to `<runtime dir>/ai-tools-remote/` with a `meta.json`
|
||||
(`tag`, `fetchedAt`). A failed sync leaves the previous set untouched.
|
||||
- Precedence per id: `DASHBOARD_AI_TOOLS_DIR` › remote › built-in. A remote set
|
||||
whose `tag` is no longer newer than the running build (the user upgraded) is
|
||||
ignored.
|
||||
- `GET /api/ai-tools` gains `catalog: { source: "builtin"|"remote", tag,
|
||||
fetchedAt }`.
|
||||
|
||||
## 4. Icons — `GET /api/ai-tools/:id/icon`
|
||||
|
||||
- Domain = the file's `faviconDomain`, else the `homepageUrl` host. It must
|
||||
be a dotted hostname (`^[a-z0-9.-]+\.[a-z]{2,}$`, no IP literal, no port).
|
||||
- The server fetches `https://<domain>/favicon.ico` (10 s, ≤ 256 KB — multi-resolution .ico files run past 100 KB — final URL
|
||||
https on the same site — last two host labels, so `github.com` →
|
||||
`github.githubassets.com` is refused but `www.` redirects pass), accepts only PNG, ICO, JPEG, GIF, WebP recognised by their magic bytes
|
||||
(the Content-Type header is ignored: sites omit or mislabel it) —
|
||||
**never SVG**, which would run script on the dashboard's origin if the URL
|
||||
were opened directly — and caches it in `<runtime dir>/ai-tool-icons/`. A
|
||||
failure is cached for 24 h as a miss.
|
||||
- Served with the cached type, `X-Content-Type-Options: nosniff`,
|
||||
`Content-Security-Policy: sandbox`, `Cache-Control: max-age=86400`; a miss is
|
||||
`204` (a 404 would log a console error per letter avatar on every visit)
|
||||
and the client keeps its letter avatar; an unknown id is `404`.
|
||||
|
||||
## 5. API
|
||||
|
||||
All writes behind `sameOriginGuard`; nothing in a body names a command.
|
||||
|
||||
| Route | Does |
|
||||
|---|---|
|
||||
| `POST /api/ai-tools/:id/install` | starts the install job → `202 {job}`; `404 ENOTOOL`, `400 ENOINSTALL` (no `install`), `409 EINSTALLED`, `409 EBUSY` |
|
||||
| `POST /api/ai-tools/:id/update` | starts the update job; `400 ENOUPDATE`, `409 ENOTINSTALLED`, `409 ENOTNPM` (npm update on a tool installed outside npm), `409 EBUSY` |
|
||||
| `POST /api/ai-tools/update-all` | one job that updates, in catalog order, every npm-managed tool with `updateAvailable` plus every installed tool with a `self` updater; its log concatenates each step's; `400 ENOTHING` when the list is empty |
|
||||
| `GET /api/ai-tools/job` | the current or last job, or `null` |
|
||||
| `GET /api/ai-tools/versions` | `{ [id]: { npmManaged, installed, latest, updateAvailable } }` for tools with `update.kind === "npm"` |
|
||||
| `GET /api/ai-tools/:id/icon` | § 4 |
|
||||
|
||||
`GET /api/ai-tools` rows gain `install` (`{kind, package?|url?}` or null) and
|
||||
`update` (`{kind}`, plus `args` for `self`, or null) so the dialog can show the exact command.
|
||||
|
||||
## 6. UI (`AiToolsSettings.tsx`, `useAiTools.ts`)
|
||||
|
||||
- Avatar: `<img src="/api/ai-tools/:id/icon">`, letter avatar on `onError`.
|
||||
- Available row with `install`: **Install** → confirm dialog showing
|
||||
`npm install -g <package>@latest` or `Download and run <url> with bash`,
|
||||
plus the homepage link. On confirm: POST, then poll `GET /job` every 1 s
|
||||
while `running`; the log renders under that row in a `<pre>` (monospace,
|
||||
max-height, auto-scroll); on end the list refetches.
|
||||
- Installed row with `update`: **Update** (same dialog and log) — for an npm
|
||||
update only when `npmManaged`, else the `Installed outside npm` note — and a
|
||||
badge `v<latest> available` when `updateAvailable`.
|
||||
- Header: **Update tools** beside **Refresh**, enabled when a badge or a
|
||||
`self` updater exists; its dialog lists what will run.
|
||||
- Every job button is disabled while a job runs; a page opened mid-job picks
|
||||
it up from `GET /job`.
|
||||
|
||||
## 7. Testing
|
||||
|
||||
Server (`npm run test:server`): field validation (bad package names, a script
|
||||
off the homepage's host, `install` in `DASHBOARD_AI_TOOLS_DIR` rejected,
|
||||
`update: npm` without npm install); jobs with an injected exec (argv exactly as
|
||||
§ 2, npm resolved from the shell's own PATH ahead of the unit's, one-at-a-time `EBUSY`, log cap,
|
||||
timeout, enable-after-install, EACCES hint); script download guards (size,
|
||||
https, redirect host); version parse/compare, the 24 h cache, npm-managed by realpath under the global prefix; sync
|
||||
(newer-only, precedence, atomic replace, a failed sync keeps the old set,
|
||||
release-trust `launcher`); icon guards (type allowlist without SVG, size,
|
||||
IP-literal domain refused, headers). Client (`npm run test:client`): Install
|
||||
confirm shows the exact command, the log polls and stops, badge and **Update
|
||||
tools**, avatar fallback.
|
||||
|
||||
## 8. Docs
|
||||
|
||||
`CLAUDE.md` marker, `AGENTS.md`, `docs/API.md` (six routes), `server/README.md`,
|
||||
`server/data/ai-tools/README.md` (the two fields, the trust rules),
|
||||
`server/openapi-extra/` + regenerated `openapi.yaml`, the env var
|
||||
`AGENTHUB_AI_TOOLS_SYNC` wherever env vars are listed, and the
|
||||
`update-project-docs` pass.
|
||||
Reference in New Issue
Block a user