AgentHub/docs/plans/server-mode/S3-PLAN.md

# S3 plan: one Linux user per account (2026-10-08, after S0 and a 4-way code review)

Scope (user, 2026-10-08):
- own workspaces, sessions and stats;
- one Linux user per account;
- own skills and plugins: an admin baseline, plus the member's own;
- own AI Proxy, with the admin's shared ones usable by everyone;
- one shared set of tool binaries;
- the shared Claude pool;
- a new Codex pool.

The S0 results and test users are recorded in `CHECKLIST.md`.

## Decisions 2026-10-08 (after the review)

- `hidepid=invisible` is required: `install-system` refuses without it.
- **Hide shared proxy keys** (new phase P1b): AgentHub becomes a loopback endpoint for AI Proxy profiles.
  - The pane gets `ANTHROPIC_BASE_URL`/`base_url` = `http://127.0.0.1:<port>/api/proxy-gw/<profile>` and a per-pane token as its key.
  - AgentHub checks the token (pane alive, the lane owner may use the profile), swaps in the real key, and streams the request to the profile's `baseUrl`.
  - Pool OAuth tokens are NOT covered: that ceiling stays documented.
- **Codex ChatGPT accounts may be used concurrently** (user's choice, against the recommendation). Refresh tokens rotate, so P7 starts with a spike on a real spare account. The design to verify:
  - AgentHub owns the refresh: one canonical `auth.json` per account in the pool;
  - it refreshes before expiry under a lock and rewrites every live copy;
  - it checks whether codex re-reads `auth.json` and what happens when two copies refresh.
  - If codex refreshes on its own and logs the others out, report back to the user before building.
- Members do NOT get (a) the Add-workspace **Worktree** mode (managed worktrees) or (b) the runtime layer. They use **Repo** mode plus the prompt box's **New worktree** (session worktrees). Both refusals are `EMEMBERLANE`, and admins keep both on ownerless workspaces.

## Architecture

**Identity**
- AgentHub user `alice` maps to Linux user `ah-alice`. The prefix cannot clash with a system account or the host account.
- Her home is `$AGENTHUB_USERS_ROOT/alice`, which is today's `folderAllowed` rule; in server mode it becomes mandatory.
- Admins get a Linux user too.
- Ownerless (pre-S3) workspaces stay host-run and staff-only.

**Dashboard process.** In server mode only, it runs as a system unit:
- `User=<host>`
- `SupplementaryGroups=agenthub`
- `AmbientCapabilities=CAP_DAC_READ_SEARCH`
- `InaccessiblePaths=-/etc/shadow -/etc/gshadow -/etc/sudoers -/etc/sudoers.d -/root`
- `Restart=always`
- `KillMode=process`

It must **not** set `NoNewPrivileges` or `CapabilityBoundingSet`, because both block `sudo`. `/home` stays visible, because the host's DB lives there.

**Three privileges, each narrow:**
1. **Read anything.** `CAP_DAC_READ_SEARCH` covers every tool and every file mode, with no per-read cost and no per-tool code. It does NOT cover `/proc/<pid>/cwd` of another uid; that needs CAP_SYS_PTRACE, which is root-equivalent, so liveness changes instead (see P3).
2. **Act as a member.** sudoers: `<host> ALL=(%agenthub-members) NOPASSWD: <tmux>, <git>, <gh>, /opt/agenthub/libexec/write-as`. It never allows root.
3. **Provision.** sudoers: `<host> ALL=(root) NOPASSWD: /opt/agenthub/sbin/agenthub-provision`, a fixed script with a strict name check.

**Panes**
- Each account has its own tmux server, with its socket at `<home>/.agenthub/tmux/sock`.
  - The socket's dir is setgid `agenthub`, and the socket is 0770 with `server-access -a <host>`.
  - S0 showed that tmux keeps group rw across attach, and that another member is refused.
- The server is started once via `sudo -u ah-x tmux -S … new-session`. After that every command is a plain host-side `tmux -S` client (no sudo): send-keys, respawn, paste, kill, attach.
- Pane env:
  - `HOME`
  - `PATH=/opt/agenthub/tools/bin:…`
  - `CODEX_HOME=<home>/.codex`
  - `CLAUDE_DASHBOARD_PORT`
  - `DISABLE_AUTOUPDATER=1`
  - none of the host's `XDG_RUNTIME_DIR`/`DISPLAY`.

**Hooks**
- A hook is authenticated by the peer uid, read from `/proc/net/tcp`. S0 confirmed this: a connection from `agh-test1` mapped to uid 1002.
- That uid's account may only touch its own sessions, runs and lanes, and `transcript_path` must lie under that account's home. Today any local process can forge any hook.
- Claude hooks are installed via `/etc/claude-code/managed-settings.json` and call `node /opt/agenthub/app/scripts/hook-handler.js`. The Codex and agy forwarders are written into each member's home at provisioning.

**Secrets in argv**
- `/proc` on this host is not `hidepid`. Every member's `ps` therefore sees what the dashboard passes as `tmux -e TOKEN=…` and `sudo … env K=V`.
- Mount `/proc` with `hidepid=invisible`, plus a `gid=` exemption for the host. `install-system` refuses to continue without it.

## Not offered for member-owned workspaces in S3 v1

These answer 403 `EMEMBERLANE` in server mode:
- managed worktrees (`POST /api/lanes/worktree`). Members use session worktrees instead (`claude --worktree`, made by Claude as the member).
- the runtime layer: lanes up, down, hook, profile, slots, proof, lane-mcp, env seed, merge driver and gc. These spawn as the host, write into `LANES_ROOT` under the host's home, and kill and probe pids.

Boot-time cleanups (`clearLaneProxies`, `removeStageDoc`) skip member lanes.

## Phases (order: P0 → P1 → P1b → P2 → P3 → P4 → P5 → P8 review → P6 → P7)

Branch: `server-s3`. Commits happen only when the user asks.

### P0. Leaks that exist today (no sudo; can ship at any time)
- [x] Make `GET /api/ai-tools/:id/config/:index` staff-only. Today a member can read the host's catalog config files.
- [x] Members' `/cc-config/commands|skills`: `?cwd=` must lie inside one of their own workspaces. Today they can list another user's skills.
- [x] (moved to P3) `cc-reload`: before S3 every pane shares the host's `~/.claude`, so reloading member panes after a host plugin change is CORRECT today. Restrict it to panes on the changed home only once panes have their own (P3).
- Verify: one server-mode test per leak.

### P1. Per-user AI Proxy (no sudo; also useful without S3)
- [x] Store layout:
  - shared = today's `agenthub-proxies/`, writable by admins only;
  - personal = `agenthub-proxies/users/<userId>/`, writable by its owner;
  - a user's names are unique across both (`freeName` checks the shared ones).
- [x] API: the list stays an array and each item gets `scope: "shared"|"personal"`. `/ai-proxies` leaves `ADMIN_WRITE_PREFIXES`, and route checks take its place. A member may test only their own profiles; the stored-key fallback still requires the saved `baseUrl`.
- [x] `resolveProxy(tool, name, ownerId)` looks in personal, then shared. `dashboard_runs.proxy` keeps the name, and a resume resolves with the lane owner's id.
- [x] Client: Settings shows two groups, and the picker lists both.
- [x] Docs: a shared key is hidden from the browser only; a pane can read it (the same ceiling as the pool).
- Done 2026-10-08: server-mode-proxies.test.js (11), AiProxyConfig tests (+4); full suites server 2657/2660 (3 = lane-detect, port 3000 held by a foreign next-server), client 1135/1135.
- Verify: server tests (A cannot see, use or test B's; a member cannot write shared; an admin can), client tests, a snapshot diff review.

### P1b. Hide shared proxy keys (no sudo)
- [x] `/api/proxy-gw/:profile/*`:
  - loopback only;
  - a per-pane token (hash in `dashboard_runs`, alive pane only), and the run's lane owner may use the profile;
  - streams body and SSE both ways and swaps `x-api-key`/`Authorization`;
  - never logs a key;
  - has a timeout.
- [x] `proxyForSpawn` points Claude/Codex at the gateway, never at the real key, for shared profiles (and for personal ones too: one code path).
- Done 2026-10-08: server-mode-proxy-gateway.test.js (11); verified the real CLIs' requests: claude 2.1.293 → `<base>/v1/messages?beta=true` with `Authorization: Bearer <ANTHROPIC_AUTH_TOKEN>`, codex 0.155.1 → `<base_url>/responses` with `Bearer <env_key>`. Server mode only; local mode unchanged.
- Verify: an e2e against a fake upstream (header swapped, streaming kept, a dead pane's token refused, another member's profile refused).

### P2. Host setup (sudo; on this machine with the agh-test users)
- [x] `sudo agenthub server install-system` is idempotent, has `--uninstall`, and does the following:
  - creates groups `agenthub` and `agenthub-members`;
  - creates `/srv/agenthub/users` (0751) and `/opt/agenthub/{tools,app,libexec,sbin}`;
  - writes the sudoers file and checks it with `visudo -c`;
  - writes the system unit and the managed-settings hooks;
  - checks `hidepid`.
- [x] `service-manager` gets a system variant used when server mode is on and the system unit exists; stop and restart go through sudoers `systemctl … agenthub`.
- [x] `plugin-bootstrap`: in server mode it never starts, stops or repoints the server (deps only). Under any uid other than the host it does nothing (otherwise a member's SessionStart would spawn a second dashboard).
- [x] Provision helper (root, bash):
  - checks the name against `^[a-z][a-z0-9_-]{0,28}$`, makes the Linux name `ah-<name>`, and refuses an existing account;
  - create: `useradd`, the home, `.agenthub/{tmux,handoff}`, the Codex/agy hook forwarders, the baseline settings;
  - disable: `usermod -L`, then end that account's tmux server;
  - never deletes.
- [x] `agenthub users add` and the Users page call it; `users.linux_user` is set.
- Done 2026-10-08 and INSTALLED on this machine (system unit runs the server-s3 worktree on :4820; ah-admin, ah-vinnt provisioned; undo: /tmp/agh-spike/UNDO.sh step 0). Findings: `InaccessiblePaths=/etc/shadow` breaks sudo (PAM) — only /root is hidden; a host with its own `(ALL) NOPASSWD: ALL` makes the sudoers allowlist moot (keep the host out of `sudo` on a real server); `usermod --unlock` refuses a passwordless account (enable = clear the expiry); the test runner must not see the host's install (`AGENTHUB_SYSTEM_UNIT_DIR`, `AGENTHUB_SERVER_CONF` in scripts/test-server.js). Tests: server-s3-system.test.js (14); full suite 2679/2682 (3 = port 3000).
- Verify: on this machine, `agenthub users add alice` creates `ah-alice`; running it again changes nothing.

### P3. Spawning and acting as the member
- [x] `server/lib/run-as.js`:
  - `accountForLane` / `accountForCwd` return `{linuxUser, home, uid}`, or null (null = the host, which is always the case in local mode);
  - `sudoArgv(acct, cmd, args, env)`.
- [x] `tmux.js`:
  - one socket per account, chosen from session name → lane → owner;
  - start via sudo and add `server-access`;
  - `listSessions`, `liveSessions` and `serverPid` loop over every socket, with per-socket caches.
  - Also: `pty-attach` gets `-S`; same change in `cc-update-all-helper` and `lanes shell`.
- [x] `worktree.js` `git()`: member lanes go through `sudo -u`. The same for `git-exclude` and `gh`. In `git-ops`, the patch goes via stdin and the temp index under the member's tmp.
- [x] fs writes in member lanes go through `/opt/agenthub/libexec/write-as`: `lane-file-write`, the `.gitignore` append. Callers keep their existing guards.
- [x] Pane handoff files go to `<home>/.agenthub/handoff/` (member-owned, setgid `agenthub` 2770; the host writes 0644). These are:
  - proxy settings;
  - the Codex proxy (`-c` flags, never a file in the member's `CODEX_HOME`);
  - workspace context and prompt files;
  - drop uploads.
- [x] `cc-reload`: only panes whose Claude home is the one that changed (host-run panes for a host plugin update).
- [x] Liveness: a member session is alive iff its pane's tmux session is live. No `/proc/<pid>/cwd` read; without this, live member sessions get reaped.
- [x] Hooks: peer-uid auth, `transcript_path` under the account's home, and `threadOf` and the agy check accept member homes.
- [x] Member-lane refusals (`EMEMBERLANE`).
- Done 2026-10-08, verified LIVE on this machine as member `s3probe` (ah-s3probe):
  - a shell pane and a Claude pane run as the member; attach over /ws-pty works;
  - stage, commit and file save through the API run as the member, and the file stays the member's;
  - Claude → `/api/proxy-gw` → a fake upstream received the REAL key, and the pane showed "pong";
  - member-hook hooks were accepted by peer uid and bound to the workspace;
  - the 0600 transcript was served.
  Found on the way:
  - sudo LOGS its argv to the journal, so no secret may cross sudo: the member's tmux server starts with a secret-free `agenthub-boot` session, and the real one is created as a client;
  - tmux gives a pane created by an unattached client THAT CLIENT's PATH, so member-socket calls carry MEMBER_PATH;
  - a member's first Claude run asks for theme, trust and bypass consent → P6 seeds onboarding + trust of the home (bypass consent stays the member's);
  - git commit messages appear in the sudo journal (not secret; privacy note for P8);
  - an OLD dashboard on the same DB (the 4830 dev server) abandons member runs, because it cannot see their tmux.
- Verify:
  - unit tests with fakes: argv builders, socket choice, the peer-uid parser on fixture text;
  - an opt-in integration script (`AGENTHUB_S3_IT=1`, real test users): start a pane, commit, save a file, ingest a hook;
  - a browser e2e as two members.

### P4. Reads across homes
- [x] `homes()` = the host plus each enabled user's home (`run-as.memberHomes`). Done per home: Claude transcript resolvers (`claude-home.projectsDirFor`/`allProjectsDirs`), session sync, codex-usage, codex-ingest, codex threadOf, agy hook path. Moved to P5 (no member can run agy/Cursor before /opt): antigravity and cursor pollers. Per-member quota readers: P5. Legacy-import marker: not needed (the 30 s sync sweeps member dirs). Originally per home:
  - Claude session sync (watchers plus the 30 s poll);
  - the codex-usage, codex-ingest, antigravity and cursor pollers (`running` keyed by home);
  - the legacy-import marker.
- [x] Transcript routes: fixed at the resolvers instead (cwd → the member's projects dir), which covers every caller. Verified live: the member's 0600 transcript was served.
- [x] Quota readers per member home: Claude, Codex, Cursor, Grok, Kimi, OpenCode. agy's live `/usage` stays host-only, because it needs the agy binary, which updates itself.
  Done 2026-10-09: `quotas({home})` (`server/lib/quota.js`) reads each tool at its default path under the member's home through `run-as`'s owner-checked reads (`opencode.db` + WAL from a private copy); `GET /api/pricing/quota` gives a developer/analyst their own (`accountForUser`), nothing when unprovisioned, the host's to an admin. No member Claude profiles (P6). Test: `server-s3-quota.test.js`. Not yet checked live on the test users.
- Verify: tests with two homes in a tmp dir, then a real check on the test users.

### P5. Shared tools in /opt (S4)
- [x] `/opt/agenthub/tools` is owned by the host: the dashboard updates it, members get r-x.
- [x] Publish step after an install or update (and once at `install-system`):
  - copy the versioned binary to `tools/<id>/<version>` and flip `bin/<id>`;
  - npm tools use `--prefix /opt/agenthub/tools/npm`.
- [x] For members, publish the running build (scripts, server, bin, mcp, statusline, production deps; no client) to `/opt/agenthub/app/<version>` at server start in server mode. Members need it for hooks, the CLI, MCP and the statusline.
- [x] The pane `PATH` puts `/opt/agenthub/tools/bin` first, and the catalog detects tools there.
- Done 2026-10-08 (`server/lib/shared-tools.js`, `agenthub server tools`):
  - a publish copies an ELF file, an npm package folder, or a script's own version folder; a shared `bin` folder is refused;
  - publishing runs after each install/update job, in the background at server start, and on the CLI command;
  - a member's catalog-tool pane runs `<root>/bin/<command>`, or gets 409 ENOTSHARED;
  - the app scripts are published with the `agenthub` launcher;
  - the agy and cursor pollers also read member homes.
  Live: claude, codex, agy and cursor-agent published (1.6 GB); claude, codex, cursor-agent and agenthub run as `ah-s3probe` (agy was not run: it self-updates).
  Follow-up: per-member quota readers. A member's Claude runs on the shared pool, and the quota route already masks accounts for members.
- Verify: after an update from the UI, a member pane sees the new version.

### P6. Skills and plugins
- [x] Seed a new member's `~/.claude.json`: `hasCompletedOnboarding`, theme, and trust for their home (live finding 2026-10-08). The Bypass Permissions consent stays the member's own.
  - Done: `agenthub-provision create` seeds onboarding + trust of the home (merged; a non-JSON file is left alone); the theme follows AgentHub's dark/light mode (`member-claude.setTheme`). Test: `server-s3-provision.test.js` runs the real script with stub `id`/`useradd`/`chown`/`runuser` and its fixed paths pointed at a temp dir.
- [x] Spike: does `/etc/claude-code/managed-settings.json` (`enabledPlugins` plus marketplaces) give a new Linux user the admin's baseline plugins? **Yes, from the first interactive session on.** Run 2026-10-09 on `agh-test1` (not a new user), Claude Code 2.1.293, with `sg-all@sg` (no hooks, public git marketplace) in a ~80 s window. The file was then restored byte for byte, and `agh-test1`'s `~/.claude` was restored from a tarball and verified.
  - Before any session: `claude plugin marketplace list` and `plugin list` were both `[]`. The CLI alone syncs nothing.
  - First session, onboarding NOT seeded: it stopped at the theme picker and synced nothing. The provision seed (P6, above) is a precondition.
  - Onboarding seeded and NOT logged in: within seconds the session cloned the `sg` marketplace, fetched `sg-all` 0.2.0 into `~/.claude/plugins/cache/` and marked it `.in_use`.
  - `installed_plugins.json` stayed empty, so `claude plugin list` shows `[]` even then. A managed-enabled plugin is loaded from the cache with no user-scope record. Check a member's plugins by the cache, not by `plugin list`.
  - So: install-system can put the baseline in managed settings for anything a marketplace can express. The `plugin-baseline.json` fallback stays for the rest, and it applies to every Claude Code on the machine, the host's included (enabling a plugin the host already has changes nothing for it).
  - Not checked: skills actually listed inside a logged-in session (no login on the test user).
- [x] Fallback (built, opt-in): with `/etc/agenthub/plugin-baseline.json` (`{"marketplaces": ["owner/repo", …], "plugins": ["name@marketplace", …]}`, root's, the admin edits it), creating a user runs `claude plugin marketplace add <source>` for each marketplace, then `claude plugin install --scope user <plugin>` for each plugin, AS the new member (`sudo -n -u ah-<name> HOME=… /opt/agenthub/tools/bin/claude …`, in the background; `agenthub users provision` runs it too, in the foreground, and is the retry). No file = nothing runs. An entry starting with `-` is skipped; `-y` is never passed. Sudoers: `/opt/agenthub/tools/bin/claude` joins the run-as-member list (no new power: the host already runs any command as a member through tmux). `server/lib/member-claude.js`, test `server-s3-provision.test.js` (fake sudo). Not verified live.
- [ ] A member's own plugins work from their terminal (`/plugin`) as soon as P3 lands, with no code.
- [ ] Later, optional: Settings for members, using `cc-config` with `home` = `req.user`'s and writes via `sudo -u`.

#### P6 spike: how to run it (a human, on the S3 box)
`scripts/server/spike-managed-plugins.sh` (never run by AgentHub). It changes `/etc/claude-code/managed-settings.json`, which EVERY Claude Code on the machine reads, the host's own sessions included, so run it in a quiet moment and undo right after.
1. Write the fragment to test, e.g. `/tmp/spike-fragment.json`:
   ```json
   {
     "extraKnownMarketplaces": {
       "agenthub": { "source": { "source": "github", "repo": "<owner>/agenthub" } }
     },
     "enabledPlugins": { "agenthub@agenthub": true }
   }
   ```
2. Note the current file: `sudo sha256sum /etc/claude-code/managed-settings.json`.
3. Run: `sudo bash scripts/server/spike-managed-plugins.sh run /tmp/spike-fragment.json 60`. It backs the file up once (`managed-settings.json.agenthub-spike-bak`, or a `.agenthub-spike-none` marker when there was none), merges only those two keys, creates the throwaway user `agh-spike` (not an AgentHub member), lists its marketplaces/plugins, opens one interactive `claude` for 60 s in a private tmux socket, lists again, and prints a verdict per plugin. Report: `/var/tmp/agenthub-spike-managed/` (`verdict.txt`, `session-screen.txt`, the before/after JSON).
   - If the screen shows a login prompt, the result before login may not be final: attach with `sudo runuser -u agh-spike -- tmux -L agh-spike attach` during the wait, or rerun after `undo` with a longer wait.
4. Undo (always): `sudo bash scripts/server/spike-managed-plugins.sh undo` — restores the file byte for byte (or removes it), ends `agh-spike`'s processes and deletes the user and its home, then checks install-system's mark (`/etc/agenthub/managed-settings.sha256`) matches again. Check step 2's hash matches.
5. Record here: Claude Code version, `verdict.txt`, and whether plugins appeared before a session, only after one, or never. "Listed after" ⇒ managed settings are enough, and install-system can write the baseline there (the fallback stays for plugins managed settings cannot express). "Absent" ⇒ keep the fallback.

### P7. Codex pool
- [x] `codex-pool.js`, modelled on `claude-pool.js`, with two kinds of account:
  - `apikey`: any concurrency, the key in env;
  - `chatgpt`: `auth.json`, and ONE live pane per account, because refresh tokens rotate. It is copied into the member's `CODEX_HOME` at spawn and copied back under a lock when the pane ends; the newest `last_refresh` wins.
- [x] `pickAccount` for Codex, `dashboard_runs.account`, `agenthub accounts add --tool codex`, and quota from that account's rollouts. Built 2026-10-09: an API key goes `-e AGENTHUB_POOL_KEY` + a `-c` provider (Codex's TUI ignores `CODEX_API_KEY`/`OPENAI_API_KEY`); a ChatGPT `auth.json` is written only into a member's CODEX_HOME (never the host's login, never over the member's own, sticky while a pane of theirs runs on one); copy-back runs on the run-heal beat (newest `last_refresh`). Not yet run against a real account.
- **Spike done 2026-10-08** (codex 0.161.0, a free ChatGPT account): two `CODEX_HOME` copies of one `auth.json` ran at the same time; a refresh in one copy (forced by an expired `access_token` — 0.161 ignores `last_refresh`) rotated its refresh token, and the OLD refresh token still refreshed in a second and a third copy minutes later; every copy kept working and nothing was logged out. So a ChatGPT account may run in several panes at once: copy `auth.json` per pane, copy the newest back when a pane ends. Unverified: whether reuse of an old refresh token is still accepted days later (keep the copy-back so a new pane starts from the newest).

### P8. Security review and docs
Review ran 2026-10-08 (four lenses: root/sudo, cross-member, secrets, local-mode). Root cause of every serious finding: the dashboard (host uid, `CAP_DAC_READ_SEARCH`) connects to, writes into and reads from paths inside a member's home, which the member can swap. Fixes:
- [x] F1 tmux: every call on a member's socket runs AS the member (`sudo -u`), attach included; a session's env + argv go in a launch file written by `write-as` (stdin) and run by `/opt/agenthub/libexec/pane-launch` — nothing secret or private crosses sudo; no boot session, no `server-access`, no socket chmod (drop `chmod` from sudoers).
- [x] F2 the host never writes, chmods or deletes inside a member's home: handoff files, drops, context copies and git scratch all go through `write-as` (new modes: `unlink`, `mkdir`).
- [x] F3 a member's file is read only after `open` + `fstat` says it is a regular file owned by that member (O_NONBLOCK: a FIFO cannot hang the event loop) — transcripts, subagents, Codex/agy/Cursor journals, the lane file reads.
- [x] F4 provision: root never touches a path inside an existing home; `.agenthub` is made by the member (`runuser`), 0700.
- [x] F5 the ambient capability is dropped for every child the dashboard spawns (`setpriv`), so host panes, git hooks and npm do not inherit read-everything.
- [x] F6 `accountForPath` fails closed for a path under the users root with no usable account (`EMEMBERGONE`), never "run it as the host".
- [x] F7 small: `buildArgv` gets `laneId` (ENOTSHARED never fired); peerUid compares loopback addresses; `testProxy` `redirect: "manual"`; `sudoArgv` key check; `startSessionSync` catches `allProjectsDirs`; `tmux.__reset` resets `memberImpl`; `accountForLane` checks `s3Enabled` first; run.js maps ENOTSHARED/EMEMBERGONE 409; absolute `git`/`tmux`/`gh` paths from server.conf in sudo argv. NOT done: commit message via `-F -` — the sudo log is root/adm-readable and root reads the repo anyway (documented ceiling).
- [x] F8 member hooks: the transcript path is checked before the Codex mapping reads it; a member hook touches only a lane inside its own home (status + detection); Claude/Codex/agy/Cursor journals speak only for their own account (`sameOwner`: cwd + existing row; a Codex thread id never takes over another account's row). `livenessReap`: kept — a member session no pane holds already falls through to the idle gate (documented).
- [x] F9 install-system: stops a detached dashboard before the unit starts; keeps env-file keys the bootstrap did not pass (`DASHBOARD_PORT`); `--uninstall` rewrites the user unit before enabling it; `chown -R -h` / `cp --no-dereference`; `chmod o-rwx` the host's `~/.claude`; UI text for `systemd-system`.
- [x] F10 shared tools: never copy a tool folder that is a direct child of `$HOME`, and skip files matching `CONFIG_SECRET_RE`.
- [x] F11 Claude via the gateway: the pane token goes in a per-run `--settings` file, so a user/project `env.ANTHROPIC_*` cannot override it.
- Accepted ceilings (document): a host-account compromise is every member's (it may sudo as any member); install-system trusts the host's tree; members may make the host POST to any URL through a personal proxy (they have a shell on the box anyway); the gateway forwards any `/v1/` path; sudo logs branch names, PR titles and paths.
- [x] A multi-lens adversarial review of P2–P7 before merging (2026-10-09, lenses root/sudo, cross-member, secrets, local-mode; 12 confirmed findings, each fixed with a test in `server/__tests__/server-s3-review2.test.js`, `drop-caps.test.js`, `server-mode-proxies.test.js`):
  - F12 lane file read/save: the workspace's real path must stay the same account's (`git-diff.js` `laneRoot`, `EFOREIGN`), so a folder swapped for a link never picks the account a read or `write-as` runs as;
  - F13 `excludeFromGit`/`excludeFromRepoRoot`: the git dir the member's git prints must realpath into the same account's files; read with `readFileSafe` (no FIFO hang), never a host write for a member;
  - F14 node-pty `tmux attach` goes through `setpriv` (`drop-caps.ptyArgv`);
  - F15 `GET /lanes/browse` and `GET /lanes/branches`: a developer stays in their own folder (real path), `EFOLDER`;
  - F16 a by-id transcript/subagent scan looks only in the session owner's projects dir (`claude-home.sessionProjectsDirs`);
  - F17 Codex status mirror: a journal flips only its own account's lanes (`sameOwner`), `MAX_FILES` per home;
  - F18 a member's home is no workspace under S3 (`folderAllowed`), and the dot folders at the top of a member's home are `ESECRET` for every workspace file view (`inMemberDotDir`);
  - F19 reads use `ownerForRead`, never the throwing `accountForPath`: `projectsDirFor`, the liveness reaper, `cc-reload`, the Codex pool's live check;
  - F20 an AI Proxy edit rewrites a live gateway pane's own `--settings` file (its own token) and deletes a dead pane's.
  - Not done (optional in the finding): deleting a run's handoff `proxy-*.json` when the run ends — its token dies with the run.
- [x] A third review (2026-10-09; 10 confirmed findings, each fixed with a test in `server/__tests__/server-s3-review3.test.js`, `codex-pool.test.js`, `server-s3-system.test.js`):
  - F21 a workspace inside a member's home is owned by that member only (`run-as.assertOwnerFits`: `createLane`, `assignWorkspace`, `spawnRun`, `spawnShell` — `EOWNERFOLDER`), so no pane carries another owner's token;
  - F22 the transcript snapshot copies through `openSafe` (a member's planted link is never copied into the host's data dir);
  - F23 a member's long first prompt stays in its launch file (no `run-prompts` file the member cannot read);
  - F24 disable ends the logins first and locks only a Linux account that exists; F25 `provision --all` goes on past a failing user;
  - F26 the Codex settle pass names a session from its own home's index;
  - F27 the workspace memory view reads through `openSafe`;
  - F28 a non-admin's `?cwd=` on `/cc-config` and `/codex-config` names only their own workspaces (`scope.ownCwdQuery`), and the config `file` reads never serve a credential file (`isSecretConfigPath`);
  - F29 the Codex pool copies a refresh back only from a home it placed the account in, never dated in the future, with the same `id_token` identity (signature unchecked: documented ceiling);
  - F30 the startup AGENTS.md cleanup skips members' folders.
- [x] Review round 5 (2026-10-09; 5 confirmed findings, each fixed with a test in `server/__tests__/server-s3-review5.test.js`, `cursor-usage.test.js`, `run.test.js`, `codex-pool.test.js`):
  - F31 the Cursor slug walk (`cwdFromSlug`) follows no symlink and gives up after 500 directories, once per transcript — a member's crafted tree or link loop no longer freezes the dashboard;
  - F32 a member's Antigravity conversation is read from a private copy (`readFileSafe` of the `.db` and its `-wal`), so a FIFO `-journal`/`-wal` beside it, which SQLite opens by name, blocks nothing;
  - F33 a non-member's drops go to `<data dir>/drops/` (0700), never a predictable `/tmp/agenthub-drops-<uid>` another account could create first;
  - F34 a developer's extra workspace folder is checked against their own folder (`scope.folderAllowed`, real path) before anything probes it, and again on the path it records (`403 EFOLDER`);
  - F35 the Codex pool's quota reads a run's rollout only from the `CODEX_HOME` of the account its workspace runs as, never one dated in the future. Accepted ceiling (documented): a member can still forge their own session's reading.
  - F36 (critical) `GET /api/cc-config/file` never serves the data dir (`isSecretConfigPath`): an analyst read an admin gateway pane's live pane token from `proxy-settings/`, plus `vapid-keys.json` and the database;
  - F37 every tmux call on a member's socket is bounded (`memberCall`: 3 s, SIGKILL, 1 MiB) with a 60 s back-off — a SIGSTOPped member server froze the event loop for good;
  - F38 a failing member server makes only that member's panes unknown (`liveSessions().has` true for them), never null for everyone; member list errors are cached;
  - F39 `execIn` and `write-as` (every sync sudo into a member) stop after 5 s (`MEMBER_EXEC`) — a FIFO `include.path` or a FIFO write target hung git and write-as;
  - F40 transcripts and journals are read with `readLines` (chunked, a line over 64 MiB is skipped, never built) — readline crashed the server on a one-line 512 MB sparse file (import-history, sessions, codex-usage);
  - F41 `readFileSafe*` read a member's file only up to a cap (`max`; 1 MiB logins/settings, 16 MiB indexes/journals, 256 MiB SQLite copies, 64 MiB default), and transcript-cache skips a member transcript over 256 MiB;
  - F42 the transcript snapshot skips a member file over 256 MiB and writes zero runs as holes — a sparse file was copied out as real blocks into the host's disk;
  - F43 an async route's rejection goes to `next(err)` (Express `Layer.handle_request`), and `refuseS3Folder` uses `ownerForRead` — `POST /api/lanes/worktree` with a gone member's folder crashed the server;
  - F44 a workspace outside the users root may be owned only by an admin (live role: `assertOwnerFits` at create, assign and every spawn), and demoting/disabling a user ends their host panes — a developer-owned host folder started host-account panes;
  - F45 a pane's cwd must be its workspace's account's (`ECWDACCOUNT`, checked before anything is written for the pane);
  - F46 a pane token (dashboard and AI Proxy gateway) is accepted only from the uid its pane runs as (`peerIsPaneAccount`);
  - F47 the Codex pool counts a home's login as placed only when AgentHub placed it there AND the id_token's email/account match — a ChatGPT Team's seats share `account_id`; the live-pane stickiness follows the live run;
  - F48 a path an analyst names in a GET (`browse`, `branches`, `/run/files`) is scoped like a developer's (`ownFolder`, `insideOwnWorkspace`);
  - F49 a drop is saved as the account of the workspace the client names (`?laneId=`, or the pane's run id), not the requester's.
- [x] Update (2026-10-09, branch `s3-docs`):
  - `docs/SERVER-MODE.md`: an S3 section and a runbook;
  - CLAUDE.md rules: a pane runs as its owner; the dashboard never uses sudo to root except the provision helper; hooks are authenticated by peer uid; `hidepid` is required;
  - the design plan moved back from the deleted AgentHub-Server repo into [`README.md`](README.md); the operator docs are the runbook in `docs/SERVER-MODE.md`.

## Known risks
- `sudo` costs about 10 ms per git call. That is fine: an open Changes panel makes 5 calls every 30 s.
- The host's tmux client and the member's server must be the same binary, so the path is pinned.
- A member can revoke `server-access` or chmod their socket. Only their own panes go dark, and the UI says so.
- Per-member tmux servers live in the system unit's cgroup. `KillMode=process` keeps them across restarts (the rule stays), and Quit kills them explicitly.
- Existing usernames that do not fit the Linux rule (uppercase, dots) are refused with a clear message, and the admin renames them.