feat(server-s3): one Linux user per account (S3) — experimental

Server mode can now run each account as its own Linux user (ah-<name>)
when `agenthub server install-system` has set S3 up. The dashboard
reaches a member only through sudo -u / write-as / pane-launch, reads a
member's files only through owner- and type-checked opens, and hooks
are authenticated by peer uid. Local mode and server mode without S3
are unchanged.

EXPERIMENTAL: tested with fake sudo and fake accounts only. It has not
yet been verified on a real install with real member accounts, sudo or
a real Codex login. Do not offer it to real users yet.

- S3 core: provisioning, per-member tmux servers, the proxy key
  gateway, shared tools in /opt, dropped capabilities for children
  (P0–P5)
- P4: plan and quota per member home
- P6: ~/.claude.json seed; opt-in plugin baseline installed as the
  member; managed-settings spike script (not run)
- P7: Codex account pool (`agenthub accounts add --tool codex`)
- Roles merged with main's admin / analyst / developer: analysts write
  only their own rows, under S3 as well
- Per-account prefs: one implementation (Chat UI + onboarding)
- Security reviews F1–F49 (five rounds), each fix with a regression
  test: symlink/TOCTOU-safe member reads, no host writes in a member
  home, credentials and the data dir never served, owner and folder
  account must agree, pane tokens bound to the pane's uid, bounded
  sudo/tmux/git calls and member reads (no hang or crash from a member),
  async route rejections answer 500 instead of exiting
- Docs: docs/SERVER-MODE.md (S3 model, accepted ceilings, operator
  runbook), docs/plans/server-mode/S3-PLAN.md, CLAUDE.md S3 rules

Tests: server 2838/2841 (3 need 127.0.0.1:3000, held on this box);
client 1162/1162; tsc clean; headers check passes.

Co-Authored-By: Claude <noreply@anthropic.com>

docs/plans/onboarding/README.md

@@ -0,0 +1,45 @@
+# First-run setup and tour (per account)
+
+Asked 2026-10-08 (modelled on Orca's 3-step welcome). Decisions:
+
+- **Bypass is never remembered** (CLAUDE.md rule). The wizard remembers a default *permission mode* (Auto / Accept edits / Ask / Plan); bypass stays the red per-session switch.
+- **Per account, on the server.** The choices follow the account to any browser. Local mode is one account, `local`.
+- **The tutorial is a step-by-step spotlight tour.**
+
+## Server
+
+- [x] The `user_prefs` table: `user_key` (`u<id>`, or `local`), `prefs` (JSON), `onboarded_at`, `updated_at`. It is migration-safe (`CREATE IF NOT EXISTS`).
+- [x] `server/routes/me.js`, mounted at `/api/me`:
+  - `GET /prefs` returns `{prefs, onboardedAt}`;
+  - `PUT /prefs` merges `{prefs}`, takes only keys in `PREF_KEYS`, string values of at most 8 KB each, and `null` deletes a key;
+  - `POST /onboarded` marks the account as onboarded.
+  - Writes are same-origin. A member may write their own prefs, and nobody else's.
+- [x] S3: when a member saves a mode, the theme in their Claude Code `~/.claude/settings.json` follows it (dark or light), written through `write-as`.
+- [x] P6: `agenthub-provision create` seeds the member's `~/.claude.json` with `hasCompletedOnboarding` and trust for the home folder. It runs as the member (`runuser` + the shared node). Verified: trusting the home also covers folders under it.
+
+## Client
+
+- [x] `lib/userPrefs.ts`:
+  - **On login:** hydrate localStorage from the server, then re-apply the theme, language and pet.
+  - **On save:** wrapping `localStorage.setItem` for `PREF_KEYS` pushes each change (debounced).
+- [x] New preference keys:
+  - `agenthub.defaultTool`, read by AddLaneModal before the install default;
+  - `agenthub.permissionMode`, read by WorkspaceHome. `bypassPermissions` is never accepted here.
+- [x] `components/onboarding/OnboardingWizard.tsx`, mounted in Layout. It shows when the account is not onboarded AND has no workspace (an account that already has one is marked onboarded silently). Steps:
+  1. Default agent (detected tools as a grid, plus "show more") and the default permission mode.
+  2. Appearance: System, Dark and Light as preview cards, a theme grid for dark and for light below them, then language and terminal font.
+  3. Pet: none or an installed pet, and its size.
+  4. Notifications: enable, sound and preview, send a test.
+
+  The footer has Skip, Back and Continue (Ctrl+Enter). The last step's button is "Add your first workspace", which starts the tour.
+- [x] `components/onboarding/Tour.tsx`: a spotlight on `data-tour` targets. Each step moves on when its condition holds, and the step reached is saved in prefs (`agenthub.tour`). The steps:
+  - Add workspace
+  - the folder field
+  - Create
+  - the prompt box
+  - sending the prompt
+  - the session pane
+  - done
+- [x] "Run setup again" lives in Settings → Appearance (every account can open it; System is not a member's page).
+- [x] i18n in English and Vietnamese. Tests: the server route; the client wizard, tour and prefs; snapshots.
+- [x] Docs: `docs/API.md`, `docs/SERVER-MODE.md` and the CLAUDE.md marker.

docs/plans/server-mode/CHECKLIST.md

@@ -18,12 +18,12 @@ Goal (2026-10-07): finish the server part — S2 ownership as proposed, review i
 Decided: own Linux user per account; skills/plugins = admin baseline + user's own; AI Proxy = user's own + admin's shared (no key shown); tool binaries shared under `/opt/agenthub/tools`; Claude pool shared (exists); NEW Codex multi-account pool. Spike allowed on this machine with sudo (undo log: `/tmp/agh-spike/UNDO.sh`).
 
 - [x] S0 spike (2026-10-08, this machine; test users agh-test1/2 kept for S3 dev, undo `/tmp/agh-spike/UNDO.sh`) → results below
-- [ ] Per-user AI Proxy store (+ admin shared)
-- [ ] S3 provisioning + spawn/attach as the account
-- [ ] S4 shared tools in /opt
-- [ ] Per-user skills/plugins (cc-config per account HOME, admin baseline)
-- [ ] Codex account pool
-- [ ] Security review, docs, suites
+- [x] Per-user AI Proxy store (+ admin shared) — S3-PLAN P1/P1b
+- [x] S3 provisioning + spawn/attach as the account — S3-PLAN P2/P3
+- [x] S4 shared tools in /opt — S3-PLAN P5
+- [ ] Per-user skills/plugins (cc-config per account HOME, admin baseline) — seed + opt-in baseline built (P6); managed-settings spike not run, member Settings not built
+- [x] Codex account pool — S3-PLAN P7 (fake accounts only; no real Codex login tried)
+- [x] Security review, docs, suites — P8 F1–F11, second review F12–F20 (2026-10-09), docs/SERVER-MODE.md runbook
 
 ### S0 results (claude 2.1.293, codex 0.155.1, tmux 3.7c, node 26)
 

docs/plans/server-mode/S3-PLAN.md

@@ -0,0 +1,293 @@
+# S3 plan: one Linux user per account (2026-10-08, after S0 and a 4-way code review)
+
+Scope (user, 2026-10-08):
+- own workspaces, sessions and stats;
+- one Linux user per account;
+- own skills and plugins: an admin baseline, plus the member's own;
+- own AI Proxy, with the admin's shared ones usable by everyone;
+- one shared set of tool binaries;
+- the shared Claude pool;
+- a new Codex pool.
+
+The S0 results and test users are recorded in `CHECKLIST.md`.
+
+## Decisions 2026-10-08 (after the review)
+
+- `hidepid=invisible` is required: `install-system` refuses without it.
+- **Hide shared proxy keys** (new phase P1b): AgentHub becomes a loopback endpoint for AI Proxy profiles.
+  - The pane gets `ANTHROPIC_BASE_URL`/`base_url` = `http://127.0.0.1:<port>/api/proxy-gw/<profile>` and a per-pane token as its key.
+  - AgentHub checks the token (pane alive, the lane owner may use the profile), swaps in the real key, and streams the request to the profile's `baseUrl`.
+  - Pool OAuth tokens are NOT covered: that ceiling stays documented.
+- **Codex ChatGPT accounts may be used concurrently** (user's choice, against the recommendation). Refresh tokens rotate, so P7 starts with a spike on a real spare account. The design to verify:
+  - AgentHub owns the refresh: one canonical `auth.json` per account in the pool;
+  - it refreshes before expiry under a lock and rewrites every live copy;
+  - it checks whether codex re-reads `auth.json` and what happens when two copies refresh.
+  - If codex refreshes on its own and logs the others out, report back to the user before building.
+- Members do NOT get (a) the Add-workspace **Worktree** mode (managed worktrees) or (b) the runtime layer. They use **Repo** mode plus the prompt box's **New worktree** (session worktrees). Both refusals are `EMEMBERLANE`, and admins keep both on ownerless workspaces.
+
+## Architecture
+
+**Identity**
+- AgentHub user `alice` maps to Linux user `ah-alice`. The prefix cannot clash with a system account or the host account.
+- Her home is `$AGENTHUB_USERS_ROOT/alice`, which is today's `folderAllowed` rule; in server mode it becomes mandatory.
+- Admins get a Linux user too.
+- Ownerless (pre-S3) workspaces stay host-run and staff-only.
+
+**Dashboard process.** In server mode only, it runs as a system unit:
+- `User=<host>`
+- `SupplementaryGroups=agenthub`
+- `AmbientCapabilities=CAP_DAC_READ_SEARCH`
+- `InaccessiblePaths=-/etc/shadow -/etc/gshadow -/etc/sudoers -/etc/sudoers.d -/root`
+- `Restart=always`
+- `KillMode=process`
+
+It must **not** set `NoNewPrivileges` or `CapabilityBoundingSet`, because both block `sudo`. `/home` stays visible, because the host's DB lives there.
+
+**Three privileges, each narrow:**
+1. **Read anything.** `CAP_DAC_READ_SEARCH` covers every tool and every file mode, with no per-read cost and no per-tool code. It does NOT cover `/proc/<pid>/cwd` of another uid; that needs CAP_SYS_PTRACE, which is root-equivalent, so liveness changes instead (see P3).
+2. **Act as a member.** sudoers: `<host> ALL=(%agenthub-members) NOPASSWD: <tmux>, <git>, <gh>, /opt/agenthub/libexec/write-as`. It never allows root.
+3. **Provision.** sudoers: `<host> ALL=(root) NOPASSWD: /opt/agenthub/sbin/agenthub-provision`, a fixed script with a strict name check.
+
+**Panes**
+- Each account has its own tmux server, with its socket at `<home>/.agenthub/tmux/sock`.
+  - The socket's dir is setgid `agenthub`, and the socket is 0770 with `server-access -a <host>`.
+  - S0 showed that tmux keeps group rw across attach, and that another member is refused.
+- The server is started once via `sudo -u ah-x tmux -S … new-session`. After that every command is a plain host-side `tmux -S` client (no sudo): send-keys, respawn, paste, kill, attach.
+- Pane env:
+  - `HOME`
+  - `PATH=/opt/agenthub/tools/bin:…`
+  - `CODEX_HOME=<home>/.codex`
+  - `CLAUDE_DASHBOARD_PORT`
+  - `DISABLE_AUTOUPDATER=1`
+  - none of the host's `XDG_RUNTIME_DIR`/`DISPLAY`.
+
+**Hooks**
+- A hook is authenticated by the peer uid, read from `/proc/net/tcp`. S0 confirmed this: a connection from `agh-test1` mapped to uid 1002.
+- That uid's account may only touch its own sessions, runs and lanes, and `transcript_path` must lie under that account's home. Today any local process can forge any hook.
+- Claude hooks are installed via `/etc/claude-code/managed-settings.json` and call `node /opt/agenthub/app/scripts/hook-handler.js`. The Codex and agy forwarders are written into each member's home at provisioning.
+
+**Secrets in argv**
+- `/proc` on this host is not `hidepid`. Every member's `ps` therefore sees what the dashboard passes as `tmux -e TOKEN=…` and `sudo … env K=V`.
+- Mount `/proc` with `hidepid=invisible`, plus a `gid=` exemption for the host. `install-system` refuses to continue without it.
+
+## Not offered for member-owned workspaces in S3 v1
+
+These answer 403 `EMEMBERLANE` in server mode:
+- managed worktrees (`POST /api/lanes/worktree`). Members use session worktrees instead (`claude --worktree`, made by Claude as the member).
+- the runtime layer: lanes up, down, hook, profile, slots, proof, lane-mcp, env seed, merge driver and gc. These spawn as the host, write into `LANES_ROOT` under the host's home, and kill and probe pids.
+
+Boot-time cleanups (`clearLaneProxies`, `removeStageDoc`) skip member lanes.
+
+## Phases (order: P0 → P1 → P1b → P2 → P3 → P4 → P5 → P8 review → P6 → P7)
+
+Branch: `server-s3`. Commits happen only when the user asks.
+
+### P0. Leaks that exist today (no sudo; can ship at any time)
+- [x] Make `GET /api/ai-tools/:id/config/:index` staff-only. Today a member can read the host's catalog config files.
+- [x] Members' `/cc-config/commands|skills`: `?cwd=` must lie inside one of their own workspaces. Today they can list another user's skills.
+- [x] (moved to P3) `cc-reload`: before S3 every pane shares the host's `~/.claude`, so reloading member panes after a host plugin change is CORRECT today. Restrict it to panes on the changed home only once panes have their own (P3).
+- Verify: one server-mode test per leak.
+
+### P1. Per-user AI Proxy (no sudo; also useful without S3)
+- [x] Store layout:
+  - shared = today's `agenthub-proxies/`, writable by admins only;
+  - personal = `agenthub-proxies/users/<userId>/`, writable by its owner;
+  - a user's names are unique across both (`freeName` checks the shared ones).
+- [x] API: the list stays an array and each item gets `scope: "shared"|"personal"`. `/ai-proxies` leaves `ADMIN_WRITE_PREFIXES`, and route checks take its place. A member may test only their own profiles; the stored-key fallback still requires the saved `baseUrl`.
+- [x] `resolveProxy(tool, name, ownerId)` looks in personal, then shared. `dashboard_runs.proxy` keeps the name, and a resume resolves with the lane owner's id.
+- [x] Client: Settings shows two groups, and the picker lists both.
+- [x] Docs: a shared key is hidden from the browser only; a pane can read it (the same ceiling as the pool).
+- Done 2026-10-08: server-mode-proxies.test.js (11), AiProxyConfig tests (+4); full suites server 2657/2660 (3 = lane-detect, port 3000 held by a foreign next-server), client 1135/1135.
+- Verify: server tests (A cannot see, use or test B's; a member cannot write shared; an admin can), client tests, a snapshot diff review.
+
+### P1b. Hide shared proxy keys (no sudo)
+- [x] `/api/proxy-gw/:profile/*`:
+  - loopback only;
+  - a per-pane token (hash in `dashboard_runs`, alive pane only), and the run's lane owner may use the profile;
+  - streams body and SSE both ways and swaps `x-api-key`/`Authorization`;
+  - never logs a key;
+  - has a timeout.
+- [x] `proxyForSpawn` points Claude/Codex at the gateway, never at the real key, for shared profiles (and for personal ones too: one code path).
+- Done 2026-10-08: server-mode-proxy-gateway.test.js (11); verified the real CLIs' requests: claude 2.1.293 → `<base>/v1/messages?beta=true` with `Authorization: Bearer <ANTHROPIC_AUTH_TOKEN>`, codex 0.155.1 → `<base_url>/responses` with `Bearer <env_key>`. Server mode only; local mode unchanged.
+- Verify: an e2e against a fake upstream (header swapped, streaming kept, a dead pane's token refused, another member's profile refused).
+
+### P2. Host setup (sudo; on this machine with the agh-test users)
+- [x] `sudo agenthub server install-system` is idempotent, has `--uninstall`, and does the following:
+  - creates groups `agenthub` and `agenthub-members`;
+  - creates `/srv/agenthub/users` (0751) and `/opt/agenthub/{tools,app,libexec,sbin}`;
+  - writes the sudoers file and checks it with `visudo -c`;
+  - writes the system unit and the managed-settings hooks;
+  - checks `hidepid`.
+- [x] `service-manager` gets a system variant used when server mode is on and the system unit exists; stop and restart go through sudoers `systemctl … agenthub`.
+- [x] `plugin-bootstrap`: in server mode it never starts, stops or repoints the server (deps only). Under any uid other than the host it does nothing (otherwise a member's SessionStart would spawn a second dashboard).
+- [x] Provision helper (root, bash):
+  - checks the name against `^[a-z][a-z0-9_-]{0,28}$`, makes the Linux name `ah-<name>`, and refuses an existing account;
+  - create: `useradd`, the home, `.agenthub/{tmux,handoff}`, the Codex/agy hook forwarders, the baseline settings;
+  - disable: `usermod -L`, then end that account's tmux server;
+  - never deletes.
+- [x] `agenthub users add` and the Users page call it; `users.linux_user` is set.
+- Done 2026-10-08 and INSTALLED on this machine (system unit runs the server-s3 worktree on :4820; ah-admin, ah-vinnt provisioned; undo: /tmp/agh-spike/UNDO.sh step 0). Findings: `InaccessiblePaths=/etc/shadow` breaks sudo (PAM) — only /root is hidden; a host with its own `(ALL) NOPASSWD: ALL` makes the sudoers allowlist moot (keep the host out of `sudo` on a real server); `usermod --unlock` refuses a passwordless account (enable = clear the expiry); the test runner must not see the host's install (`AGENTHUB_SYSTEM_UNIT_DIR`, `AGENTHUB_SERVER_CONF` in scripts/test-server.js). Tests: server-s3-system.test.js (14); full suite 2679/2682 (3 = port 3000).
+- Verify: on this machine, `agenthub users add alice` creates `ah-alice`; running it again changes nothing.
+
+### P3. Spawning and acting as the member
+- [x] `server/lib/run-as.js`:
+  - `accountForLane` / `accountForCwd` return `{linuxUser, home, uid}`, or null (null = the host, which is always the case in local mode);
+  - `sudoArgv(acct, cmd, args, env)`.
+- [x] `tmux.js`:
+  - one socket per account, chosen from session name → lane → owner;
+  - start via sudo and add `server-access`;
+  - `listSessions`, `liveSessions` and `serverPid` loop over every socket, with per-socket caches.
+  - Also: `pty-attach` gets `-S`; same change in `cc-update-all-helper` and `lanes shell`.
+- [x] `worktree.js` `git()`: member lanes go through `sudo -u`. The same for `git-exclude` and `gh`. In `git-ops`, the patch goes via stdin and the temp index under the member's tmp.
+- [x] fs writes in member lanes go through `/opt/agenthub/libexec/write-as`: `lane-file-write`, the `.gitignore` append. Callers keep their existing guards.
+- [x] Pane handoff files go to `<home>/.agenthub/handoff/` (member-owned, setgid `agenthub` 2770; the host writes 0644). These are:
+  - proxy settings;
+  - the Codex proxy (`-c` flags, never a file in the member's `CODEX_HOME`);
+  - workspace context and prompt files;
+  - drop uploads.
+- [x] `cc-reload`: only panes whose Claude home is the one that changed (host-run panes for a host plugin update).
+- [x] Liveness: a member session is alive iff its pane's tmux session is live. No `/proc/<pid>/cwd` read; without this, live member sessions get reaped.
+- [x] Hooks: peer-uid auth, `transcript_path` under the account's home, and `threadOf` and the agy check accept member homes.
+- [x] Member-lane refusals (`EMEMBERLANE`).
+- Done 2026-10-08, verified LIVE on this machine as member `s3probe` (ah-s3probe):
+  - a shell pane and a Claude pane run as the member; attach over /ws-pty works;
+  - stage, commit and file save through the API run as the member, and the file stays the member's;
+  - Claude → `/api/proxy-gw` → a fake upstream received the REAL key, and the pane showed "pong";
+  - member-hook hooks were accepted by peer uid and bound to the workspace;
+  - the 0600 transcript was served.
+  Found on the way:
+  - sudo LOGS its argv to the journal, so no secret may cross sudo: the member's tmux server starts with a secret-free `agenthub-boot` session, and the real one is created as a client;
+  - tmux gives a pane created by an unattached client THAT CLIENT's PATH, so member-socket calls carry MEMBER_PATH;
+  - a member's first Claude run asks for theme, trust and bypass consent → P6 seeds onboarding + trust of the home (bypass consent stays the member's);
+  - git commit messages appear in the sudo journal (not secret; privacy note for P8);
+  - an OLD dashboard on the same DB (the 4830 dev server) abandons member runs, because it cannot see their tmux.
+- Verify:
+  - unit tests with fakes: argv builders, socket choice, the peer-uid parser on fixture text;
+  - an opt-in integration script (`AGENTHUB_S3_IT=1`, real test users): start a pane, commit, save a file, ingest a hook;
+  - a browser e2e as two members.
+
+### P4. Reads across homes
+- [x] `homes()` = the host plus each enabled user's home (`run-as.memberHomes`). Done per home: Claude transcript resolvers (`claude-home.projectsDirFor`/`allProjectsDirs`), session sync, codex-usage, codex-ingest, codex threadOf, agy hook path. Moved to P5 (no member can run agy/Cursor before /opt): antigravity and cursor pollers. Per-member quota readers: P5. Legacy-import marker: not needed (the 30 s sync sweeps member dirs). Originally per home:
+  - Claude session sync (watchers plus the 30 s poll);
+  - the codex-usage, codex-ingest, antigravity and cursor pollers (`running` keyed by home);
+  - the legacy-import marker.
+- [x] Transcript routes: fixed at the resolvers instead (cwd → the member's projects dir), which covers every caller. Verified live: the member's 0600 transcript was served.
+- [x] Quota readers per member home: Claude, Codex, Cursor, Grok, Kimi, OpenCode. agy's live `/usage` stays host-only, because it needs the agy binary, which updates itself.
+  Done 2026-10-09: `quotas({home})` (`server/lib/quota.js`) reads each tool at its default path under the member's home through `run-as`'s owner-checked reads (`opencode.db` + WAL from a private copy); `GET /api/pricing/quota` gives a developer/analyst their own (`accountForUser`), nothing when unprovisioned, the host's to an admin. No member Claude profiles (P6). Test: `server-s3-quota.test.js`. Not yet checked live on the test users.
+- Verify: tests with two homes in a tmp dir, then a real check on the test users.
+
+### P5. Shared tools in /opt (S4)
+- [x] `/opt/agenthub/tools` is owned by the host: the dashboard updates it, members get r-x.
+- [x] Publish step after an install or update (and once at `install-system`):
+  - copy the versioned binary to `tools/<id>/<version>` and flip `bin/<id>`;
+  - npm tools use `--prefix /opt/agenthub/tools/npm`.
+- [x] For members, publish the running build (scripts, server, bin, mcp, statusline, production deps; no client) to `/opt/agenthub/app/<version>` at server start in server mode. Members need it for hooks, the CLI, MCP and the statusline.
+- [x] The pane `PATH` puts `/opt/agenthub/tools/bin` first, and the catalog detects tools there.
+- Done 2026-10-08 (`server/lib/shared-tools.js`, `agenthub server tools`):
+  - a publish copies an ELF file, an npm package folder, or a script's own version folder; a shared `bin` folder is refused;
+  - publishing runs after each install/update job, in the background at server start, and on the CLI command;
+  - a member's catalog-tool pane runs `<root>/bin/<command>`, or gets 409 ENOTSHARED;
+  - the app scripts are published with the `agenthub` launcher;
+  - the agy and cursor pollers also read member homes.
+  Live: claude, codex, agy and cursor-agent published (1.6 GB); claude, codex, cursor-agent and agenthub run as `ah-s3probe` (agy was not run: it self-updates).
+  Follow-up: per-member quota readers. A member's Claude runs on the shared pool, and the quota route already masks accounts for members.
+- Verify: after an update from the UI, a member pane sees the new version.
+
+### P6. Skills and plugins
+- [x] Seed a new member's `~/.claude.json`: `hasCompletedOnboarding`, theme, and trust for their home (live finding 2026-10-08). The Bypass Permissions consent stays the member's own.
+  - Done: `agenthub-provision create` seeds onboarding + trust of the home (merged; a non-JSON file is left alone); the theme follows AgentHub's dark/light mode (`member-claude.setTheme`). Test: `server-s3-provision.test.js` runs the real script with stub `id`/`useradd`/`chown`/`runuser` and its fixed paths pointed at a temp dir.
+- [ ] Spike: does `/etc/claude-code/managed-settings.json` (`enabledPlugins` plus marketplaces) give a new Linux user the admin's baseline plugins? (Script ready, NOT run — see "P6 spike: how to run it" below.)
+- [x] Fallback (built, opt-in): with `/etc/agenthub/plugin-baseline.json` (`{"marketplaces": ["owner/repo", …], "plugins": ["name@marketplace", …]}`, root's, the admin edits it), creating a user runs `claude plugin marketplace add <source>` for each marketplace, then `claude plugin install --scope user <plugin>` for each plugin, AS the new member (`sudo -n -u ah-<name> HOME=… /opt/agenthub/tools/bin/claude …`, in the background; `agenthub users provision` runs it too, in the foreground, and is the retry). No file = nothing runs. An entry starting with `-` is skipped; `-y` is never passed. Sudoers: `/opt/agenthub/tools/bin/claude` joins the run-as-member list (no new power: the host already runs any command as a member through tmux). `server/lib/member-claude.js`, test `server-s3-provision.test.js` (fake sudo). Not verified live.
+- [ ] A member's own plugins work from their terminal (`/plugin`) as soon as P3 lands, with no code.
+- [ ] Later, optional: Settings for members, using `cc-config` with `home` = `req.user`'s and writes via `sudo -u`.
+
+#### P6 spike: how to run it (a human, on the S3 box)
+`scripts/server/spike-managed-plugins.sh` (never run by AgentHub). It changes `/etc/claude-code/managed-settings.json`, which EVERY Claude Code on the machine reads, the host's own sessions included, so run it in a quiet moment and undo right after.
+1. Write the fragment to test, e.g. `/tmp/spike-fragment.json`:
+   ```json
+   {
+     "extraKnownMarketplaces": {
+       "agenthub": { "source": { "source": "github", "repo": "<owner>/agenthub" } }
+     },
+     "enabledPlugins": { "agenthub@agenthub": true }
+   }
+   ```
+2. Note the current file: `sudo sha256sum /etc/claude-code/managed-settings.json`.
+3. Run: `sudo bash scripts/server/spike-managed-plugins.sh run /tmp/spike-fragment.json 60`. It backs the file up once (`managed-settings.json.agenthub-spike-bak`, or a `.agenthub-spike-none` marker when there was none), merges only those two keys, creates the throwaway user `agh-spike` (not an AgentHub member), lists its marketplaces/plugins, opens one interactive `claude` for 60 s in a private tmux socket, lists again, and prints a verdict per plugin. Report: `/var/tmp/agenthub-spike-managed/` (`verdict.txt`, `session-screen.txt`, the before/after JSON).
+   - If the screen shows a login prompt, the result before login may not be final: attach with `sudo runuser -u agh-spike -- tmux -L agh-spike attach` during the wait, or rerun after `undo` with a longer wait.
+4. Undo (always): `sudo bash scripts/server/spike-managed-plugins.sh undo` — restores the file byte for byte (or removes it), ends `agh-spike`'s processes and deletes the user and its home, then checks install-system's mark (`/etc/agenthub/managed-settings.sha256`) matches again. Check step 2's hash matches.
+5. Record here: Claude Code version, `verdict.txt`, and whether plugins appeared before a session, only after one, or never. "Listed after" ⇒ managed settings are enough, and install-system can write the baseline there (the fallback stays for plugins managed settings cannot express). "Absent" ⇒ keep the fallback.
+
+### P7. Codex pool
+- [x] `codex-pool.js`, modelled on `claude-pool.js`, with two kinds of account:
+  - `apikey`: any concurrency, the key in env;
+  - `chatgpt`: `auth.json`, and ONE live pane per account, because refresh tokens rotate. It is copied into the member's `CODEX_HOME` at spawn and copied back under a lock when the pane ends; the newest `last_refresh` wins.
+- [x] `pickAccount` for Codex, `dashboard_runs.account`, `agenthub accounts add --tool codex`, and quota from that account's rollouts. Built 2026-10-09: an API key goes `-e AGENTHUB_POOL_KEY` + a `-c` provider (Codex's TUI ignores `CODEX_API_KEY`/`OPENAI_API_KEY`); a ChatGPT `auth.json` is written only into a member's CODEX_HOME (never the host's login, never over the member's own, sticky while a pane of theirs runs on one); copy-back runs on the run-heal beat (newest `last_refresh`). Not yet run against a real account.
+- **Spike done 2026-10-08** (codex 0.161.0, a free ChatGPT account): two `CODEX_HOME` copies of one `auth.json` ran at the same time; a refresh in one copy (forced by an expired `access_token` — 0.161 ignores `last_refresh`) rotated its refresh token, and the OLD refresh token still refreshed in a second and a third copy minutes later; every copy kept working and nothing was logged out. So a ChatGPT account may run in several panes at once: copy `auth.json` per pane, copy the newest back when a pane ends. Unverified: whether reuse of an old refresh token is still accepted days later (keep the copy-back so a new pane starts from the newest).
+
+### P8. Security review and docs
+Review ran 2026-10-08 (four lenses: root/sudo, cross-member, secrets, local-mode). Root cause of every serious finding: the dashboard (host uid, `CAP_DAC_READ_SEARCH`) connects to, writes into and reads from paths inside a member's home, which the member can swap. Fixes:
+- [x] F1 tmux: every call on a member's socket runs AS the member (`sudo -u`), attach included; a session's env + argv go in a launch file written by `write-as` (stdin) and run by `/opt/agenthub/libexec/pane-launch` — nothing secret or private crosses sudo; no boot session, no `server-access`, no socket chmod (drop `chmod` from sudoers).
+- [x] F2 the host never writes, chmods or deletes inside a member's home: handoff files, drops, context copies and git scratch all go through `write-as` (new modes: `unlink`, `mkdir`).
+- [x] F3 a member's file is read only after `open` + `fstat` says it is a regular file owned by that member (O_NONBLOCK: a FIFO cannot hang the event loop) — transcripts, subagents, Codex/agy/Cursor journals, the lane file reads.
+- [x] F4 provision: root never touches a path inside an existing home; `.agenthub` is made by the member (`runuser`), 0700.
+- [x] F5 the ambient capability is dropped for every child the dashboard spawns (`setpriv`), so host panes, git hooks and npm do not inherit read-everything.
+- [x] F6 `accountForPath` fails closed for a path under the users root with no usable account (`EMEMBERGONE`), never "run it as the host".
+- [x] F7 small: `buildArgv` gets `laneId` (ENOTSHARED never fired); peerUid compares loopback addresses; `testProxy` `redirect: "manual"`; `sudoArgv` key check; `startSessionSync` catches `allProjectsDirs`; `tmux.__reset` resets `memberImpl`; `accountForLane` checks `s3Enabled` first; run.js maps ENOTSHARED/EMEMBERGONE 409; absolute `git`/`tmux`/`gh` paths from server.conf in sudo argv. NOT done: commit message via `-F -` — the sudo log is root/adm-readable and root reads the repo anyway (documented ceiling).
+- [x] F8 member hooks: the transcript path is checked before the Codex mapping reads it; a member hook touches only a lane inside its own home (status + detection); Claude/Codex/agy/Cursor journals speak only for their own account (`sameOwner`: cwd + existing row; a Codex thread id never takes over another account's row). `livenessReap`: kept — a member session no pane holds already falls through to the idle gate (documented).
+- [x] F9 install-system: stops a detached dashboard before the unit starts; keeps env-file keys the bootstrap did not pass (`DASHBOARD_PORT`); `--uninstall` rewrites the user unit before enabling it; `chown -R -h` / `cp --no-dereference`; `chmod o-rwx` the host's `~/.claude`; UI text for `systemd-system`.
+- [x] F10 shared tools: never copy a tool folder that is a direct child of `$HOME`, and skip files matching `CONFIG_SECRET_RE`.
+- [x] F11 Claude via the gateway: the pane token goes in a per-run `--settings` file, so a user/project `env.ANTHROPIC_*` cannot override it.
+- Accepted ceilings (document): a host-account compromise is every member's (it may sudo as any member); install-system trusts the host's tree; members may make the host POST to any URL through a personal proxy (they have a shell on the box anyway); the gateway forwards any `/v1/` path; sudo logs branch names, PR titles and paths.
+- [x] A multi-lens adversarial review of P2–P7 before merging (2026-10-09, lenses root/sudo, cross-member, secrets, local-mode; 12 confirmed findings, each fixed with a test in `server/__tests__/server-s3-review2.test.js`, `drop-caps.test.js`, `server-mode-proxies.test.js`):
+  - F12 lane file read/save: the workspace's real path must stay the same account's (`git-diff.js` `laneRoot`, `EFOREIGN`), so a folder swapped for a link never picks the account a read or `write-as` runs as;
+  - F13 `excludeFromGit`/`excludeFromRepoRoot`: the git dir the member's git prints must realpath into the same account's files; read with `readFileSafe` (no FIFO hang), never a host write for a member;
+  - F14 node-pty `tmux attach` goes through `setpriv` (`drop-caps.ptyArgv`);
+  - F15 `GET /lanes/browse` and `GET /lanes/branches`: a developer stays in their own folder (real path), `EFOLDER`;
+  - F16 a by-id transcript/subagent scan looks only in the session owner's projects dir (`claude-home.sessionProjectsDirs`);
+  - F17 Codex status mirror: a journal flips only its own account's lanes (`sameOwner`), `MAX_FILES` per home;
+  - F18 a member's home is no workspace under S3 (`folderAllowed`), and the dot folders at the top of a member's home are `ESECRET` for every workspace file view (`inMemberDotDir`);
+  - F19 reads use `ownerForRead`, never the throwing `accountForPath`: `projectsDirFor`, the liveness reaper, `cc-reload`, the Codex pool's live check;
+  - F20 an AI Proxy edit rewrites a live gateway pane's own `--settings` file (its own token) and deletes a dead pane's.
+  - Not done (optional in the finding): deleting a run's handoff `proxy-*.json` when the run ends — its token dies with the run.
+- [x] A third review (2026-10-09; 10 confirmed findings, each fixed with a test in `server/__tests__/server-s3-review3.test.js`, `codex-pool.test.js`, `server-s3-system.test.js`):
+  - F21 a workspace inside a member's home is owned by that member only (`run-as.assertOwnerFits`: `createLane`, `assignWorkspace`, `spawnRun`, `spawnShell` — `EOWNERFOLDER`), so no pane carries another owner's token;
+  - F22 the transcript snapshot copies through `openSafe` (a member's planted link is never copied into the host's data dir);
+  - F23 a member's long first prompt stays in its launch file (no `run-prompts` file the member cannot read);
+  - F24 disable ends the logins first and locks only a Linux account that exists; F25 `provision --all` goes on past a failing user;
+  - F26 the Codex settle pass names a session from its own home's index;
+  - F27 the workspace memory view reads through `openSafe`;
+  - F28 a non-admin's `?cwd=` on `/cc-config` and `/codex-config` names only their own workspaces (`scope.ownCwdQuery`), and the config `file` reads never serve a credential file (`isSecretConfigPath`);
+  - F29 the Codex pool copies a refresh back only from a home it placed the account in, never dated in the future, with the same `id_token` identity (signature unchecked: documented ceiling);
+  - F30 the startup AGENTS.md cleanup skips members' folders.
+- [x] Review round 5 (2026-10-09; 5 confirmed findings, each fixed with a test in `server/__tests__/server-s3-review5.test.js`, `cursor-usage.test.js`, `run.test.js`, `codex-pool.test.js`):
+  - F31 the Cursor slug walk (`cwdFromSlug`) follows no symlink and gives up after 500 directories, once per transcript — a member's crafted tree or link loop no longer freezes the dashboard;
+  - F32 a member's Antigravity conversation is read from a private copy (`readFileSafe` of the `.db` and its `-wal`), so a FIFO `-journal`/`-wal` beside it, which SQLite opens by name, blocks nothing;
+  - F33 a non-member's drops go to `<data dir>/drops/` (0700), never a predictable `/tmp/agenthub-drops-<uid>` another account could create first;
+  - F34 a developer's extra workspace folder is checked against their own folder (`scope.folderAllowed`, real path) before anything probes it, and again on the path it records (`403 EFOLDER`);
+  - F35 the Codex pool's quota reads a run's rollout only from the `CODEX_HOME` of the account its workspace runs as, never one dated in the future. Accepted ceiling (documented): a member can still forge their own session's reading.
+  - F36 (critical) `GET /api/cc-config/file` never serves the data dir (`isSecretConfigPath`): an analyst read an admin gateway pane's live pane token from `proxy-settings/`, plus `vapid-keys.json` and the database;
+  - F37 every tmux call on a member's socket is bounded (`memberCall`: 3 s, SIGKILL, 1 MiB) with a 60 s back-off — a SIGSTOPped member server froze the event loop for good;
+  - F38 a failing member server makes only that member's panes unknown (`liveSessions().has` true for them), never null for everyone; member list errors are cached;
+  - F39 `execIn` and `write-as` (every sync sudo into a member) stop after 5 s (`MEMBER_EXEC`) — a FIFO `include.path` or a FIFO write target hung git and write-as;
+  - F40 transcripts and journals are read with `readLines` (chunked, a line over 64 MiB is skipped, never built) — readline crashed the server on a one-line 512 MB sparse file (import-history, sessions, codex-usage);
+  - F41 `readFileSafe*` read a member's file only up to a cap (`max`; 1 MiB logins/settings, 16 MiB indexes/journals, 256 MiB SQLite copies, 64 MiB default), and transcript-cache skips a member transcript over 256 MiB;
+  - F42 the transcript snapshot skips a member file over 256 MiB and writes zero runs as holes — a sparse file was copied out as real blocks into the host's disk;
+  - F43 an async route's rejection goes to `next(err)` (Express `Layer.handle_request`), and `refuseS3Folder` uses `ownerForRead` — `POST /api/lanes/worktree` with a gone member's folder crashed the server;
+  - F44 a workspace outside the users root may be owned only by an admin (live role: `assertOwnerFits` at create, assign and every spawn), and demoting/disabling a user ends their host panes — a developer-owned host folder started host-account panes;
+  - F45 a pane's cwd must be its workspace's account's (`ECWDACCOUNT`, checked before anything is written for the pane);
+  - F46 a pane token (dashboard and AI Proxy gateway) is accepted only from the uid its pane runs as (`peerIsPaneAccount`);
+  - F47 the Codex pool counts a home's login as placed only when AgentHub placed it there AND the id_token's email/account match — a ChatGPT Team's seats share `account_id`; the live-pane stickiness follows the live run;
+  - F48 a path an analyst names in a GET (`browse`, `branches`, `/run/files`) is scoped like a developer's (`ownFolder`, `insideOwnWorkspace`);
+  - F49 a drop is saved as the account of the workspace the client names (`?laneId=`, or the pane's run id), not the requester's.
+- [x] Update (2026-10-09, branch `s3-docs`):
+  - `docs/SERVER-MODE.md`: an S3 section and a runbook;
+  - CLAUDE.md rules: a pane runs as its owner; the dashboard never uses sudo to root except the provision helper; hooks are authenticated by peer uid; `hidepid` is required;
+  - the AgentHub-Server operator docs.
+
+## Known risks
+- `sudo` costs about 10 ms per git call. That is fine: an open Changes panel makes 5 calls every 30 s.
+- The host's tmux client and the member's server must be the same binary, so the path is pinned.
+- A member can revoke `server-access` or chmod their socket. Only their own panes go dark, and the UI says so.
+- Per-member tmux servers live in the system unit's cgroup. `KillMode=process` keeps them across restarts (the rule stays), and Quit kills them explicitly.
+- Existing usernames that do not fit the Linux rule (uppercase, dots) are refused with a clear message, and the admin renames them.