fix(server-s3): list a member's tmux panes asynchronously

A frozen member (SIGSTOPped tmux server) cost every user one synchronous
3 s block of the event loop per minute (F37's back-off ceiling). Member
listing now runs through async execFile (same 3 s / SIGKILL / 1 MiB
bounds, same back-off) into the listed cache the synchronous readers
(liveSessions, listSessions, listOutput) already use: stale-while-
revalidate, 15 s max age, cold or failing = that member's panes UNKNOWN
(F38 unchanged). A session this process creates or kills is written into
the member's cached answer at once, and a list in flight across that
change is dropped. Host tmux calls, local mode and every click's call
(new-session, send-keys, kill, has-session) are unchanged.

review5's stalled-member test now polls for 4 s past the timeout and
asserts each poll < 100 ms, event-loop stall < 500 ms and one sudo
(old code: first poll 3157 ms).
This commit is contained in:
2026-10-09 15:21:31 +07:00
parent 84e2a9f0ac
commit 4407b3bb16
5 changed files with 206 additions and 57 deletions
+7 -1
View File
@@ -567,7 +567,13 @@ a pane really is. Each case below is refused or bounded, with a test in
- **A member's stopped or flooding tmux server.** Every call on it has a 3 s
limit and a 1 MiB answer cap; one that hit either is not asked again for a
minute. Meanwhile only that member's panes count as unknown (kept, not
ended); everyone else's are still checked. Every other synchronous call
ended); everyone else's are still checked. Listing a member's panes (every
poll: the pane list, the workspace list, the 10 s heal beat, the liveness
reaper) never waits for them: it runs in the background and the poll reads
its last answer, so a stopped server stalls nobody. Until a first answer,
or once the last is 15 s old, their panes count as unknown too. A call a
click makes on a member's pane (start, type, stop) still waits, up to the
3 s. Every other synchronous call
that runs as a member (git in their folder, `write-as`) stops after 5 s.
- **A member's huge or sparse files.** Transcripts and journals are read line
by line with a 64 MiB cap per line (a 512 MB line used to crash the server);
+1 -1
View File
@@ -267,7 +267,7 @@ Review ran 2026-10-08 (four lenses: root/sudo, cross-member, secrets, local-mode
- F34 a developer's extra workspace folder is checked against their own folder (`scope.folderAllowed`, real path) before anything probes it, and again on the path it records (`403 EFOLDER`);
- F35 the Codex pool's quota reads a run's rollout only from the `CODEX_HOME` of the account its workspace runs as, never one dated in the future. Accepted ceiling (documented): a member can still forge their own session's reading.
- F36 (critical) `GET /api/cc-config/file` never serves the data dir (`isSecretConfigPath`): an analyst read an admin gateway pane's live pane token from `proxy-settings/`, plus `vapid-keys.json` and the database;
- F37 every tmux call on a member's socket is bounded (`memberCall`: 3 s, SIGKILL, 1 MiB) with a 60 s back-off — a SIGSTOPped member server froze the event loop for good;
- F37 every tmux call on a member's socket is bounded (`memberCall`: 3 s, SIGKILL, 1 MiB) with a 60 s back-off — a SIGSTOPped member server froze the event loop for good. Member LISTING is async (`refreshList`: `execFile`, same limits) into the `listed` cache the sync readers use (stale-while-revalidate, 15 s max age, cold = unknown; a session this process makes or kills is written into the answer at once), so a frozen member no longer costs a 3 s block per back-off on the poll path; a click's call (new-session, send-keys, kill, has-session) is still a sync ≤ 3 s;
- F38 a failing member server makes only that member's panes unknown (`liveSessions().has` true for them), never null for everyone; member list errors are cached;
- F39 `execIn` and `write-as` (every sync sudo into a member) stop after 5 s (`MEMBER_EXEC`) — a FIFO `include.path` or a FIFO write target hung git and write-as;
- F40 transcripts and journals are read with `readLines` (chunked, a line over 64 MiB is skipped, never built) — readline crashed the server on a one-line 512 MB sparse file (import-history, sessions, codex-usage);