nntrivi2001
ca821008e2
Measured on a copy of a real 1.2 GB / 253K-event database, 10 simulated browsers + 10 hooks/s for 90 s: event-loop delay p99 1239 ms → 32 ms (max 2.0 s → 94 ms), 266 failed requests → 0, 74 → 111 requests/s served, server RSS 633 → 346 MB. Entry bundle 975 KB raw → 394 KB (106 KB brotli). Server - /api/analytics: the events-table scans run in the read worker (scoped-stats eventAggregates, read-pool job analyticsEvents); runRead takes maxAgeMs — analytics 10 s, /api/workflows 30 s — keyed by the args - /api/workflows/skill-usage: +event_type and a tool_name range instead of LIKE, so it uses idx_events_tool (784 ms → a few ms) - idx_events_session_type_tool (session_id, event_type, tool_name, created_at) replaces idx_events_session_type: covering reads for session stats, the hook dedupes and the import resume point (1.2 s → 20 ms); 64 MB cache_size and 256 MB mmap_size on both connections - WAL: the server sets wal_autocheckpoint = 0 and checkpoints PASSIVE every DASHBOARD_WAL_CHECKPOINT_MS (5 s) from the read worker's own writable connection; a failed checkpoint restores the automatic one (hook COMMIT p95 109 ms → 15 ms at 100 hooks/s) - tmux: one list-sessions answers a whole GET /api/lanes and the heal beat (liveSessions), shared for 1 s and cleared by this process's new/kill — GET /api/run's sync spawns had blocked 60 s of a 90 s run; the heal beat skips tmux when nothing runs; attach options chained into 2 spawns - hooks: TurnDuration/APIError deduped against one read per hook instead of a query per transcript entry (23 ms/hook on a 763-turn session); one lane scan per hook; hourly upserts skip rows that did not grow; frames are sent after the transaction commits (and after the pane binding); new_event carries its row id; a split pane's (slot 1–9) hooks no longer rebind the lane's session_id - watchdog liveness probe runs ps asynchronously; transcript newline scan via indexOf; startup subagent backfill fingerprints the files it already checked - pty attach: backpressure (pause past 1 MB unsent, resume under 256 KB) - static: hashed assets served from their pre-compressed .br/.gz sibling Client - hidden tabs neither refetch on WS events nor poll (useLiveRefresh, panes, lane cards, git header, changes panel, settings); no per-tab hourly /updates/check (the server already checks) - refetch storms throttled (workspace tree/session list 3 s, /api/run only on a run-key change, notification pane lookup only when shown); WS reconnect jitter; terminal retry backoff 1.5 → 30 s, stops on 1008 - heavier throttles for aggregates (15 s) and detail views; Monitor keepIfSame; GET in-flight dedupe in request() - lazy Dashboard/History, xterm (with the first terminal), lane modals, Sagi (only while enabled); only the active locale loads; Workspace chunk prefetched on a /workspace deep link - Sagi: no work while disabled, pupil via CSS variables, memo avatar - build: .br/.gz siblings, sourcemap hidden; service worker no longer caches /assets/ (cache v3 drops the old one) Tests: scripts/test-server.js drops the GIT_DIR-family variables a hook exports, so a suite run from a linked worktree's pre-commit can no longer commit into the real repository or set core.bare. Docs: ARCHITECTURE, DATABASE, API, DEPLOYMENT, client README, docs/plans/perf/README.md (audit, checklist, results).
Performance pass — 10+ concurrent users, lean local mode
Audit 2026-10-07 against a copy of the real DB (1.2 GB, 253K events). Baseline,
10 simulated browsers + 10 hooks/s for 90 s: event-loop delay p99 1239 ms, request
p95 ~10 s, 266 failed. Bench tools: /tmp/ahperf/tools/ (clients.js, hookload.js,
bench.js). Target: ELD p99 < 100 ms, request p95 < 300 ms, 0 failed.
Dev (4830) and prod (4820) keep sharing one dashboard.db.
Server
- S1
/api/workflows/skill-usage:+event_type+ range instead of LIKE (784 ms → ~5 ms) - S2
/api/analyticsintoread-pool.jsJOBS + short memo keyed by scope/args - S3
/api/workflowsresult memo (30 s) - S4
idx_events_session_type_toolreplacesidx_events_session_type(partial index skipped: the 30 s workflows memo covers it);cache_size/mmap_size - S5 WAL:
wal_autocheckpoint=0on the main connection, PASSIVE checkpoint off-thread, fallback - S6 tmux: 1 s
list-sessionsmemo (cleared by this process's new/kill), oneliveSessions()perGET /api/lanes, attach options chained (4 spawns → 2), asyncpsfor the watchdog, heal tick skipped when nothing runs - S7 hooks: hoist TurnDuration/APIError statements + watermark; one
listLanes()per hook; hourly upsert only on change - S8 hooks.js
session_idrebind skipped for slot 1–9 panes - S9 startup subagent backfill: do not re-parse unstampable files every boot
- S10 pty-attach backpressure
- S11 serve pre-compressed
.br/.gzassets - S12 broadcasts only after the hook transaction commits
- S13
new_eventcarriesid - [~] S14 codex-usage memo — DROPPED: an in-memory "already read" map survives Clear data and blocks the re-import (caught by codex-usage tests); the pass costs ~5 ms per 15 s
- S15 transcript newline scan via
indexOf
Client
- C1 no WS-triggered refetch in hidden tabs; hidden-tab pollers paused; drop hourly
/updates/check - C2 throttle WorkspaceTree / WorkspaceSessionList /
/api/runrefetchers; lazy notification lookup - C3 WS reconnect jitter; terminal retry backoff
- C4 heavier throttles for aggregates / detail refetches; Monitor
keepIfSame; GET in-flight dedupe - C5 build: pre-compress assets,
sourcemap: "hidden" - C6 load only the active locale; lazy routes/Sagi/modals/xterm
- C7 Sagi re-render fixes; service-worker asset cache
Verify
npm run test:server(2556/2556),npm run test:client(1084/1084),tsc -b, header check; headless Chrome smoke (en + vi, 4 pages, terminal attach)- re-run the 10-client bench (server side, same old-client simulation) — see Result
Result (server, clients.js 10 90 10, same simulated old client)
| before | after | |
|---|---|---|
| event-loop delay p99 / max | 1239 ms / 2.0 s | 31.8 ms / 94 ms |
| failed requests | 266 | 0 |
| requests served | 74/s | 111/s |
| hook p95 / p99 (10/s) | — | 7.6 / 28.8 ms |
| RSS | 633 MB | 346 MB |
Biggest single cause after the first round: 2,247 synchronous tmux list-sessions
from GET /api/run = 60 s of a 90 s run blocked (fork of a 600 MB process); the 1 s
memo removed it. The client fixes (C1–C4) cut the request rate itself on top.