AgentHub/docs/plans/perf

Performance pass — 10+ concurrent users, lean local mode

Audit 2026-10-07 against a copy of the real DB (1.2 GB, 253K events). Baseline, 10 simulated browsers + 10 hooks/s for 90 s: event-loop delay p99 1239 ms, request p95 ~10 s, 266 failed. Bench tools: /tmp/ahperf/tools/ (clients.js, hookload.js, bench.js). Target: ELD p99 < 100 ms, request p95 < 300 ms, 0 failed.

Dev (4830) and prod (4820) keep sharing one dashboard.db.

Server

• S1 /api/workflows/skill-usage: +event_type + range instead of LIKE (784 ms → ~5 ms)
• S2 /api/analytics into read-pool.js JOBS + short memo keyed by scope/args
• S3 /api/workflows result memo (30 s)
• S4 idx_events_session_type_tool replaces idx_events_session_type (partial index skipped: the 30 s workflows memo covers it); cache_size/mmap_size
• S5 WAL: wal_autocheckpoint=0 on the main connection, PASSIVE checkpoint off-thread, fallback
• S6 tmux: 1 s list-sessions memo (cleared by this process's new/kill), one liveSessions() per GET /api/lanes, attach options chained (4 spawns → 2), async ps for the watchdog, heal tick skipped when nothing runs
• S7 hooks: hoist TurnDuration/APIError statements + watermark; one listLanes() per hook; hourly upsert only on change
• S8 hooks.js session_id rebind skipped for slot 1–9 panes
• S9 startup subagent backfill: do not re-parse unstampable files every boot
• S10 pty-attach backpressure
• S11 serve pre-compressed .br/.gz assets
• S12 broadcasts only after the hook transaction commits
• S13 new_event carries id
• [~] S14 codex-usage memo — DROPPED: an in-memory "already read" map survives Clear data and blocks the re-import (caught by codex-usage tests); the pass costs ~5 ms per 15 s
• S15 transcript newline scan via indexOf

Client

• C1 no WS-triggered refetch in hidden tabs; hidden-tab pollers paused; drop hourly /updates/check
• C2 throttle WorkspaceTree / WorkspaceSessionList / /api/run refetchers; lazy notification lookup
• C3 WS reconnect jitter; terminal retry backoff
• C4 heavier throttles for aggregates / detail refetches; Monitor keepIfSame; GET in-flight dedupe
• C5 build: pre-compress assets, sourcemap: "hidden"
• C6 load only the active locale; lazy routes/Sagi/modals/xterm
• C7 Sagi re-render fixes; service-worker asset cache

Verify

• npm run test:server (2556/2556), npm run test:client (1084/1084), tsc -b, header check; headless Chrome smoke (en + vi, 4 pages, terminal attach)
• re-run the 10-client bench (server side, same old-client simulation) — see Result

Result (server, clients.js 10 90 10, same simulated old client)

	before	after
event-loop delay p99 / max	1239 ms / 2.0 s	31.8 ms / 94 ms
failed requests	266	0
requests served	74/s	111/s
hook p95 / p99 (10/s)	—	7.6 / 28.8 ms
RSS	633 MB	346 MB

Biggest single cause after the first round: 2,247 synchronous tmux list-sessions from GET /api/run = 60 s of a 90 s run blocked (fork of a 600 MB process); the 1 s memo removed it. The client fixes (C1–C4) cut the request rate itself on top.