feat(server-mode): spend quota per user — no new session, prompts refused once used up

An admin sets a default spend quota on the Users page and overrides it per
user (Edit user → Spend quota: default / no limit / their own), in dollars
per local day or per sliding 5 hours. Spend is the users page's own
hourlyTokenRows + calculateCost over the sessions a user owns.

Over the quota, until it opens again:
- spawnRun refuses every start/resume/home prompt with EQUOTA (429);
- a minute sweep writes each pane's AGENTHUB_QUOTA_FILE (host: data dir,
  member: handoff dir via write-as) and hook-handler.js exits 2 on
  UserPromptSubmit from that file alone (Claude Code, Codex with trusted
  hooks) — fail-open, never asks the dashboard, never for Antigravity;
- a member's omp on the shared login loses its gateway key (clientsFor);
- panes no hook can stop (Antigravity, Cursor, Codex without trusted
  hooks, omp on its own login) are ended.

GET /api/me/quota feeds a "Quota N%" chip in the status bar.
New: server/lib/user-quota.js, user_quotas table, PUT /api/users/quota-default,
PATCH /api/users/:id {quota}, docs in SERVER-MODE.md § Spend quotas.
This commit is contained in:
2026-10-28 14:31:11 +07:00
parent 9e3d6f4ca1
commit fa27b299e7
29 changed files with 2044 additions and 23 deletions
+1
View File
@@ -327,6 +327,7 @@ graph TD
| `routes/settings.js` | System info (DB size, hook status, server uptime, transcript cache stats), data export as one versioned JSON bundle and matching import/restore (`POST /api/settings/import` via `server/lib/data-transfer.js` — idempotent, session-atomic, non-destructive; consolidates machines), session cleanup (abandon stale, purge old), clear all data (including the fired-alert feed and webhook delivery log; alert *rules* and webhook *targets* are preserved as user configuration), reset pricing, reinstall hooks |
| `routes/alerts.js` | HTTP surface for the rules-based alerting engine: alert-rule CRUD (`GET/POST /api/alerts/rules`, `PATCH/DELETE /api/alerts/rules/:id` — rule_type is immutable after creation, config re-validated against the stored type on PATCH), the fired-alert feed (`GET /api/alerts` with `?unacked=true` + pagination, response carries `total` and `unacked` counts), and acknowledgement (`POST /api/alerts/:id/ack`, `POST /api/alerts/ack-all`, broadcasting `alert_updated`), and the read-only dry run `POST /api/alerts/rules/preview` (`{rule_type, config, days?, cooldown_seconds?}` → what `previewRule` says it would have fired over the last `days`, 1–90, default 7). Every rule mutation calls `invalidateRuleCache()` so the evaluation engine picks up changes immediately |
| `routes/budgets.js` + `lib/budgets.js` | Spend budgets: a budget IS an alert rule of type `budget` (CRUD through `/api/alerts/rules`). `lib/budgets.js` computes a period's bounds in the user's local time (`tz_offset` saved with the rule; day / week from Monday / month), its spend priced like `GET /api/pricing/cost` by the day tokens were spent (via `dailyTokenRows` + `costWhere`, scoped to `tool`/`lane`), and the pace projection to the period's end; `GET /api/budgets/status` lists every enabled budget with `spend`, `pct`, `projected` and the thresholds `crossed` |
| `lib/user-quota.js` | Server mode's spend quota per user (`user_quotas`, row 0 = the default; `PUT /api/users/quota-default`, `PATCH /api/users/:id {quota}`, `GET /api/me/quota`): `limit_usd` per local `day` or sliding `5h`, spend = the users page's `hourlyTokenRows` + `calculateCost` over owned sessions. Over it: `spawnRun` throws `EQUOTA` (429); a 60 s `sweep` writes each pane's `AGENTHUB_QUOTA_FILE` (the hook handler then exits 2 on `UserPromptSubmit`, from the file alone, fail-open), leaves the owner's omp keys out of the gateway's clients file and ends panes no hook can stop |
| `lib/alerts.js` | Rule evaluation engine for the alerting feature. Six rule types — `cost_threshold` (a session's cost ≥ `usd`, on token-bearing events like `token_threshold`) and `budget` (on the sweep: each of `thresholds` % of `limit_usd` for the current `period` fires ONCE per period, deduped on `details.period_start` + `details.threshold` and bypassing the cooldown) plus the four original ones: `event_pattern` (match `event_type` / `tool_name` / `summary_contains`, optionally requiring ≥ `count` matching events within `window_minutes` — counted via a dynamically built, statement-cached SQL query), `token_threshold` (session total tokens ≥ `total_tokens`, only evaluated on token-bearing events: PostToolUse / Stop / SubagentStop / SessionEnd), `inactivity` (active session whose `updated_at` — bumped on every ingested event — is older than `minutes`), and `status_duration` (agent stuck in `working`/`waiting` with no activity for `minutes`, joined against active sessions). Event-driven types run via `evaluateEvent()` called from `routes/hooks.js` **after** the ingest transaction commits and the HTTP response is sent — alerting can never slow down or fail hook ingestion, and `evaluateEvent` is itself fully try/catch-guarded per rule. Time-based types run via `sweepTimeRules()` on a 60 s unref'd interval (same pattern as the hooks watchdog). `fireAlert()` applies per-(rule, session, agent) cooldown dedup (`cooldown_seconds`, default 300) by checking the most recent `alert_events` row for the scope, then persists and broadcasts `alert_triggered`. Enabled rules are cached in memory (hook ingest is hot) and invalidated on every CRUD mutation. `validateRuleConfig()` normalizes + validates type-specific config and is shared with the routes. After persisting and broadcasting a fired alert, `fireAlert()` hands it to `lib/webhooks.js` `dispatchAlert()` fire-and-forget (lazy-required to keep the module graph acyclic) — webhook delivery never blocks or fails alert firing |
| `routes/webhooks.js` | HTTP surface for universal webhook targets: target CRUD (`GET/POST /api/webhooks`, `PATCH/DELETE /api/webhooks/:id` — `type` is immutable after creation), a redacted provider catalog (`GET /api/webhooks/providers`, drives the UI form), a synchronous test probe (`POST /api/webhooks/:id/test` — always 200, the `ok` flag carries the downstream delivery result), and a per-target delivery log (`GET /api/webhooks/:id/deliveries`). Validation is registry-driven: required URL (per provider), per-provider config fields, https enforcement, generic-family secret/headers. **Security**: target URLs are masked (host + last 4 chars) and secret config fields + custom-header values are redacted in every response — full URLs, signing secrets, and credentials (routing keys, api keys, bot tokens) are stored server-side and never leave the server. PATCH uses "set-flag" semantics (omit `url`/`secret`/`headers`/`config` to leave unchanged); `config` is merged over the existing config so one field can change without re-sending secrets. Every mutation calls `invalidateWebhookCache()` |
| `lib/webhook-providers.js`| Declarative registry of the 14 first-class providers (+ generic). Each entry declares a `family` (`chat` / `api` / `generic`), a payload `format`ter, URL resolution (`urlFrom(config)` for Telegram/Opsgenie that derive the endpoint, `defaultUrl` for PagerDuty, or a user-supplied URL), optional `authFrom(config)` headers (Opsgenie GenieKey), and the credential `fields` the UI renders + the route validates. Formatters emit each platform's native body: Slack Block Kit, Discord embed, Teams Adaptive Card wrapped in the Power Automate Workflows `{ type: "message", attachments: [...] }` envelope (the legacy O365-connector MessageCard transport was retired May 2026), Google Chat text, Mattermost/Rocket.Chat Slack-style attachments, Telegram sendMessage (HTML), PagerDuty Events API v2 (with `dedup_key`), Opsgenie Alert API, Splunk On-Call/VictorOps, and the generic `{ event, alert }` envelope. A provider may also declare `verifyResponse(body)` to veto a 2xx that actually signals failure (Splunk On-Call returns 200 with `result:"failure"`). `publicProviders()` returns the redacted catalog. Adding a provider = one registry entry + a formatter — no route or delivery changes |