feat: per-tool settings, persistent settings file, Codex and Antigravity sessions, tokens and hooks (#40 B, C1, C2, D)
Part B: Model Pricing, Hook Configuration and CLAUDE_HOME move from Settings → Dashboard to the Claude Code tab's AgentHub sub-tab (ClaudeAgentHubSettings.tsx); CODEX_HOME is editable in Codex → AgentHub. A home that is or contains the OS home is refused (EBADHOME). Settings persistence: dashboard-saved settings go to ~/.config/agenthub/settings.env (AGENTHUB_SETTINGS_FILE), outside the plugin cache an update wipes. Load order: real env > the checkout's .env > that file. The first CLAUDE_HOME change pins DASHBOARD_DATA_DIR so the DB stays put. C1: codex-usage.js reads Codex rollout journals into sessions (new column tool) and token_usage, tracked per file in codex_rollouts; forks deduped by total vector, sub-agent threads folded into the root; pricing-sync also prices OpenAI models; PricingSettings with a vendor filter. C2: codex-hooks.js installs AgentHub's hooks into $CODEX_HOME/hooks.json on a click, through a stable forwarder; Codex trust is never written. The hook route creates tool='codex' sessions and folds sub-agent hooks into the root. D: antigravity-usage.js reads agy's per-conversation SQLite files (tokens from ModelUsageStats) into tool='antigravity' sessions; antigravity-hooks.js installs the named hook `agenthub` (PreInvocation, PostToolUse, Stop — never PreToolUse) into ~/.gemini/config/hooks.json; Settings → Antigravity tab; Gemini pricing. Whole-change review fixes: sameOriginGuard on both home PUTs, routers.missing on a CLAUDE_HOME change, reset-pricing scope, GET /api/sessions?tool=, clear-data covering the new tables, Codex child-thread events moving no status, hooks owning status only while current, EFOREIGN for a forwarder that is not ours, agy quiet window and turn rules, AGENTHUB_SETTINGS_FILE honoured from .env. Docs, OpenAPI and the CLAUDE.md marker updated. Server 2141, client 779.
This commit is contained in:
@@ -0,0 +1,202 @@
|
||||
# Antigravity parity, part D: sessions, tokens and hooks
|
||||
|
||||
Issue: #40 part D. Date: 2026-09-29. Status: self-reviewed (the user asked for
|
||||
autonomous work with a self-review per part). Mirrors parts C1 and C2
|
||||
(`2026-09-29-codex-usage-design.md`, `2026-09-29-codex-hooks-design.md`).
|
||||
|
||||
## Spike: what agy offers
|
||||
|
||||
Verified on agy 1.2.13 in an isolated `HOME`. agy keeps its login in the OS
|
||||
keyring, so that run used the user's own account.
|
||||
|
||||
### Tokens
|
||||
|
||||
Not in the transcript, but in agy's own SQLite file per conversation:
|
||||
`~/.gemini/antigravity-cli/conversations/<conversation id>.db`.
|
||||
|
||||
- `gen_metadata` has one protobuf row per model call. Field 1 is a message
|
||||
whose field 19 is the model name (`gemini-3.8-flash`) and whose field 4 is
|
||||
`ModelUsageStats`.
|
||||
- The field numbers were read from the proto descriptor embedded in the
|
||||
binary: 2 input, 3 output, 4 cache write, 5 cache read, 9 thinking.
|
||||
- Input does not include cache reads. Output includes thinking (171 = 73 + 98).
|
||||
- `trajectory_meta.cascade_id` is the conversation id.
|
||||
- `trajectory_metadata_blob` holds the workspace as a `file://` URI.
|
||||
- agy 1.0.0 wrote a `.pb` file instead, and Google's service refuses 1.0.0.
|
||||
|
||||
### Hooks
|
||||
|
||||
Documented at antigravity.google/docs/hooks.
|
||||
- `~/.gemini/config/hooks.json` maps hook **names** to their events:
|
||||
`PreToolUse`/`PostToolUse` (`{matcher, hooks}` groups), and `PreInvocation`,
|
||||
`PostInvocation`, `Stop` (flat handler lists). A handler is
|
||||
`{type: "command", command, timeout (s)}`.
|
||||
- agy asks for no trust.
|
||||
- The payload carries `conversationId`, `workspacePaths`, `transcriptPath`
|
||||
(`brain/<id>/.system_generated/logs/transcript_full.jsonl`) and `modelName`.
|
||||
`PostToolUse` adds `toolCall {name, args}`, `stepIdx` and `error`. `Stop` adds
|
||||
`terminationReason` and `fullyIdle`.
|
||||
- The hook runs with its cwd in `~/.gemini/config`, and `AGENTHUB_LANE_ID`
|
||||
does not reach it.
|
||||
|
||||
### Traps found in the spike
|
||||
|
||||
- **A `PreToolUse` hook that answers `{}` DENIES the tool** ("tool call denied
|
||||
by pre-tool hook"). There is no neutral answer:
|
||||
- `allow` would override the user's permission rules;
|
||||
- `ask` would force a prompt.
|
||||
- **No permission event, and no Stop when a run ends on an API error** or on a
|
||||
permission that headless mode refused.
|
||||
- `invocationNum` restarts at 0 for every run, including `--continue`.
|
||||
- **Running `agy` self-updates `~/.local/bin/agy`**, even with another `HOME`.
|
||||
The spike took the user's install from 1.0.0 to 1.2.13; the old binary is kept
|
||||
as `agy.<n>.old`.
|
||||
|
||||
## Design
|
||||
|
||||
1. **Sessions and tokens** (`server/lib/antigravity-usage.js`, every
|
||||
`DASHBOARD_ANTIGRAVITY_POLL_MS`, default 15 s).
|
||||
- Each conversation file is opened read-only when its mtime (or its
|
||||
non-empty `-wal`'s) changes; the mtime lives in `antigravity_conversations`.
|
||||
- It becomes a `tool = 'antigravity'` session: its name and start come from
|
||||
the transcript, its cwd from the blob, its model from the last generation.
|
||||
- Tokens are written whole, per model.
|
||||
- The status reflects the session's last activity: the newest of its file
|
||||
(a non-empty `-wal` included), its transcript and its last hook event. It
|
||||
is quiet after ten minutes when it has any hook event — mid-turn a long
|
||||
tool call writes nothing, and after a `Stop` agy may be waiting for the
|
||||
user, with no exit hook to say otherwise — and after two minutes when it
|
||||
has none.
|
||||
- When a session settles to `completed`, its agents complete too, its
|
||||
waiting flags are cleared, and its antigravity lane is set `idle`, but only
|
||||
if the hooks left it `running` and it has no `run_id`. A `Stop` that marked
|
||||
it `error` keeps that `error`.
|
||||
- A file that is not agy's (`SQLITE_NOTADB`, `SQLITE_CORRUPT`, `no such
|
||||
table`) is recorded with `session_id` `''` and retried only once it
|
||||
changes or at the next server start; any other failure is retried on the
|
||||
next pass.
|
||||
- Only the CLI's directory is read.
|
||||
- Gemini families are added to `pricing-sync.js`, taken from Vertex AI's or
|
||||
the Gemini API's LiteLLM entries.
|
||||
2. **Hooks** (`server/lib/antigravity-hooks.js`, Settings → Antigravity →
|
||||
**Antigravity hooks**, `GET/POST/DELETE /api/settings/antigravity-hooks`,
|
||||
`sameOriginGuard` on the writes).
|
||||
- The named hook `agenthub` has `PreInvocation`, `PostToolUse` and `Stop`,
|
||||
each `timeout: 5`. It never has `PreToolUse`.
|
||||
- Every other named hook is kept, and a non-object file is refused
|
||||
(`EBADJSON`).
|
||||
- The command is the stable path `~/.gemini/config/agenthub-hook.sh`. The
|
||||
script runs `scripts/hook-handler.js <Event> antigravity`, prints `{}`, and
|
||||
is refreshed at every start when it is ours.
|
||||
3. **Route** (`antigravityHookInput` in `server/routes/hooks.js`).
|
||||
- A hook is accepted only with a UUID `conversationId` and a
|
||||
`transcriptPath` under the CLI's directory. That keeps out the
|
||||
Antigravity IDE, which loads the same global file.
|
||||
- `session_id` is the `conversationId` and `cwd` is `workspacePaths[0]`.
|
||||
- A `PreInvocation` becomes `UserPromptSubmit` when `invocationNum` is 0,
|
||||
when the session has no event yet, when its last event is a `Stop`, when
|
||||
the session is not `active`, or when its last event is more than two
|
||||
minutes old; any other is dropped. `PostToolUse` carries
|
||||
`tool_name`/`tool_input` and `step`. `Stop` stays `Stop` — as
|
||||
`stop_reason: "error"` when it carries a non-empty `error` or a
|
||||
`terminationReason` matching `/error|fail/i`, and dropped while
|
||||
`fullyIdle` is false. Every other event is dropped.
|
||||
4. **Settings → Antigravity** (`AntigravityConfig.tsx`) holds Model Pricing
|
||||
(`vendor="google"`) and the hooks card. There is no sub-tab bar, because agy
|
||||
has no config surface AgentHub edits.
|
||||
|
||||
## Limits (documented, not fixable from here)
|
||||
|
||||
- There is no "needs your permission" state for an Antigravity lane: agy has no
|
||||
hook for it.
|
||||
- A lane that a dashboard pane owns (`run_id`) keeps its run-owned status, as
|
||||
it does for Codex.
|
||||
- The Antigravity IDE's sessions are not read.
|
||||
- Sub-agents: none were seen in a CLI run, so nothing folds them.
|
||||
- The formats are undocumented: a future agy may move them. A generation that
|
||||
no longer decodes is skipped, not guessed.
|
||||
|
||||
## Tests
|
||||
|
||||
- **Server:**
|
||||
- usage: the protobuf reader; the session and per-model tokens; no re-read;
|
||||
quiet → completed; the stuck lane set idle, while a paned one is kept;
|
||||
deleted stays deleted; another tool's row untouched; a hook-made cwd kept;
|
||||
a non-SQLite file skipped;
|
||||
- hooks: install with three events and no PreToolUse; the forwarder prints
|
||||
`{}`; other named hooks kept; remove; EBADJSON; a hook switched off is
|
||||
reported; the refresh is foreign-safe;
|
||||
- route: a turn on the antigravity lane only; later PreInvocations dropped;
|
||||
tool calls in the timeline; IDE and bad-id hooks ignored; the settings
|
||||
routes, including the cross-origin refusal;
|
||||
- handler: the `antigravity` marker, and an unknown marker dropped;
|
||||
- pricing: Gemini from Vertex AI, and another vendor refused.
|
||||
- **Client:** the hooks card (install, a hook switched off, Refresh, Remove, a
|
||||
refusal).
|
||||
- **End to end:** an isolated server on :4851 with `AGENTHUB_GEMINI_DIR` set.
|
||||
The hook was installed through the API, then a real `agy -p` read a file.
|
||||
Seen:
|
||||
- the lane went `running`;
|
||||
- the run ended with no Stop (a shell command refused in headless mode), and
|
||||
the poll set the lane back to `idle`;
|
||||
- the session had its name, cwd, model and tokens, and `view_file` in its
|
||||
timeline;
|
||||
- Settings → Antigravity showed the card as installed, with its three events.
|
||||
|
||||
## Found in the code review, fixed
|
||||
|
||||
- **Our own read-only open looked like activity.** On a file agy closed
|
||||
cleanly, better-sqlite3's read-only open leaves an empty `-wal`, whose fresh
|
||||
mtime flipped the session back to `active` with "Last active" set to the read
|
||||
time. An empty `-wal` no longer counts.
|
||||
- **The poll undid the hooks' `running` within 15 s.** Activity is now the
|
||||
newest of the file, the transcript and the session's last hook event. The
|
||||
quiet window is ten minutes, not two, when that event is not a `Stop`,
|
||||
because a long tool call writes nothing until it ends.
|
||||
- **A failed run showed as "ready for input".** A `Stop` carrying an `error`,
|
||||
or a `terminationReason` naming one, now arrives as `stop_reason: "error"`,
|
||||
and the poll keeps that `error`.
|
||||
- **Stops and turns.**
|
||||
- A `Stop` with `fullyIdle: false` is dropped, since background work is
|
||||
still running.
|
||||
- The first `PreInvocation` after a `Stop` is a new prompt, whatever its
|
||||
`invocationNum`.
|
||||
- **A script that is not ours** at the forwarder's path is never overwritten
|
||||
or removed (`EFOREIGN`, 400), in `codex-hooks.js` too.
|
||||
- **A rewritten `hooks.json` keeps its mode**, in `codex-hooks.js` too.
|
||||
- **Session id and symlinks.**
|
||||
- The session id is the file name's UUID; `cascade_id` is no longer read.
|
||||
- A symlinked `~/.gemini` is matched under both paths (`agyCliDirs`).
|
||||
- **Settling and bad files.**
|
||||
- A file that is not agy's is recorded, so it is not re-read and warned
|
||||
about every 15 s.
|
||||
- Settling a session to `completed` also completes its agents and clears
|
||||
their waiting flags.
|
||||
- **Docs.** `.env.example` now lists `DASHBOARD_ANTIGRAVITY_POLL_MS` and
|
||||
`AGENTHUB_GEMINI_DIR`.
|
||||
|
||||
## Found in the whole-change review, fixed
|
||||
|
||||
- **An open agy looked finished after a Stop.** The ten-minute window applied
|
||||
only while the last event was not a `Stop`, so a session waiting at the
|
||||
prompt settled to `completed` two minutes after each turn. agy fires no exit
|
||||
hook, so an open agy and a closed one look the same: any session with hook
|
||||
events now gets ten minutes, a hookless one two.
|
||||
- **A turn after a missed Stop was lost.** A `PreInvocation` is now also a
|
||||
prompt when the session has no event yet, is not `active` (a run that ended
|
||||
with no Stop, a session the poll settled), or its last event is over two
|
||||
minutes old — not only on `invocationNum: 0` or after a `Stop`.
|
||||
- **Error Stops.** Only a non-empty `error` counts, and a `terminationReason`
|
||||
counts when it matches `/error|fail/i`.
|
||||
- **A bad file was never retried, a busy one was given up on.** A file that is
|
||||
not agy's is recorded with `session_id` `''`, and those rows are deleted at
|
||||
every server start (`startAntigravityUsagePoll`), so a newer AgentHub tries
|
||||
it again; any other read failure (a busy file, our own error) is no longer
|
||||
recorded, so the next pass retries it.
|
||||
- **Settling left the session waiting.** It now clears the session's own
|
||||
waiting flag as well as its agents'.
|
||||
- **A forwarder path that is not a file of ours** (other content, an unreadable
|
||||
file, a directory — `scriptIsOurs`) is never overwritten (`EFOREIGN`) or
|
||||
removed, in `codex-hooks.js` too.
|
||||
- **A symlinked `hooks.json`** (a dotfiles repo's) left with no hook is emptied
|
||||
through the link instead of being unlinked, in both installers.
|
||||
@@ -0,0 +1,121 @@
|
||||
# Codex parity, part C2: Codex hooks
|
||||
|
||||
Issue: #40 part C. Date: 2026-09-29. Status: self-reviewed (the user asked for
|
||||
autonomous work with a self-review per part). Builds on C1
|
||||
(`2026-09-29-codex-usage-design.md`).
|
||||
|
||||
## Problem
|
||||
|
||||
C1 reads Codex sessions and tokens from the rollout journals, every 15 s. That
|
||||
leaves out what only a live signal gives:
|
||||
- a Codex session's tool calls in its timeline;
|
||||
- "Codex is waiting for you" when it asks permission, on the lane card and as a notification;
|
||||
- a Codex lane's running/idle without a 2-minute lag after a turn ends.
|
||||
|
||||
## What Codex offers (verified on 0.155.1, isolated `CODEX_HOME` + a fake model server)
|
||||
|
||||
- **Where hooks live:** `$CODEX_HOME/hooks.json`, in the same shape as Claude Code's `hooks` block:
|
||||
`{"hooks": {"<Event>": [{"matcher"?, "hooks": [{"type": "command", "command", "timeout"}]}]}}`.
|
||||
- Events: `SessionStart`, `UserPromptSubmit`, `PreToolUse`, `PermissionRequest`, `PostToolUse`, `Stop`, `SessionEnd`, and others.
|
||||
- Timeouts above 3 s are clamped (with a startup warning) for `SessionEnd`/`Interrupt`.
|
||||
- **The stdin payload is Claude's shape:**
|
||||
- `session_id` equals the rollout's thread id, so it matches C1's session row.
|
||||
- Also: `transcript_path` (the rollout), `cwd`, `hook_event_name`, `model`, `permission_mode`, and `turn_id` on turn-scoped events.
|
||||
- Tool events carry `tool_name` (`Bash`), `tool_input` and `tool_use_id`. `PermissionRequest` carries `tool_input.command`/`.description`.
|
||||
- The hook inherits Codex's environment, including `AGENTHUB_LANE_ID`.
|
||||
- **Trust:** a hook runs only once the user trusts its exact definition.
|
||||
- Codex asks at startup ("Hooks need review → Trust all and continue"), or the user runs `/hooks`.
|
||||
- Trust is stored in `$CODEX_HOME/config.toml` as `[hooks.state."<hooks.json path>:<event_snake>:<group>:<hook>"] trusted_hash = "sha256:…"`.
|
||||
- An untrusted hook is silently skipped by `codex exec`.
|
||||
- AgentHub never writes that trust and never passes `--dangerously-bypass-hook-trust`: skipping the user's review is not ours to do.
|
||||
|
||||
## Design
|
||||
|
||||
1. **A stable command:** `'<CODEX_HOME>/agenthub-hook.sh' <Event>`.
|
||||
- Trust is keyed on the definition, so the command must not change when AgentHub updates. A plugin install moves to a new cache directory on every update.
|
||||
- The script is AgentHub's own:
|
||||
- it runs this build's `scripts/hook-handler.js <Event> codex`, with the server's `node` (falling back to `node` on PATH, like the CLI launcher);
|
||||
- it sets the server's `CLAUDE_HOME`, so the handler finds the discovery file.
|
||||
- Every server start rewrites the script's content when it is installed and stale. Only the content changes, so the trusted definition stays valid.
|
||||
- Every value is single-quoted.
|
||||
2. **Install / remove** (`server/lib/codex-hooks.js`), only on an explicit click in **Settings → Codex → AgentHub → Codex hooks**:
|
||||
- `install`:
|
||||
- Writes the script (0755).
|
||||
- Merges one group per event into `hooks.json`: `SessionStart`, `UserPromptSubmit`, `PreToolUse`, `PermissionRequest`, `PostToolUse`, `Stop`, `Interrupt`, `SessionEnd`, each with `timeout: 3`.
|
||||
- Drops any earlier hook of ours first (matched by its exact command, never a group). Keeps every other entry.
|
||||
- Refuses a `hooks.json` that is not a JSON object (`EBADJSON`, the router's rule for `settings.local.json`).
|
||||
- Writes with temp + rename.
|
||||
- `remove`:
|
||||
- Takes our hooks out (a group left empty goes too) and deletes the script.
|
||||
- Deletes `hooks.json` when nothing else is left in it.
|
||||
- `status`:
|
||||
- Per event: whether our hook is present, and whether `config.toml` holds a trust entry for its position.
|
||||
- Whether the file is readable.
|
||||
- Win32 is refused (`EUNSUPPORTED`): the command is a POSIX shell script.
|
||||
- Routes: `GET/POST/DELETE /api/settings/codex-hooks`.
|
||||
3. **Handler:** `scripts/hook-handler.js` sends `tool: "codex"` when its second argument is `codex`. The route reads `tool` through an allowlist (`codex`, else `claude`).
|
||||
4. **Hook route, for `tool = codex`:**
|
||||
- **Sub-agent threads.** A hook from one would describe a child thread, not the user's session. We cannot see whether Codex fires hooks there, so this is guarded, not assumed.
|
||||
- When `transcript_path` is a rollout under `$CODEX_HOME/sessions` whose `session_meta.session_id` differs from its `id`, the hook is a sub-agent's.
|
||||
- Tool and permission events are attributed to the root session.
|
||||
- `SessionStart`/`SessionEnd`/`Stop`/`UserPromptSubmit` are dropped.
|
||||
- The meta read is cached per path.
|
||||
- **`PermissionRequest`** becomes the `Notification` Claude uses for a permission prompt, with the message `Codex needs your permission: <description or command>`. The awaiting flag, the lane's `needs_action`, the alert rules and the browser notification then work unchanged.
|
||||
- **`ensureSession`** creates the row with `tool = 'codex'`, named `Codex <8>` (C1 renames it from the prompt or thread name). It stores no `transcript_path`.
|
||||
- **The Claude transcript block** (tokens, titles, errors, compactions, turn durations), the SessionEnd transcript read, the sub-agent JSONL scan and the workflow ingest are skipped: they parse Claude's format, and C1 owns a Codex session's tokens.
|
||||
- **`touchLaneFromHook`** resolves the lane by `(cwd, tool)`, so a Codex hook reaches the Codex lane and never a Claude lane at the same directory. Stage detection runs for `claude` only: a non-claude lane carries no pipeline.
|
||||
5. **Who owns a Codex session's status.**
|
||||
- A session that has hook events is **hook-fed**. While its hooks are current (see the whole-change review below), they own its status, `ended_at` and `updated_at`, as for Claude.
|
||||
- C1's poll still writes its tokens, model and name, and `settleActive` leaves it alone while its hooks are current.
|
||||
- A session with no events keeps C1's journal-activity status.
|
||||
- A hook-fed session whose Codex process dies without `SessionEnd` stays active until the stale sweep (`STALE_MINUTES`, 3 h). The liveness reap probes `claude` processes only.
|
||||
6. **The lane mirror** (`codex-ingest.js`) skips a lane whose bound session got a hook event in the last 2 minutes: the hooks are current, and the mtime mirror would call a lane that just stopped "running" for up to 2 minutes.
|
||||
|
||||
## Found in the end-to-end check, fixed
|
||||
|
||||
- **Declining a permission (or Esc) ends the turn with `Interrupt`, not `Stop`.** The lane stayed `running` with a stale "needs your permission" until the next prompt. `Interrupt` is now installed and becomes `Stop`, since Codex is back at its prompt either way; a sub-agent thread's `Interrupt` is dropped. Adding it to an install the user already trusted made Codex ask about that one hook only, and the other seven kept their trust.
|
||||
- In the TUI, `SessionStart` fires on the first prompt, when Codex creates the thread, not when the TUI opens.
|
||||
|
||||
## Found in the code review, fixed
|
||||
|
||||
- **Remove/Reinstall deleted a user's hook along with ours** when it sat in our group, or when its command merely contained `agenthub-hook.sh`. Ours is now matched by its exact command shape, one hook at a time, and a group is dropped only when that leaves it empty.
|
||||
- **Install/remove had no `sameOriginGuard`.** Any web page could POST to install the hooks (Codex's trust prompt was the only stop). Both write routes now carry it.
|
||||
- **A symlinked `CODEX_HOME`** could show "untrusted" forever and turn the sub-agent guard off, if Codex reports the resolved path. Trust and `threadOf` now accept the home as given or resolved (`codexHomes` in `codex-home.js`).
|
||||
- **A session the journal poll imported first had no main agent**, so its agent-level state stayed empty. `ensureSession` now creates the missing `<id>-main` for a non-claude session.
|
||||
- Minor:
|
||||
- `refreshScript` leaves a file that is not ours alone (`foreign`), restores a lost exec bit, and a failure is logged.
|
||||
- Writes go through a symlinked `hooks.json` to its target, and a failed write leaves no temp file behind.
|
||||
- The card has **Refresh**, shows Remove and the grid on a partial install, and gives each event screen-reader text.
|
||||
- Left as is:
|
||||
- Trust is matched by position, so a stale entry at the same position reads as trusted.
|
||||
- The mirror takes over again after 2 minutes of hook silence during a long tool call.
|
||||
- A sub-agent hook that arrives before its rollout's first line creates its own session.
|
||||
- Whichever AgentHub server started last owns the forwarder.
|
||||
|
||||
## Found in the whole-change review, fixed
|
||||
|
||||
- **Hooks owned a session's status forever.** A session whose hooks were removed, or no longer trusted, stayed on its last hook-given status. Hooks now own it only while current (`hooksCurrent` in `codex-usage.js`: the newest event at or after the newest journal write minus 2 minutes); otherwise the journal decides again, in `settleActive` too.
|
||||
- **A permission prompt could become a Compaction.** The Notification → `Compaction` tag matches `compact|compress|…` in the message, and a Codex `PermissionRequest` quoting a command such as `tar --compress` matched it. The tag now applies to `tool = 'claude'` only.
|
||||
- **A sub-agent's tool call looked like the user resuming.** A child thread's `PreToolUse`/`PostToolUse`, folded onto the root session, is marked `_codexChild` and records work only: it no longer reactivates the session, clears its waiting flag, sets the main agent working, clears the lane's `needs_action` or moves the lane's status. Its own `Stop` is dropped, so anything it set would never be undone.
|
||||
- **A forwarder path that is not a file of ours** — other content, an unreadable file, a directory (`scriptIsOurs`) — is never overwritten (install answers 400 `EFOREIGN`) or removed (remove skips it).
|
||||
- **A rewritten `hooks.json` keeps the mode the user gave it**, and a symlinked one left with no hook is emptied through the link instead of being unlinked.
|
||||
- The route's `tool` allowlist is now `codex`, `antigravity` (part D), else `claude`.
|
||||
|
||||
## Not changed
|
||||
|
||||
- Claude's hook path, apart from the parameter.
|
||||
- Hook installs for Claude.
|
||||
- `agenthub doctor`.
|
||||
- The notification title, which still says "Claude needs your confirmation"; the body names Codex.
|
||||
- No sub-agent rows for Codex.
|
||||
|
||||
## Tests
|
||||
|
||||
- **Server:**
|
||||
- `codex-hooks.js`: install into an empty home, and next to a user's own hooks (kept); reinstall replaces ours; remove leaves theirs and deletes a file of ours only; a non-JSON file refused; status with and without a trust entry; the script refreshed and its command unchanged.
|
||||
- Route: a codex hook creates a `tool = codex` session with no `transcript_path` and no Claude transcript read; `PermissionRequest` stamps awaiting and the codex lane's `needs_action` while the claude lane at the same cwd is untouched; no stage detection on a codex lane; a sub-agent thread's `Stop` dropped and its `PreToolUse` attributed to the root.
|
||||
- C1 does not overwrite a hook-fed session's status.
|
||||
- The mirror skips a lane with a fresh hook.
|
||||
- The handler sends `tool`.
|
||||
- **Client:** the Codex hooks section (status, install, remove).
|
||||
- **Browser:** the isolated server, a real `codex exec` against a fake model with the installed hooks trusted through the TUI prompt, then the session's timeline and a lane's `needs_action`.
|
||||
@@ -0,0 +1,123 @@
|
||||
# Codex parity, part C1: sessions, token usage and pricing
|
||||
|
||||
Issue: #40 part C. Date: 2026-09-29. Status: self-reviewed (the user asked for
|
||||
autonomous work with a self-review per part). Part C2 (Codex hooks) is separate.
|
||||
|
||||
## Problem
|
||||
|
||||
Codex emits no hook AgentHub listens to, so a Codex session never reaches the
|
||||
`sessions` table and its tokens cost nothing on any page. The pricing table
|
||||
and the automatic pricing sync know Claude models only.
|
||||
|
||||
## What Codex writes (verified on 0.155, 178 real rollouts)
|
||||
|
||||
`$CODEX_HOME/sessions/YYYY/MM/DD/rollout-<ts>-<thread id>.jsonl`, append-only:
|
||||
|
||||
- **Line 1** is `session_meta`, with these payload fields:
|
||||
- `id`: the thread.
|
||||
- `session_id`: the ROOT session. It is the same as `id` except in a sub-agent thread, whose `thread_source` is `subagent`. Older CLIs omit it.
|
||||
- `cwd`, `timestamp`, `cli_version`.
|
||||
- `forked_from_id`: set when the file is a fork.
|
||||
- **`turn_context.payload.model`** is the model for the lines that follow. It can change mid-session, or be a router id such as `clinepass/cline-pass/deepseek-v4-pro`.
|
||||
- **`event_msg` with `payload.type == "token_count"`** carries two things:
|
||||
- `info.total_token_usage`: a cumulative total for the thread.
|
||||
- `info.last_token_usage`: the delta.
|
||||
|
||||
`info` may be null. The same total is often emitted twice in a row.
|
||||
- **Token fields:**
|
||||
- `input_tokens` includes `cached_input_tokens`.
|
||||
- `output_tokens` includes `reasoning_output_tokens`.
|
||||
- `cache_write_input_tokens` is separate.
|
||||
- **A fork COPIES its parent's whole history** into its own file, `token_count` events included, with rewritten timestamps. It then carries on from the parent's cumulative total. Summing the fork file naively counts the parent twice.
|
||||
|
||||
## Design
|
||||
|
||||
1. **`sessions.tool`** is a new column: `TEXT NOT NULL DEFAULT 'claude'`, added with the additive `ALTER TABLE` idiom the `source` column used. Every existing row and every hook-created row stays `claude`.
|
||||
2. **`codex_rollouts`** is a new table recording how far each file has been read:
|
||||
- Columns: `path` PK, `session_id` (root), `thread_id`, `size`, `mtime_ms`, `offset`, `state` JSON, `updated_at`.
|
||||
- `state` holds the current model, the last total seen, the per-model usage sums, and the meta.
|
||||
- A file whose size and mtime are unchanged is not read again.
|
||||
- A grown file is read from `offset` up to its last newline.
|
||||
- A shrunken file is re-read from 0.
|
||||
- Because this table lives in the DB, a session the user deleted stays deleted until Codex writes to its file again, while wiping the DB re-imports everything.
|
||||
3. **`server/lib/codex-usage.js`** does the ingest:
|
||||
- **`parseRolloutChunk(text, state, skip)`** is pure. It JSON-parses only the lines it needs, picked by five needles (`NEEDLES`): `session_meta`, `turn_context`, `token_count`, and a prompt as `user_message` (older CLIs) or `UserMessage` (0.149+). Each new total adds its `last_token_usage` to the current model, mapped as:
|
||||
- input = `input − cached`
|
||||
- cache_read = `cached`
|
||||
- cache_write = `cache_write_input_tokens`
|
||||
- output = `output_tokens`
|
||||
|
||||
OpenAI bills reasoning as output.
|
||||
- A total equal to the previous one is skipped.
|
||||
- **Forks:** when a file with `forked_from_id` is read from offset 0, the parent's file is found by the `-<id>.jsonl` name. Every total vector in the parent is skipped in the child: the copied prefix matches the parent exactly, and the child's own totals never do. If the parent's file is gone, nothing is skipped and the overcount is logged.
|
||||
- **`ingestCodexUsage({ broadcast })`:**
|
||||
- Walks every rollout (no newest-40 cap) and yields to the event loop between files.
|
||||
- For each root session touched, it sums its threads' usage per model and writes it with `stmts.setTokenUsage`. The totals are exact recomputes, so the high-water baseline of `replaceTokenUsage` would be wrong here.
|
||||
- It then inserts or updates the `sessions` row:
|
||||
- `tool='codex'`
|
||||
- `name` = the `thread_name` from `session_index.jsonl`, else the first prompt (first line, 60 chars; `user_message` or the 0.149+ `item_completed` `UserMessage`), else `Codex <8>`
|
||||
- `cwd` and `started_at` from the root meta
|
||||
- `model` = the latest model
|
||||
- `status` = `active` while any of the session's files was written in the last 2 minutes, else `completed`, with `ended_at` = the newest journal's mtime
|
||||
- metadata `{source:"codex-rollout"}`
|
||||
- It never touches a row whose `tool` is not `codex`, and never sets `transcript_path`, so the Claude transcript parsers and sweeps skip these rows.
|
||||
- It broadcasts `session_created`/`session_updated`.
|
||||
- One pass at a time. It runs at startup (in the background) and then every `DASHBOARD_CODEX_POLL_MS`, the codex lane poll's own setting (`0` turns both off).
|
||||
4. **Pricing sync learns OpenAI.**
|
||||
- `familyOf` also accepts `gpt-…`, `o<digit>…` and `codex-…` ids, dropping a `-YYYY-MM-DD` suffix.
|
||||
- `toRow` takes `litellm_provider == "openai"` entries. A cache write costs `cache_creation_input_token_cost` or 0, since OpenAI has no cache-write charge by default. The fast rate is 0. The display name comes from `displayName` (`gpt-5.6-sol` → `GPT-5.6 Sol`).
|
||||
- Long-context (>272k) and priority/flex tiers are not modelled. Codex's context window stays under 272k.
|
||||
- `POST /api/settings/reset-pricing` starts a sync right after reseeding, so OpenAI rows wiped by a reset return at once.
|
||||
5. **Model Pricing in Codex → AgentHub.**
|
||||
- The pricing section of `ClaudeAgentHubSettings.tsx` becomes `components/settings/PricingSettings.tsx` with a `vendor` prop. The Claude tab shows `claude-…` rows; the Codex tab shows OpenAI rows.
|
||||
- A **Show all models** switch shows every row, since one table prices every tool's sessions (a router model may serve either). A row added outside the filter turns the switch on, so it never vanishes from view.
|
||||
- Reset Defaults stays on the Claude tab only and sends `{scope: "claude"}`: it deletes and reseeds only `claude-…` rows, so OpenAI rows (and a custom router row) survive it; a reset with no body wipes every row, and OpenAI rows come back from the sync.
|
||||
6. **Claude-only surfaces exclude Codex sessions.**
|
||||
- The Workspace's external sessions, which offer attach and `claude --resume`, list `tool === 'claude'` only.
|
||||
- The Sessions list and session detail show a **Codex** badge.
|
||||
|
||||
## Found in the browser check, fixed
|
||||
|
||||
- The Claude liveness reap and the transcript watchdog (`routes/hooks.js`) probed Codex sessions for a `claude` process and reaped them: both now select `tool = 'claude'` only.
|
||||
- The pricing sync had already run when the first ingest wrote new models; a pass that writes sessions now triggers a sync.
|
||||
|
||||
## Found in the code review, fixed
|
||||
|
||||
- **A journal met under a new path counted twice** (CODEX_HOME moved, or a symlinked alias): there is now one `codex_rollouts` row per thread, and reading a file drops the thread's rows under other paths.
|
||||
- **The Claude resume picker offered Codex sessions** (`RunSetup.tsx`), which `claude --resume` cannot open: it lists `tool === 'claude'` only.
|
||||
- **A failed price-list fetch was retried on every Codex pass** (every 15 s): a failure now waits for the sync's own 15-minute check.
|
||||
- **A fork's copied history arriving after the first read was counted**: the parent's totals are skipped until the fork's first own total (`state.pastFork`), on every read, not just the one from offset 0.
|
||||
- **A journal written in the future** (a clock running ahead) kept its session `active`: it no longer counts as activity.
|
||||
- **A large journal blocked the event loop**: one pass reads at most 16 MB of a file, and a fork's parent is streamed.
|
||||
- **A model gone from a re-read journal kept its old usage row**: a Codex session's usage is rewritten whole.
|
||||
- **A sub-agent file read before its root fixed the session's cwd, start and model**: the root thread's file sets them once it is read.
|
||||
- **The startup cleanup completed active Codex sessions** and **a Claude lane's purge deleted a Codex lane's sessions** at the same directory: both are scoped by tool.
|
||||
- **The codex lane mirror flapped** (`codex-ingest.js`, predates C1): a lane with an old journal beside the live one went running → idle → running on alternate passes, and a journal written in the current millisecond read as future. One verdict per lane now, with a second of slack.
|
||||
|
||||
## Found in the whole-change review, fixed
|
||||
|
||||
- **Hooks kept a session's status forever.** A Codex session with any hook event kept its hook-given status even after the hooks were removed or no longer trusted, so it could stay `active` for good. Hooks now own the status only while current — `hooksCurrent`: the session's newest event is at or after its newest journal write minus 2 minutes — and otherwise the journal decides again, in `settleActive` too.
|
||||
- **Clear data stopped Codex (and Antigravity) sessions ever coming back.** `POST /api/settings/clear-data` kept `codex_rollouts`, so a cleared session was never re-imported, or came back half on its next write: it now empties `codex_rollouts` and `antigravity_conversations` too.
|
||||
- **The per-thread lookup scanned the table**: `idx_codex_rollouts_thread` indexes `codex_rollouts(thread_id)`.
|
||||
- **A busy Codex lane pushed Claude rows out of a page**: `GET /api/sessions` takes `?tool=`, applied before the limit, and the Workspace attach list and the run form's resume picker ask for `tool: "claude"`.
|
||||
- **Reset Defaults wiped the OpenAI rows** a user had edited: the Claude tab's reset is scoped to `claude-…` rows (above).
|
||||
|
||||
Left as is: a journal rewritten to the same or a larger size is not detected (Codex only appends), and a fork without a Codex thread name is named after the prompt it copied from its parent.
|
||||
|
||||
## Not changed
|
||||
|
||||
Hook ingestion, `transcript-cache.js`, `import-history.js` and remote sync still handle Claude Code only. Codex lanes keep their idle/running mirror in `codex-ingest.js`. Session↔lane binding is untouched.
|
||||
|
||||
## Tests
|
||||
|
||||
- **Server:**
|
||||
- Chunk parser: mapping, duplicate totals, model switch, null info, partial last line.
|
||||
- Fork skip, with and without the parent file.
|
||||
- Sub-agent thread folded into the root session.
|
||||
- Incremental read after an append; a shrunken file re-read.
|
||||
- An unchanged file not re-read; a deleted session not resurrected.
|
||||
- A claude row never touched.
|
||||
- Pricing sync for an OpenAI family.
|
||||
- Reset triggers a sync.
|
||||
- **Client:** `PricingSettings` vendor filter and the show-all switch; the resume picker excluding codex sessions; the Codex badge.
|
||||
- **Browser:** an isolated server on a temp `CODEX_HOME` with copied real rollouts.
|
||||
@@ -0,0 +1,110 @@
|
||||
# Settings persistence + editable CODEX_HOME
|
||||
|
||||
Issue: #40 follow-up to part B. Date: 2026-09-29. Status: self-reviewed (user
|
||||
asked for autonomous work with self-review per part).
|
||||
|
||||
## Problem
|
||||
|
||||
`setClaudeHome` persists through `writeEnvFile`, which writes `<repo>/.env`.
|
||||
For a plugin install that directory is the plugin CACHE, replaced on every
|
||||
update, so a saved CLAUDE_HOME silently reverts. Two more faults ride along:
|
||||
|
||||
- The data dir (database, runtime, `ai-tools.json`) is `getDataDir()` =
|
||||
`<CLAUDE_HOME>/agenthub`, recomputed on every call. Changing CLAUDE_HOME at
|
||||
runtime moves every later `getDataDir()` caller mid-process, and the next
|
||||
start opens an EMPTY database under the new home — indistinguishable from
|
||||
data loss.
|
||||
- `writeEnvFile` appends a blank line on every save.
|
||||
|
||||
An editable CODEX_HOME (deferred from part B) has the same persistence problem,
|
||||
plus one of its own: the AI Router layer `<name>.config.toml` lives in
|
||||
`$CODEX_HOME`, so after a change a codex lane on a router would start with
|
||||
`-p <name>` against a home that has no such layer.
|
||||
|
||||
## Design
|
||||
|
||||
1. **A settings file outside both the repo and CLAUDE_HOME:**
|
||||
`$XDG_CONFIG_HOME/agenthub/settings.env` (default `~/.config/agenthub/`),
|
||||
overridable with `AGENTHUB_SETTINGS_FILE` (the server test runner points it
|
||||
at a temp path). Dir 0700, file 0600, atomic write, one `KEY=value` per
|
||||
line, no blank-line growth. New module `server/lib/settings-env.js`.
|
||||
2. **Load order at startup** (`server/index.js`, replacing `loadDotEnv`): the
|
||||
real environment wins, then `<repo>/.env`, then the settings file
|
||||
(`loadEnvFiles({owned: [.env], files: [settings]})`). The checkout's `.env`
|
||||
ranks above the per-user file because it is more specific: a dev checkout
|
||||
whose `.env` names its own `DASHBOARD_DATA_DIR` must not be pointed at the
|
||||
live database by a pin saved from the plugin dashboard. A key is only set
|
||||
when still unset, exactly as before. The keys the real environment or `.env`
|
||||
supplied are remembered (`envOwned`), so a route can say a saved value will
|
||||
be overridden at the next start. Keys that came from a file are named in
|
||||
`AGENTHUB_FILE_KEYS`: a self-restart spawns the successor with this
|
||||
process's env, and without the marker the successor would count them as the
|
||||
real environment's and warn falsely. A saved path may not contain a line
|
||||
break (it would inject a second `KEY=`).
|
||||
3. **Pin the data dir on the first CLAUDE_HOME change.** Before switching,
|
||||
`setClaudeHome` writes `DASHBOARD_DATA_DIR=<the data dir in use>` to the
|
||||
settings file and to `process.env` when neither already sets it. The
|
||||
database and runtime dir stay where the plugin bootstrap and the CLI (which
|
||||
never read the settings file) look for them. The discovery file
|
||||
(`<CLAUDE_HOME>/.agenthub.json`) is NOT pinned and follows CLAUDE_HOME, as it
|
||||
did before this change; hooks under a different CLAUDE_HOME then fall back to
|
||||
port 4820.
|
||||
4. **`validateHomeDir(label, input)`** in `settings-env.js` is the one path
|
||||
check both homes use: `~/` expansion, absolute, `path.resolve`, exists, is
|
||||
a directory, and neither IS nor CONTAINS the OS home (`EBADHOME`) — `/`,
|
||||
`/home`, `~` itself, or a symlink, bind mount or differently-cased path to
|
||||
any of them, compared by device and inode against the home and each of its
|
||||
ancestors.
|
||||
5. **`setCodexHome`** (`server/lib/codex-home.js`) and
|
||||
`GET/PUT /api/settings/codex-home`. PUT re-applies every AI Router profile a
|
||||
codex lane uses (`reapplyCodexLayers` in `router-apply.js`), so the layer
|
||||
exists in the new home, and answers
|
||||
`{ ok, codex_home, env_owned, routers: { updated, failed } }`.
|
||||
`GET/PUT /api/settings/claude-home` gain `env_owned` (additive).
|
||||
6. **UI:** a shared `HomeDirSetting` component renders CLAUDE_HOME (Claude Code
|
||||
→ AgentHub) and CODEX_HOME (new Codex → AgentHub sub-tab). It shows the
|
||||
env-owned warning, the router re-apply failures, and — for CODEX_HOME —
|
||||
that a running codex pane keeps its old home until restarted.
|
||||
|
||||
## Not changed
|
||||
|
||||
Existing `<repo>/.env` files keep working, and rank above the settings file: a
|
||||
key a `.env` sets is `env_owned`, so a value saved from the UI for it is
|
||||
overridden at the next start, and the UI says so. The file is per OS user, so every AgentHub server
|
||||
of that user (a plugin install and a dev checkout) reads the same one. Stores that live
|
||||
under CLAUDE_HOME by design (`agenthub-routers/`, `agenthub-profiles/`,
|
||||
`agenthub-git-credentials/`) still follow it. The CLI and bootstrap do not read
|
||||
the settings file; pinning the data dir is what keeps them consistent.
|
||||
|
||||
## Tests
|
||||
|
||||
Server: settings-env parse/load order/`envOwned`/write (no blank lines, 0600),
|
||||
`validateHomeDir` refusals, `setClaudeHome` pinning `DASHBOARD_DATA_DIR`,
|
||||
`setCodexHome` + the route (refusal 400, success, router re-apply called),
|
||||
OpenAPI entry. Client: `HomeDirSetting` save/error/env warning, the Codex
|
||||
sub-tab. Browser check on an isolated server.
|
||||
|
||||
## Found in the whole-change review, fixed
|
||||
|
||||
- **A dev checkout's `.env` lost to the plugin dashboard's pin.** The settings
|
||||
file ranked above `<repo>/.env`, so a `DASHBOARD_DATA_DIR` pinned from the
|
||||
plugin dashboard pointed a dev checkout's server at the live database. The
|
||||
order is now real environment > `.env` > settings file, and a `.env` key
|
||||
counts as `env_owned`.
|
||||
- **An old line could win.** `writeSetting` replaced only the first line for a
|
||||
key; it now drops every earlier one, `KEY = value` included, and writes
|
||||
through a symlinked settings file to its target.
|
||||
- **A case-insensitive or bind-mounted path to the home passed.**
|
||||
`validateHomeDir` now compares device and inode with the home and each
|
||||
ancestor.
|
||||
- **Another site could set a home.** `PUT /api/settings/claude-home` and
|
||||
`/codex-home` carry `sameOriginGuard` (403 `EBADORIGIN`): a home decides
|
||||
which config a pane runs.
|
||||
- **Lanes silently lost their router.** The router store is
|
||||
`<CLAUDE_HOME>/agenthub-routers/`, so a new home may lack a lane's profile;
|
||||
the claude-home PUT answers `routers: {missing: [...]}`
|
||||
(`lanesWithMissingRouter`) and the UI warns that those lanes start on native.
|
||||
- **Stale paths after a change.** The Claude hook status (`settings.js`),
|
||||
`scripts/install-hooks.js` and `scripts/import-history.js` read their
|
||||
CLAUDE_HOME-derived paths per call, not at load. `cc-watcher.js` still
|
||||
watches the startup home until a restart.
|
||||
Reference in New Issue
Block a user