AgentHub/docs/superpowers/specs/2026-09-29-codex-usage-design.md

Codex parity, part C1: sessions, token usage and pricing

Issue: #40 part C. Date: 2026-09-29. Status: self-reviewed (the user asked for autonomous work with a self-review per part). Part C2 (Codex hooks) is separate.

Problem

Codex emits no hook AgentHub listens to, so a Codex session never reaches the sessions table and its tokens cost nothing on any page. The pricing table and the automatic pricing sync know Claude models only.

What Codex writes (verified on 0.155, 178 real rollouts)

$CODEX_HOME/sessions/YYYY/MM/DD/rollout-<ts>-<thread id>.jsonl, append-only:

• Line 1 is session_meta, with these payload fields:

  • id: the thread.
  • session_id: the ROOT session. It is the same as id except in a sub-agent thread, whose thread_source is subagent. Older CLIs omit it.
  • cwd, timestamp, cli_version.
  • forked_from_id: set when the file is a fork.

• turn_context.payload.model is the model for the lines that follow. It can change mid-session, or be a router id such as clinepass/cline-pass/deepseek-v4-pro.

• event_msg with payload.type == "token_count" carries two things:

  • info.total_token_usage: a cumulative total for the thread.
  • info.last_token_usage: the delta.

  info may be null. The same total is often emitted twice in a row.

• Token fields:

  • input_tokens includes cached_input_tokens.
  • output_tokens includes reasoning_output_tokens.
  • cache_write_input_tokens is separate.

• A fork COPIES its parent's whole history into its own file, token_count events included, with rewritten timestamps. It then carries on from the parent's cumulative total. Summing the fork file naively counts the parent twice.

Design

1. sessions.tool is a new column: TEXT NOT NULL DEFAULT 'claude', added with the additive ALTER TABLE idiom the source column used. Every existing row and every hook-created row stays claude.
2. codex_rollouts is a new table recording how far each file has been read:
   • Columns: path PK, session_id (root), thread_id, size, mtime_ms, offset, state JSON, updated_at.
   • state holds the current model, the last total seen, the per-model usage sums, and the meta.
   • A file whose size and mtime are unchanged is not read again.
   • A grown file is read from offset up to its last newline.
   • A shrunken file is re-read from 0.
   • Because this table lives in the DB, a session the user deleted stays deleted until Codex writes to its file again, while wiping the DB re-imports everything.
3. server/lib/codex-usage.js does the ingest:

   • parseRolloutChunk(text, state, skip) is pure. It JSON-parses only the lines it needs, picked by five needles (NEEDLES): session_meta, turn_context, token_count, and a prompt as user_message (older CLIs) or UserMessage (0.149+). Each new total adds its last_token_usage to the current model, mapped as:

     • input = input − cached
     • cache_read = cached
     • cache_write = cache_write_input_tokens
     • output = output_tokens

     OpenAI bills reasoning as output.

   • A total equal to the previous one is skipped.

   • Forks: when a file with forked_from_id is read from offset 0, the parent's file is found by the -<id>.jsonl name. Every total vector in the parent is skipped in the child: the copied prefix matches the parent exactly, and the child's own totals never do. If the parent's file is gone, nothing is skipped and the overcount is logged.

   • ingestCodexUsage({ broadcast }):

     • Walks every rollout (no newest-40 cap) and yields to the event loop between files.
     • For each root session touched, it sums its threads' usage per model and writes it with stmts.setTokenUsage. The totals are exact recomputes, so the high-water baseline of replaceTokenUsage would be wrong here.
     • It then inserts or updates the sessions row:
       • tool='codex'
       • name = the thread_name from session_index.jsonl, else the first prompt (first line, 60 chars; user_message or the 0.149+ item_completed UserMessage), else Codex <8>
       • cwd and started_at from the root meta
       • model = the latest model
       • status = active while any of the session's files was written in the last 2 minutes, else completed, with ended_at = the newest journal's mtime
       • metadata {source:"codex-rollout"}
     • It never touches a row whose tool is not codex, and never sets transcript_path, so the Claude transcript parsers and sweeps skip these rows.
     • It broadcasts session_created/session_updated.
     • One pass at a time. It runs at startup (in the background) and then every DASHBOARD_CODEX_POLL_MS, the codex lane poll's own setting (0 turns both off).
4. Pricing sync learns OpenAI.
   • familyOf also accepts gpt-…, o<digit>… and codex-… ids, dropping a -YYYY-MM-DD suffix.
   • toRow takes litellm_provider == "openai" entries. A cache write costs cache_creation_input_token_cost or 0, since OpenAI has no cache-write charge by default. The fast rate is 0. The display name comes from displayName (gpt-5.6-sol → GPT-5.6 Sol).
   • Long-context (>272k) and priority/flex tiers are not modelled. Codex's context window stays under 272k.
   • POST /api/settings/reset-pricing starts a sync right after reseeding, so OpenAI rows wiped by a reset return at once.
5. Model Pricing in Codex → AgentHub.
   • The pricing section of ClaudeAgentHubSettings.tsx becomes components/settings/PricingSettings.tsx with a vendor prop. The Claude tab shows claude-… rows; the Codex tab shows OpenAI rows.
   • A Show all models switch shows every row, since one table prices every tool's sessions (a router model may serve either). A row added outside the filter turns the switch on, so it never vanishes from view.
   • Reset Defaults stays on the Claude tab only and sends {scope: "claude"}: it deletes and reseeds only claude-… rows, so OpenAI rows (and a custom router row) survive it; a reset with no body wipes every row, and OpenAI rows come back from the sync.
6. Claude-only surfaces exclude Codex sessions.
   • The Workspace's external sessions, which offer attach and claude --resume, list tool === 'claude' only.
   • The Sessions list and session detail show a Codex badge.

Found in the browser check, fixed

• The Claude liveness reap and the transcript watchdog (routes/hooks.js) probed Codex sessions for a claude process and reaped them: both now select tool = 'claude' only.
• The pricing sync had already run when the first ingest wrote new models; a pass that writes sessions now triggers a sync.

Found in the code review, fixed

• A journal met under a new path counted twice (CODEX_HOME moved, or a symlinked alias): there is now one codex_rollouts row per thread, and reading a file drops the thread's rows under other paths.
• The Claude resume picker offered Codex sessions (RunSetup.tsx), which claude --resume cannot open: it lists tool === 'claude' only.
• A failed price-list fetch was retried on every Codex pass (every 15 s): a failure now waits for the sync's own 15-minute check.
• A fork's copied history arriving after the first read was counted: the parent's totals are skipped until the fork's first own total (state.pastFork), on every read, not just the one from offset 0.
• A journal written in the future (a clock running ahead) kept its session active: it no longer counts as activity.
• A large journal blocked the event loop: one pass reads at most 16 MB of a file, and a fork's parent is streamed.
• A model gone from a re-read journal kept its old usage row: a Codex session's usage is rewritten whole.
• A sub-agent file read before its root fixed the session's cwd, start and model: the root thread's file sets them once it is read.
• The startup cleanup completed active Codex sessions and a Claude lane's purge deleted a Codex lane's sessions at the same directory: both are scoped by tool.
• The codex lane mirror flapped (codex-ingest.js, predates C1): a lane with an old journal beside the live one went running → idle → running on alternate passes, and a journal written in the current millisecond read as future. One verdict per lane now, with a second of slack.

Found in the whole-change review, fixed

• Hooks kept a session's status forever. A Codex session with any hook event kept its hook-given status even after the hooks were removed or no longer trusted, so it could stay active for good. Hooks now own the status only while current — hooksCurrent: the session's newest event is at or after its newest journal write minus 2 minutes — and otherwise the journal decides again, in settleActive too.
• Clear data stopped Codex (and Antigravity) sessions ever coming back. POST /api/settings/clear-data kept codex_rollouts, so a cleared session was never re-imported, or came back half on its next write: it now empties codex_rollouts and antigravity_conversations too.
• The per-thread lookup scanned the table: idx_codex_rollouts_thread indexes codex_rollouts(thread_id).
• A busy Codex lane pushed Claude rows out of a page: GET /api/sessions takes ?tool=, applied before the limit, and the Workspace attach list and the run form's resume picker ask for tool: "claude".
• Reset Defaults wiped the OpenAI rows a user had edited: the Claude tab's reset is scoped to claude-… rows (above).

Left as is: a journal rewritten to the same or a larger size is not detected (Codex only appends), and a fork without a Codex thread name is named after the prompt it copied from its parent.

Not changed

Hook ingestion, transcript-cache.js, import-history.js and remote sync still handle Claude Code only. Codex lanes keep their idle/running mirror in codex-ingest.js. Session↔lane binding is untouched.

Tests

• Server:
  • Chunk parser: mapping, duplicate totals, model switch, null info, partial last line.
  • Fork skip, with and without the parent file.
  • Sub-agent thread folded into the root session.
  • Incremental read after an append; a shrunken file re-read.
  • An unchanged file not re-read; a deleted session not resurrected.
  • A claude row never touched.
  • Pricing sync for an OpenAI family.
  • Reset triggers a sync.
• Client: PricingSettings vendor filter and the show-all switch; the resume picker excluding codex sessions; the Codex badge.
• Browser: an isolated server on a temp CODEX_HOME with copied real rollouts.