AgentHub/docs/DATABASE.md

# Database Schema Reference

Comprehensive database schema documentation for Agent Dashboard SQLite database.

---

## Table of Contents

- [Overview](#overview)
- [Schema Diagram](#schema-diagram)
- [Table Definitions](#table-definitions)
- [Indexes](#indexes)
- [Migrations](#migrations)
- [Query Patterns](#query-patterns)
- [Performance Optimization](#performance-optimization)
- [Data Integrity](#data-integrity)
- [Backup Strategies](#backup-strategies)

---

## Overview

Agent Dashboard uses **SQLite 3** as its primary data store with the following characteristics:

- **File-based** - Single database file, portable across systems
- **Embedded** - No separate server process required
- **ACID compliant** - Transactions ensure data integrity
- **WAL mode** - Write-Ahead Logging for better concurrency
- **Prepared statements** - Prevent SQL injection, optimize performance

```mermaid
graph TB
    subgraph "Database File"
        DB[(dashboard.db)]
    end
    
    subgraph "Tables"
        Sessions[sessions]
        Agents[agents]
        Tools[tool_executions]
        Notifs[notifications]
        Pricing[pricing_rules]
        Remote[remote_sources]
    end
    
    subgraph "Indexes"
        Idx1[session_id, status, updated_at]
        Idx2[agent_id, session_id, status]
        Idx3[agent_id, created_at]
    end
    
    DB --> Sessions
    DB --> Agents
    DB --> Tools
    DB --> Notifs
    DB --> Pricing
    DB --> Remote
    
    Sessions --> Idx1
    Agents --> Idx2
    Tools --> Idx3
    
    style DB fill:#003B57,color:#fff
```

**Database Location:**
- **Canonical (default):** `~/.claude/agenthub/dashboard.db` — shared by `npm start`, `npm run dev`, and the desktop app when it uses the same data dir
- **Override:** set `DASHBOARD_DATA_DIR` (directory) or `DASHBOARD_DB_PATH` (file path) for tests or custom deployments
- **Pinned on a CLAUDE_HOME change:** the data dir defaults to `<CLAUDE_HOME>/agenthub`, so saving a new CLAUDE_HOME from Settings → Claude Code → Overview first writes the data dir in use as `DASHBOARD_DATA_DIR` to AgentHub's settings file (`~/.config/agenthub/settings.env`, or `$XDG_CONFIG_HOME/agenthub/settings.env`, or `AGENTHUB_SETTINGS_FILE`). The next start keeps the same database instead of opening an empty one under the new home. An explicit `DASHBOARD_DATA_DIR` is left alone
- **Legacy:** repo-local `./data/dashboard.db` is migrated into the canonical location on first launch (see `server/db.js`)
- **Pre-rebrand:** an install that predates AgentHub keeps its data in `~/.claude/agent-dashboard/`. Server startup renames that directory to `~/.claude/agenthub/` once, before anything opens the database (`migrateLegacyDataDir()` in `server/lib/claude-home.js`). It is a same-filesystem rename, so it is instant even at multiple gigabytes and never leaves two copies. If the rename fails, or `DASHBOARD_DATA_DIR` is set, nothing moves and the old directory keeps being used.

---

## Schema Diagram

### Entity-Relationship Diagram

```mermaid
erDiagram
    sessions ||--o{ agents : "has many"
    agents ||--o{ tool_executions : "has many"
    sessions ||--o{ notifications : "has many"
    remote_sources ||--o{ sessions : "tags (source)"
    
    sessions {
        integer id PK "Primary key"
        text session_id UK "Unique session identifier"
        text model "Raw model slug (e.g., claude-sonnet-4-5-20250514); UI displays via formatModelName()"
        text status "active | completed"
        real total_cost "Aggregated cost from all agents"
        text source "'local' or a remote_sources.id"
        text created_at "ISO8601 timestamp"
        text updated_at "ISO8601 timestamp (bumped on every hook)"
    }
    
    agents {
        integer id PK "Primary key"
        text agent_id UK "Unique agent identifier"
        text session_id FK "Foreign key to sessions"
        text agent_type "explore, task, general-purpose, etc."
        text status "running | completed | failed"
        text current_tool "Currently executing tool (or NULL)"
        integer input_tokens "Cumulative input tokens"
        integer output_tokens "Cumulative output tokens"
        real cost "Calculated cost for this agent"
        text created_at "ISO8601 timestamp"
        text updated_at "ISO8601 timestamp"
    }
    
    tool_executions {
        integer id PK "Primary key"
        text agent_id FK "Foreign key to agents"
        text tool_name "bash, view, edit, grep, etc."
        integer duration_ms "Execution time in milliseconds"
        integer success "1 = success, 0 = failure"
        text error_message "NULL if success, error details if failed"
        text created_at "ISO8601 timestamp"
    }
    
    notifications {
        integer id PK "Primary key"
        text session_id FK "Foreign key to sessions"
        text notification_type "backgroundTaskComplete, etc."
        text message "Notification message"
        text created_at "ISO8601 timestamp"
    }
    
    pricing_rules {
        integer id PK "Primary key"
        text pattern UK "Model pattern (e.g., claude-sonnet-4)"
        real input_cost_per_1m "Input cost per 1M tokens (USD)"
        real output_cost_per_1m "Output cost per 1M tokens (USD)"
        text created_at "ISO8601 timestamp"
    }

    remote_sources {
        text id PK "Remote-source id (also used as sessions.source)"
        text label "Human-readable name"
        text host "SSH destination user@host or ~/.ssh/config alias"
        integer ssh_port "Optional SSH port (NULL = SSH default)"
        text identity_file "Optional private-key path (NULL = SSH default)"
        text remote_home "Optional remote Claude home (NULL = remote ~/.claude)"
        integer enabled "1 = eligible for sync, 0 = disabled"
        text status "idle | syncing | ok | error"
        text last_error "Last failure message, or NULL"
        text last_sync_at "ISO8601 timestamp of last successful sync, or NULL"
        text last_sync_counts "JSON blob of last sync counters, or NULL"
        text created_at "ISO8601 timestamp"
        text updated_at "ISO8601 timestamp"
    }
```

### Relationship Cardinality

```mermaid
graph LR
    Session[Session<br/>1] -->|1:N| Agents[Agents<br/>N]
    Session -->|1:N| Notifications[Notifications<br/>N]
    Agents -->|1:N| Tools[Tool Executions<br/>N]
    
    style Session fill:#3B82F6
    style Agents fill:#10B981
    style Tools fill:#F59E0B
```

---

## Table Definitions

### sessions

Tracks Claude Code sessions (one per CLI invocation or background task). Schema mirrors `server/db.js`.

> **Cursor (informational):** Rows imported from `~/.claude` JSONL transcripts may also represent **Cursor** agent sessions — Cursor happens to use the same on-disk layout as Claude Code, so such a row still reads `tool = 'claude'`.

```sql
CREATE TABLE sessions (
    id TEXT PRIMARY KEY,                                              -- UUID from Claude Code
    name TEXT,
    status TEXT NOT NULL DEFAULT 'active'
        CHECK (status IN ('active','completed','error','abandoned')),
    cwd TEXT,
    model TEXT,
    started_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now')),
    ended_at TEXT,
    metadata TEXT,
    updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now')),
    awaiting_input_since TEXT,                                        -- NULL unless Waiting
    awaiting_reason TEXT,                                             -- notification|stop|session_start|interrupted, or NULL
    transcript_path TEXT,                                             -- absolute path to JSONL transcript
    source TEXT NOT NULL DEFAULT 'local',                             -- data source: 'local' or a remote_sources.id
    tool TEXT NOT NULL DEFAULT 'claude'                               -- 'claude', 'codex' or 'antigravity'
);
```

**Columns:**

| Column | Type | Nullable | Description |
|--------|------|----------|-------------|
| `id` | TEXT | NO | Session UUID (assigned by Claude Code) |
| `name` | TEXT | YES | Human-readable label. Synced from the transcript title by `routes/hooks.js` (and the 15 s watchdog) on every event: the `custom-title` line (`/rename`, `claude -n`, picker `Ctrl+R`) always wins, otherwise the auto-generated `ai-title` fills a placeholder/auto name, otherwise the session's first user prompt (60-char label) fills it. Falls back to `Session <id8>` |
| `status` | TEXT | NO | `active`, `completed`, `error`, or `abandoned` (CHECK-constrained). Besides the `SessionEnd` hook, the 15 s watchdog's **liveness reap** also lands `active` → `completed` when no running `claude` process has the session's `cwd` (a `SessionEnd` lost while the dashboard was down); gated by `DASHBOARD_LIVENESS_IDLE_SECONDS`, disabled via `DASHBOARD_LIVENESS_PROBE=0`. Sessions with a non-`local` `source` (Remote Data Sources) are exempt from the reap and both stale sweeps — their status is reconciled from the SSH mirror by `remote-sync.js` instead |
| `cwd` | TEXT | YES | Working directory the CLI was launched from |
| `model` | TEXT | YES | Model ID as the CLI reports it — a Claude id (e.g. `claude-opus-4-7`), an OpenAI one for a Codex session (`gpt-…`), a Gemini one for an Antigravity session (`gemini-…`) |
| `started_at` | TEXT | NO | ISO 8601 timestamp |
| `ended_at` | TEXT | YES | ISO 8601 timestamp on terminal transition |
| `metadata` | TEXT | YES | JSON blob for extras (turn duration totals, thinking blocks, …) |
| `updated_at` | TEXT | NO | Bumped on every event for staleness detection |
| `awaiting_input_since` | TEXT | YES | ISO 8601 stamp set when the session is **Waiting** (Stop, SessionStart with source `startup`/`resume`/`clear`, permission Notification, or watchdog user-interrupt/Esc recovery). NULL otherwise. A SessionStart with source `compact` (auto-compaction fires mid-turn while Claude is working) leaves this column untouched, so a genuinely-active session is not mislabeled Waiting |
| `awaiting_reason` | TEXT | YES | Why the row is waiting: `notification`, `stop`, `session_start`, or `interrupted`. Set/cleared in lock-step with `awaiting_input_since` (SessionStart→`session_start`, Stop→`stop`, permission/input Notification→`notification`, watchdog/Esc recovery→`interrupted`). NULL otherwise. Exception: a `compact`-source SessionStart preserves the existing value (neither stamps `session_start` nor clears it) |
| `transcript_path` | TEXT | YES | Absolute path to the session's JSONL transcript. Written by `routes/hooks.js` on the first event that carries it (subsequent events no-op via a SQL guard) and read by the periodic compaction sweep — so the sweep touches only active session rows instead of scanning the entire `events` table for `json_extract(data,'$.transcript_path')`. Backfilled once from `events` by the `db.js` migration |
| `source` | TEXT | NO | Data source this session was captured from. `'local'` for this machine's own Claude Code history (the default); otherwise the `remote_sources.id` of the remote SSH machine it was pulled from. Powers the `sources` query filter on `/api/sessions`, `/api/events`, `/api/agents`, `/api/stats`, and `/api/analytics`, and the `sources` facet on `/api/sessions/facets`. Indexed by `idx_sessions_source` |
| `tool` | TEXT | NO | The AI coding CLI that produced the session. `'claude'` for every hook-created and imported row (the default); `'codex'` for a session ingested from Codex's rollout journals by `server/lib/codex-usage.js` (see `codex_rollouts`) or created by a Codex hook (`tool: "codex"`, `server/lib/codex-hooks.js`) — a Codex session keeps the status its hooks give it while they are current (its newest event at or after its newest journal write minus two minutes); once they stop (removed, untrusted) the journal decides again. A codex row has no `transcript_path`, and, going by the journal, its `status` is `active` only while one of its journals was written in the last two minutes. `'antigravity'` for an agy CLI conversation, ingested by `server/lib/antigravity-usage.js` (see `antigravity_conversations`) or created by an Antigravity hook (`server/lib/antigravity-hooks.js`); no `transcript_path`, and `active` until it has been quiet — no write to its conversation file (or a non-empty `-wal`) or transcript, no hook event — for ten minutes when it has hook events, two minutes when it has none; it then settles to `completed`, its agents too. The startup cleanup of stale `active` sessions and a workspace purge touch only the matching tool's rows (a purge takes the sessions of the workspace's own `tool`). The Workspace's attach/resume list offers `'claude'` rows only |
| `lane_id` | INTEGER | YES | The workspace that OWNS the session — the only sessions a workspace lists: the lane whose dashboard pane held it, stamped by `bindRunSession` (`server/routes/hooks.js`) from the hook's `run_id` — first writer wins, a nested `claude -p` excluded. NULL for a session no workspace started (a terminal, an import, a tool with no hooks). No FK: removing a workspace leaves its sessions. Backfilled once from `dashboard_runs`. Indexed by `idx_sessions_lane` |
| `archived_at` | TEXT | YES | When the user archived the session out of its workspace's lists (`PATCH /api/sessions/:id {archived}`). Hides only — nothing is deleted |

**Constraints:**
- `status` must be one of the four enum values
- `awaiting_input_since` is ignored on non-`active` sessions for UI bucketing

**Lifecycle:**

```mermaid
stateDiagram-v2
    [*] --> waiting: SessionStart startup/resume/clear (status=active + awaiting_input_since)
    active --> active: SessionStart compact (mid-turn — state preserved)
    waiting --> active: UserPromptSubmit / PreToolUse / PostToolUse
    active --> waiting: Stop (non-error) / Permission Notification
    active --> waiting: Esc cancel (watchdog marker or idle timeout)
    active --> error: Stop (stop_reason=error)
    waiting --> completed: SessionEnd
    active --> completed: SessionEnd
    waiting --> abandoned: Stale > DASHBOARD_STALE_MINUTES
    active --> abandoned: Stale > DASHBOARD_STALE_MINUTES
    completed --> active: Resumed
    error --> active: Resumed
    abandoned --> active: Resumed
    completed --> [*]
    error --> [*]
    abandoned --> [*]
```

---

### agents

Tracks main agents and subagents within a session. Main agents have id `${session_id}-main`; subagents get a fresh UUID.

```sql
CREATE TABLE agents (
    id TEXT PRIMARY KEY,
    session_id TEXT NOT NULL,
    name TEXT NOT NULL,
    type TEXT NOT NULL DEFAULT 'main' CHECK (type IN ('main','subagent')),
    subagent_type TEXT,
    status TEXT NOT NULL DEFAULT 'idle'
        CHECK (status IN ('idle','connected','working','completed','error')),
    task TEXT,
    current_tool TEXT,
    started_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now')),
    ended_at TEXT,
    parent_agent_id TEXT,
    metadata TEXT,
    updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now')),
    awaiting_input_since TEXT,                                        -- main-agent waiting flag
    awaiting_reason TEXT,                                             -- notification|stop|session_start|interrupted, or NULL
    FOREIGN KEY (session_id) REFERENCES sessions(id) ON DELETE CASCADE,
    FOREIGN KEY (parent_agent_id) REFERENCES agents(id) ON DELETE SET NULL
);
```

**Columns:**

| Column | Type | Nullable | Description |
|--------|------|----------|-------------|
| `id` | TEXT | NO | UUID (subagents) or `${session_id}-main` (main agent) |
| `session_id` | TEXT | NO | FK to `sessions.id`, cascades on delete |
| `name` | TEXT | NO | Display label (e.g. `Main Agent - {session name}` or subagent description) |
| `type` | TEXT | NO | `main` or `subagent` |
| `subagent_type` | TEXT | YES | `Explore`, `general-purpose`, `code-review`, `compaction`, … |
| `status` | TEXT | NO | `idle`, `connected`, `working`, `completed`, `error` (CHECK-constrained). The dashboard's **Waiting** badge is the UI overlay produced by `awaiting_input_since`; it is not a persisted status |
| `task` | TEXT | YES | Subagent prompt / brief |
| `current_tool` | TEXT | YES | Tool currently running (cleared on `PostToolUse`) |
| `parent_agent_id` | TEXT | YES | FK to the spawning agent for nested subagent trees (`ON DELETE SET NULL`). Set to the main agent at insert, then repointed to the true spawner by `reconcileSubagentParents` from each subagent transcript's Task tool result (`toolUseResult.agentId`), so subagents-of-subagents nest correctly instead of flattening under main |
| `metadata` | TEXT | YES | JSON blob for extras. For subagents it carries `model` (the subagent's own model, issue #185) and `tokens` — an array of per-agent token buckets parsed from the subagent's transcript. The agent-list endpoints price `tokens` at the current rates to attach a per-agent `cost` (so a subagent card shows its OWN cost, not the session total). Empty `[]` means the subagent did no billable work; absent means its transcript wasn't available to parse |
| `awaiting_input_since` | TEXT | YES | Mirrors the parent session's flag for the main agent. NULL on subagents |
| `awaiting_reason` | TEXT | YES | Why the row is waiting: `notification`, `stop`, `session_start`, or `interrupted`. Set/cleared in lock-step with `awaiting_input_since`; explains why the main agent is waiting. NULL on subagents |

**Lifecycle:**

```mermaid
stateDiagram-v2
    [*] --> Running: Agent created (SessionStart/PreToolUse)
    Running --> Running: PreToolUse (set current_tool)
    Running --> Running: PostToolUse (increment tokens, cost)
    Running --> Completed: Stop/SubagentStop hook
    Running --> Failed: Error during processing
    Completed --> [*]
    Failed --> [*]
```

**current_tool Behavior:**
- Set to tool name on `PreToolUse` hook (e.g., `"bash"`, `"view"`)
- Cleared to `NULL` on `PostToolUse` hook
- Used to show real-time tool execution in UI

---

### tool_executions

Records each tool call made by agents.

```sql
CREATE TABLE tool_executions (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    agent_id TEXT NOT NULL,
    tool_name TEXT NOT NULL,
    duration_ms INTEGER,
    success INTEGER DEFAULT 1,
    error_message TEXT,
    created_at TEXT DEFAULT (datetime('now')),
    FOREIGN KEY (agent_id) REFERENCES agents(agent_id)
);
```

**Columns:**

| Column | Type | Nullable | Description |
|--------|------|----------|-------------|
| `id` | INTEGER | NO | Auto-increment primary key |
| `agent_id` | TEXT | NO | Foreign key to `agents.agent_id` |
| `tool_name` | TEXT | NO | Tool name (`bash`, `view`, `edit`, `grep`, etc.) |
| `duration_ms` | INTEGER | YES | Execution time in milliseconds |
| `success` | INTEGER | NO | 1 = success, 0 = failure |
| `error_message` | TEXT | YES | NULL if success, error details if failed |
| `created_at` | TEXT | NO | ISO8601 timestamp of execution |

**Common Tool Names:**
- `bash` - Shell command execution
- `view` - File/directory viewing
- `edit` - File editing
- `grep` - Code search
- `glob` - File pattern matching
- `task` - Sub-agent invocation
- `sql` - SQLite query execution

**Duration Distribution:**

```mermaid
graph TB
    Tools[Tool Executions] --> Fast[Fast<br/>&lt; 100ms<br/>view, grep]
    Tools --> Medium[Medium<br/>100ms - 1s<br/>edit, bash]
    Tools --> Slow[Slow<br/>&gt; 1s<br/>task, build commands]
    
    style Fast fill:#10B981
    style Medium fill:#F59E0B
    style Slow fill:#EF4444
```

---

### notifications

Stores system notifications from Claude Code.

```sql
CREATE TABLE notifications (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    session_id TEXT NOT NULL,
    notification_type TEXT NOT NULL,
    message TEXT,
    created_at TEXT DEFAULT (datetime('now')),
    FOREIGN KEY (session_id) REFERENCES sessions(session_id)
);
```

**Columns:**

| Column | Type | Nullable | Description |
|--------|------|----------|-------------|
| `id` | INTEGER | NO | Auto-increment primary key |
| `session_id` | TEXT | NO | Foreign key to `sessions.session_id` |
| `notification_type` | TEXT | NO | Type of notification |
| `message` | TEXT | YES | Notification message content |
| `created_at` | TEXT | NO | ISO8601 timestamp |

**Common Notification Types:**
- `backgroundTaskComplete` - Background agent finished
- `errorOccurred` - Error during execution
- `systemMessage` - General system message

---

### model_pricing

Per-model pricing rules for cost calculation, keyed by `model_pattern` (a SQL-style glob; `%` matches any characters). Rates are per **million** tokens (USD). The longest matching pattern wins.

**Automatic rates.** `server/lib/pricing-sync.js` runs at server start and every 15 minutes, and fetches LiteLLM's public price list (`model_prices_and_context_window.json` on GitHub) at most once a day. Each pass (1) INSERTS a row, `source = 'sync'`, for every Claude, OpenAI or Gemini model seen in `token_usage` whose family (the id minus a `-YYYYMMDD` / `-YYYY-MM-DD` suffix; OpenAI means `gpt-…`, `o<n>…` or `codex-…`; Gemini means `gemini-…`, taken whole) has no `<family>%` row of its own, AND for every **current** model of the three vendors before anyone uses it — a list key that is a family itself, `mode` `chat`/`responses`, not past its `deprecation_date`, not an audio/search/TTS/realtime/image/embedding variant, and of the current generation (any `claude-<name>-…`; `gpt-` 5 and up; `gemini-` 3 and up — `MIN_GEN`); (2) UPDATES the rates of its own `sync` rows when the list changes; (3) adopts a row from before the `source` column (`NULL`: a seed, an old sync insert or an old edit) as `sync` only while its rates still equal the list exactly. A row edited through `PUT /api/pricing` becomes `source = 'user'` and is never touched again; a deleted row's pattern goes to `model_pricing_dismissed` and is never added back. An entry is taken only from its own vendor (`litellm_provider` `anthropic` / `openai`, and `vertex_ai-language-models` or `gemini` for Gemini); an OpenAI or Gemini row carries no cache-write charge unless the list names one, and no fast rate. Gemini's >200k-token rate is not modelled (the base rate is used). A router's model id (`clinepass/…`) is never priced automatically. `POST /api/settings/reset-pricing` empties `model_pricing_dismissed` for its scope and runs a sync right after reseeding (waiting up to 3 s), so the OpenAI and Gemini rows a reset wiped come straight back; with `{scope: "claude"}` (the Claude Code tab's **Reset Defaults**) it deletes and reseeds only `claude-…` rows, so other vendors' rows and a custom router row survive. A rate outside 0–1000 $/MTok is rejected, and a failed fetch changes nothing. Set `AGENTHUB_PRICING_SYNC=0` (or `false`/`off`) to turn it off.

```sql
CREATE TABLE model_pricing (
    model_pattern TEXT PRIMARY KEY,
    display_name TEXT NOT NULL,
    input_per_mtok REAL NOT NULL DEFAULT 0,
    output_per_mtok REAL NOT NULL DEFAULT 0,
    cache_read_per_mtok REAL NOT NULL DEFAULT 0,
    cache_write_per_mtok REAL NOT NULL DEFAULT 0,
    cache_write_1h_per_mtok REAL NOT NULL DEFAULT 0,   -- 1h-ephemeral cache-write tier
    fast_input_per_mtok REAL NOT NULL DEFAULT 0,       -- fast-mode premium rates
    fast_output_per_mtok REAL NOT NULL DEFAULT 0,
    -- Time-limited introductory (promo) rates. When intro_until is set, usage on
    -- or before that date (YYYY-MM-DD) is priced at the intro_* rates and usage
    -- after it at the standard rates. All 0 / NULL = no promo.
    intro_input_per_mtok REAL NOT NULL DEFAULT 0,
    intro_output_per_mtok REAL NOT NULL DEFAULT 0,
    intro_cache_read_per_mtok REAL NOT NULL DEFAULT 0,
    intro_cache_write_per_mtok REAL NOT NULL DEFAULT 0,
    intro_cache_write_1h_per_mtok REAL NOT NULL DEFAULT 0,
    intro_until TEXT,                                   -- promo cutoff YYYY-MM-DD, or NULL
    source TEXT,                                        -- 'sync' | 'user' | NULL (from before the column)
    updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now'))
);
```

**Columns (highlights):**

| Column | Type | Nullable | Description |
|--------|------|----------|-------------|
| `model_pattern` | TEXT | NO | Primary key. SQL-style glob (e.g. `claude-opus-4-7%`, `claude-%-haiku`). Rules are matched longest-pattern-first |
| `display_name` | TEXT | NO | Human-readable model name shown in Settings |
| `input_per_mtok` / `output_per_mtok` | REAL | NO | Standard input / output rate per 1M tokens |
| `cache_read_per_mtok` / `cache_write_per_mtok` / `cache_write_1h_per_mtok` | REAL | NO | Cache read + 5m/1h cache-write rates |
| `fast_input_per_mtok` / `fast_output_per_mtok` | REAL | NO | Fast-mode premium rates (0 = no premium) |
| `intro_*_per_mtok` | REAL | NO | Introductory (promo) rates, mirroring the standard fields |
| `intro_until` | TEXT | YES | Promo cutoff `YYYY-MM-DD`. Usage on/before it uses the intro rates; NULL = no promo. Editable per-rule in Settings |
| `source` | TEXT | YES | `sync` = kept at LiteLLM's rate by the automatic sync (shown **Auto** in Settings), `user` = edited in Settings and never touched by the sync, NULL = from before this column (adopted as `sync` only while it equals the list) |
| `updated_at` | TEXT | NO | ISO8601 timestamp of the last edit |

Standard rates and intro rates are edited independently: the pricing update path writes intro columns only when the caller sends intro fields, so a standard-rate edit never disturbs a promo (and vice versa). Clearing `intro_until` also zeroes the intro rates.

`model_pricing_dismissed (model_pattern TEXT PRIMARY KEY)` holds the pattern of every rule deleted through `DELETE /api/pricing/:pattern`, so the sync never adds it back; a reset empties it for the scope it reseeds.

**Example default rules:**

| Pattern | Input | Output | Cache read | Fast in/out |
|---------|-------|--------|------------|-------------|
| `claude-opus-5%` | $5.00 | $25.00 | $0.50 | $10 / $50 |
| `claude-sonnet-5%` | $2.00 | $10.00 | $0.20 | — |
| `claude-fable-5-1%` | $10.00 | $50.00 | $0.25 | — |

No rule ships an intro rate today: Sonnet 5's launch promo ran through `2026-08-31` and its discounted rate became the standard one, so `DEFAULT_INTRO_PRICING` is empty. `claude-fable-5-1%` exists alongside `claude-fable-5%` only because 5.1 caches cheaper — that overlap is what the longest-pattern-first match rule above resolves.

**Retired models.** Claude 3.x, Opus 4/4.1 and Sonnet 4 were dropped from the defaults. Because the startup top-up only *adds* missing rows, those rows are also deleted once from existing databases, guarded by `PRAGMA user_version` (0 → 1) so a model you deliberately re-add in Settings survives later restarts. The same one-shot migration corrects the pre-existing Sonnet 5 row to its post-promo $2 / $10 standard rate. Deleting a rule never touches `token_usage` — only the rates used to price it.

---

### remote_sources

Config for remote SSH machines the dashboard pulls Claude Code history from, so a single dashboard can consolidate sessions from several machines. **No secrets are stored** — SSH authentication defers entirely to the host's SSH stack (ssh-agent, `~/.ssh/config`, key files). Each row's `id` is used as the `source` value on every session imported from that machine (see `sessions.source`).

```sql
CREATE TABLE remote_sources (
    id TEXT PRIMARY KEY,
    label TEXT NOT NULL,
    host TEXT NOT NULL,
    ssh_port INTEGER,
    identity_file TEXT,
    remote_home TEXT,
    enabled INTEGER NOT NULL DEFAULT 1,
    status TEXT NOT NULL DEFAULT 'idle'
        CHECK (status IN ('idle','syncing','ok','error')),
    last_error TEXT,
    last_sync_at TEXT,
    last_sync_counts TEXT,
    created_at TEXT DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now')),
    updated_at TEXT DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now'))
);
```

**Columns:**

| Column | Type | Nullable | Description |
|--------|------|----------|-------------|
| `id` | TEXT | NO | Primary key. Also used as `sessions.source` for sessions pulled from this machine |
| `label` | TEXT | NO | Human-readable name shown in the UI |
| `host` | TEXT | NO | SSH destination (`user@host`) or a `~/.ssh/config` alias |
| `ssh_port` | INTEGER | YES | Optional SSH port; NULL defers to the SSH default / `~/.ssh/config` |
| `identity_file` | TEXT | YES | Optional private-key path passed to ssh (`-i`); NULL to omit |
| `remote_home` | TEXT | YES | Optional remote Claude home to read transcripts from; NULL defaults to remote `~/.claude` |
| `enabled` | INTEGER | NO | `1` = eligible for scheduled/manual syncs, `0` = disabled (default `1`) |
| `status` | TEXT | NO | Last sync status: `idle`, `syncing`, `ok`, or `error` (CHECK-constrained) |
| `last_error` | TEXT | YES | Error message from the last failed sync/test, or NULL |
| `last_sync_at` | TEXT | YES | ISO 8601 timestamp of the last successful sync, or NULL |
| `last_sync_counts` | TEXT | YES | JSON blob of the last sync's counters (imported/skipped/backfilled/errors/sessions_seen/sessions_tagged), or NULL |
| `created_at` | TEXT | YES | ISO 8601 creation timestamp |
| `updated_at` | TEXT | YES | ISO 8601 timestamp of the last edit |

Managed through the `/api/remote-sources/*` routes; sync/status changes are broadcast over the WebSocket as `remote_source.status` and, on success, `remote_data.updated` plus per-session `session_created` / `session_updated`. See [docs/API.md → Remote Data Sources](./API.md#remote-data-sources).

---

### token_usage_hourly

The same buckets as `token_usage` (session, model, speed, inference_geo, service_tier and the eight count columns, no baselines), split by `hour` — the UTC hour `YYYY-MM-DDTHH` a Claude transcript message was written in. The hook writer fills it from `TranscriptCache`'s `tokensByHour` on every transcript-bearing event (`writeHourly` in `server/lib/hourly-usage.js`, `stmts.upsertHourlyUsage`, MAX per column so a rewritten transcript never takes an hour back), and the server back-fills sessions that have tokens but no hourly rows a few seconds after each start (`backfillHourly`, one transcript per tick). Only the daily split of `GET /api/pricing/cost` reads it; `token_usage` stays the total, and what it holds beyond these rows is booked to the session's start date. Cascades with its session.

---

### codex_rollouts

How far `server/lib/codex-usage.js` has read each Codex rollout journal (`$CODEX_HOME/sessions/YYYY/MM/DD/rollout-<ts>-<thread>.jsonl`). A file whose `size` and `mtime_ms` are unchanged is not read again; a grown one is read from `offset` to its last newline, at most 16 MB per pass (`state.more` marks a file with more to read); a shrunken one from 0. There is one row per `thread_id`: the same journal met under a new path (CODEX_HOME moved, or reached through a symlink) replaces its old row instead of being counted twice. Because it lives in the DB, a Codex session the user deleted stays gone until Codex writes to that journal again, while a fresh database re-imports every journal. `POST /api/settings/clear-data` empties this table (and `antigravity_conversations`) with the sessions, so a cleared session is imported again whole instead of never, or half on its next write.

```sql
CREATE TABLE codex_rollouts (
    path TEXT PRIMARY KEY,          -- absolute journal path
    session_id TEXT NOT NULL,       -- ROOT session (a sub-agent thread's parent)
    thread_id TEXT NOT NULL,        -- the journal's own thread
    size INTEGER NOT NULL DEFAULT 0,
    mtime_ms INTEGER NOT NULL DEFAULT 0,
    offset INTEGER NOT NULL DEFAULT 0,
    state TEXT NOT NULL DEFAULT '{}', -- JSON: meta, current model, last total, per-model usage sums
    updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now'))
);
CREATE INDEX idx_codex_rollouts_session ON codex_rollouts(session_id);
CREATE INDEX idx_codex_rollouts_thread ON codex_rollouts(thread_id);
```

A session's `token_usage` rows are the sum of `state.usage` over every row with its `session_id`, rewritten whole on every change (delete, then `stmts.setTokenUsage`), bucketed `standard`/`global`/`standard`.

### antigravity_conversations

Which Antigravity CLI conversation files `server/lib/antigravity-usage.js` has read (`~/.gemini/antigravity-cli/conversations/<conversation id>.db`, agy's own SQLite), at what mtime — the newer of the file's and its non-empty `-wal`'s (an empty one is what AgentHub's own read-only open leaves). A file whose mtime is unchanged is not opened again. One that is not agy's (`SQLITE_NOTADB`, `SQLITE_CORRUPT`, `no such table`) is recorded with `session_id` `''`, so it is tried again only once it changes or at the next server start (`startAntigravityUsagePoll` deletes the `''` rows); any other read failure (a busy file, AgentHub's own error) is recorded nowhere and retried on the next pass; a changed one is re-read whole (they are small) and its session's `token_usage` rewritten whole, per model (`gen_metadata`'s `ModelUsageStats`: input without cache reads, output with thinking, cache read, cache write). As with `codex_rollouts`, a session the user deleted stays gone until agy writes to that conversation again.

```sql
CREATE TABLE antigravity_conversations (
    path TEXT PRIMARY KEY,          -- absolute conversation file path
    session_id TEXT NOT NULL,       -- the file name's id = the hooks' conversationId; '' = not agy's file
    mtime_ms INTEGER NOT NULL DEFAULT 0,
    updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now'))
);
```

### lane_sources (and `lanes.instructions`)

A workspace's own context, as on its home page (`server/lib/lane-sources.js`). `lanes.instructions` (TEXT, nullable, ≤ 16 000 characters) is appended to every new pane's system prompt; `lane_sources` holds its extra **folders** (the user's directories, never deleted by AgentHub) and uploaded context **files** (AgentHub's own copies under `<data dir>/workspace-context/<lane id>/`, deleted with the row and with the workspace). Both are read at spawn from these rows only (`launchExtras`), never from a request. Both are additive migrations; the rows cascade when the lane row is deleted.

```sql
CREATE TABLE lane_sources (
    id INTEGER PRIMARY KEY,
    lane_id INTEGER NOT NULL REFERENCES lanes(id) ON DELETE CASCADE,
    kind TEXT NOT NULL CHECK (kind IN ('folder', 'file')),
    path TEXT NOT NULL,     -- real path of the folder, or AgentHub's copy of the file
    name TEXT NOT NULL,     -- folder basename / the stored file name
    size INTEGER,           -- bytes, files only
    added_at TEXT NOT NULL
);
CREATE INDEX idx_lane_sources_lane ON lane_sources(lane_id);
```

---

## Indexes

### sessions Indexes

```sql
CREATE INDEX idx_sessions_session_id ON sessions(session_id);
CREATE INDEX idx_sessions_status ON sessions(status);
CREATE INDEX idx_sessions_updated_at ON sessions(updated_at DESC);
CREATE INDEX idx_sessions_source ON sessions(source);   -- powers the `sources` query filter

-- Partial index covering only the rows the periodic compaction sweep reads:
-- active sessions with a known transcript_path. Writes to other sessions skip
-- the index entirely, so the maintenance cost stays bounded by the small set
-- of live sessions.
CREATE INDEX idx_sessions_active_tp
    ON sessions(status, transcript_path)
    WHERE status='active' AND transcript_path IS NOT NULL;
```

**Query Patterns:**
- `SELECT * FROM sessions WHERE session_id = ?` - Primary key lookup
- `SELECT * FROM sessions WHERE status = 'active'` - Filter by status
- `SELECT * FROM sessions WHERE source IN ('local', ?)` - Filter by data source (covered by `idx_sessions_source`)
- `SELECT * FROM sessions ORDER BY updated_at DESC LIMIT 50` - Recent sessions
- `SELECT id, transcript_path FROM sessions WHERE status='active' AND transcript_path IS NOT NULL ORDER BY updated_at DESC` — periodic compaction sweep (covered by the partial index above)

### agents Indexes

```sql
CREATE INDEX idx_agents_agent_id ON agents(agent_id);
CREATE INDEX idx_agents_session_id ON agents(session_id);
CREATE INDEX idx_agents_status ON agents(status);
```

**Query Patterns:**
- `SELECT * FROM agents WHERE agent_id = ?` - Primary key lookup
- `SELECT * FROM agents WHERE session_id = ?` - All agents for session
- `SELECT * FROM agents WHERE status = 'running'` - Active agents

### events Indexes

```sql
-- Keeps the per-tool-event dedup used by subagent import an index seek instead
-- of a full events scan. importSubagentFromJsonl checks
-- `... WHERE agent_id = ? AND event_type = ? AND data LIKE '%"tool_use_id":"X"%'`
-- before inserting; on a subagent-heavy re-import this drops a large sweep from
-- tens of seconds to sub-second.
CREATE INDEX idx_events_agent_type ON events(agent_id, event_type);
```

### tool_executions Indexes

```sql
CREATE INDEX idx_tools_agent_id ON tool_executions(agent_id);
CREATE INDEX idx_tools_created_at ON tool_executions(created_at DESC);
```

**Query Patterns:**
- `SELECT * FROM tool_executions WHERE agent_id = ?` - All tools for agent
- `SELECT * FROM tool_executions ORDER BY created_at DESC LIMIT 100` - Recent tools

### notifications Indexes

```sql
CREATE INDEX idx_notifications_session_id ON notifications(session_id);
```

**Query Patterns:**
- `SELECT * FROM notifications WHERE session_id = ?` - All notifications for session

---

## Migrations

Every migration in `server/db.js` runs at module load as a check-then-act (probe a column, `ALTER` when missing; rebuild a table whose old constraint is still there). Two dashboards may start on one database at the same instant, so the whole section — the legacy-DB copy, `CREATE TABLE IF NOT EXISTS`, every probe and rebuild, the pricing top-up — runs under a cross-process lock: a `BEGIN IMMEDIATE` on the separate file `<db>-migrate.lock` (busy timeout 120 s), committed just before the prepared statements are built. The lock is a separate file because several rebuilds open their own `BEGIN` and switch `foreign_keys` off, which a surrounding transaction on the main database would break. A process that dies holding it releases it with its file handle.

### Schema Versioning

```mermaid
graph TB
    V1[Version 1<br/>Initial schema] --> V2[Version 2<br/>Add updated_at]
    V2 --> V3[Version 3<br/>Add pricing_rules]
    V3 --> VN[Version N<br/>Future migrations]
    
    style V1 fill:#3B82F6
    style V2 fill:#10B981
    style V3 fill:#F59E0B
```

### Migration Strategy

```javascript
// db.js - Schema versioning
const SCHEMA_VERSION = 3;

function runMigrations() {
  const currentVersion = db.pragma('user_version', { simple: true });
  
  if (currentVersion < 1) {
    // Initial schema
    db.exec(`
      CREATE TABLE sessions (...);
      CREATE TABLE agents (...);
      -- etc.
    `);
    db.pragma('user_version = 1');
  }
  
  if (currentVersion < 2) {
    // Add updated_at column
    db.exec(`ALTER TABLE sessions ADD COLUMN updated_at TEXT DEFAULT (datetime('now'))`);
    db.pragma('user_version = 2');
  }
  
  if (currentVersion < 3) {
    // Add pricing_rules table
    db.exec(`CREATE TABLE pricing_rules (...)`);
    db.pragma('user_version = 3');
  }
}
```

### Migration Workflow

```mermaid
sequenceDiagram
    participant App
    participant DB
    participant Migrations
    
    App->>DB: Open connection
    DB->>Migrations: Check PRAGMA user_version
    Migrations->>Migrations: Compare with SCHEMA_VERSION
    
    alt Version mismatch
        Migrations->>DB: Run migration scripts
        DB->>Migrations: Success
        Migrations->>DB: Update user_version
    else Version match
        Migrations->>App: Ready
    end
    
    App->>DB: Application queries
```

---

## Query Patterns

### Common Queries

#### List Recent Sessions

```sql
SELECT 
  s.*,
  COUNT(DISTINCT a.id) as agent_count,
  COUNT(DISTINCT t.id) as tool_count
FROM sessions s
LEFT JOIN agents a ON s.session_id = a.session_id
LEFT JOIN tool_executions t ON a.agent_id = t.agent_id
GROUP BY s.id
ORDER BY s.updated_at DESC
LIMIT 50;
```

**Performance:** ~5-10ms (with indexes)

#### Get Session with Agents

```sql
SELECT * FROM sessions WHERE session_id = 'sess_abc123';
SELECT * FROM agents WHERE session_id = 'sess_abc123';
```

**Performance:** ~1-2ms per query

#### Get Agent Tools

```sql
SELECT * FROM tool_executions 
WHERE agent_id = 'agent_xyz789'
ORDER BY created_at DESC;
```

**Performance:** ~2-5ms

#### Calculate Total Cost

```sql
SELECT 
  SUM(cost) as total_cost
FROM agents
WHERE session_id = 'sess_abc123';
```

**Performance:** ~1-2ms

### Query Optimization

```mermaid
graph TB
    Query[SQL Query] --> Explain[EXPLAIN QUERY PLAN]
    Explain --> Scan{Full Table<br/>Scan?}
    
    Scan -->|Yes| AddIndex[Add Index]
    Scan -->|No| Check{Query Time<br/>>10ms?}
    
    AddIndex --> Retest[Re-test Query]
    Retest --> Check
    
    Check -->|Yes| Optimize[Optimize Query<br/>Rewrite, Denormalize]
    Check -->|No| Done[Acceptable Performance]
    
    style AddIndex fill:#F59E0B
    style Optimize fill:#EF4444
    style Done fill:#10B981
```

---

## Performance Optimization

### SQLite Pragmas

What `server/db.js` actually sets on every open:

```javascript
db.pragma("journal_mode = WAL");             // readers never block the writer
db.pragma("foreign_keys = ON");
db.pragma("busy_timeout = 5000");            // two dashboards may share the file
db.pragma("synchronous = NORMAL");           // safe with WAL: a power cut loses only the last commits, never the file
db.pragma("journal_size_limit = 67108864");  // a checkpoint truncates the WAL back to 64 MB
```

**Upkeep.** `maintainDatabase()` (exported by `server/db.js`) runs hourly from `server/index.js` (`DASHBOARD_DB_MAINTAIN_MS`, `0` = off): a `wal_checkpoint(PASSIVE)` — it never waits on a reader, such as the other dashboard — after which the next write shrinks the WAL to `journal_size_limit`, and `PRAGMA optimize`. Nothing runs `VACUUM`: it rewrites the whole file under an exclusive lock, so space freed by a cleanup is reused for new rows instead of returned to the disk.

**Growth.** `events.data` is the hook payload TRIMMED by `trim_event_data(?)` (`server/lib/event-payload.js`, a SQL function registered on the connection in `server/db.js`, used by every event `INSERT` and by the duplicate guard): Edit/MultiEdit/NotebookEdit/Write drop `tool_response.originalFile`/`oldString`/`newString` (Write also `content`), Read drops `tool_response.file.content`/`base64`, and any string inside `tool_input`/`tool_response` over 8 KB keeps its first 6 KB and last 2 KB; `structuredPatch` and every field outside those two are kept, and `_trimmed` lists what was dropped. The full text stays in the session transcript and AgentHub's snapshot of it under `<data dir>/transcripts/`, which the Conversation tab reads. Rows stored before this are trimmed once in the background (`sweepEventPayloads`, 200 rows per transaction, `user_version` 4 after marker 3); measured on a 1.2 GB install it took `events.data` from 913 MB to ~400 MB. Freed pages are reused, not returned to the disk (no `VACUUM`). No ROW is pruned automatically. The only deletions are the user's: Settings → clear data / cleanup, a workspace's purge, `scripts/clear-data.js`. Clear data runs in ONE transaction (foreign keys off around it, so workflows are deleted explicitly), cleanup deletes in 500-id chunks inside one transaction. `GET /api/settings/export` streams `events` in keyset pages of 500 instead of loading the table.

### Prepared Statements

```javascript
// db.js - Prepared statements prevent SQL injection + optimize performance
const stmts = {
  findSession: db.prepare('SELECT * FROM sessions WHERE session_id = ?'),
  createSession: db.prepare('INSERT INTO sessions (session_id, model) VALUES (?, ?)'),
  updateSession: db.prepare('UPDATE sessions SET status = ?, total_cost = ? WHERE session_id = ?'),
  touchSession: db.prepare("UPDATE sessions SET updated_at = datetime('now') WHERE session_id = ?")
};

// Usage
const session = stmts.findSession.get('sess_abc123');
stmts.touchSession.run('sess_abc123');
```

### Transaction Batching

```javascript
// Batch multiple writes in a transaction
const insertMany = db.transaction((tools) => {
  for (const tool of tools) {
    stmts.createToolExecution.run(tool.agent_id, tool.tool_name, tool.duration_ms);
  }
});

insertMany([
  { agent_id: 'agent_1', tool_name: 'bash', duration_ms: 100 },
  { agent_id: 'agent_1', tool_name: 'view', duration_ms: 50 },
  // ... more tools
]);
```

### Performance Benchmarks

| Operation | Without Optimization | With Optimization | Improvement |
|-----------|---------------------|-------------------|-------------|
| Session list (50) | 25ms | 5ms | 5x faster |
| Hook processing | 15ms | 2ms | 7.5x faster |
| Batch insert (100 tools) | 500ms | 50ms | 10x faster |

---

## Data Integrity

### Foreign Key Constraints

```sql
-- Enabled by default in db.js
PRAGMA foreign_keys = ON;
```

**Constraint Enforcement:**

```mermaid
graph TB
    Insert[INSERT agent] --> Check{session_id exists?}
    Check -->|Yes| Allow[Insert Allowed]
    Check -->|No| Reject[FOREIGN KEY constraint failed]
    
    Delete[DELETE session] --> Cascade{Cascade enabled?}
    Cascade -->|Yes| DeleteChildren[Delete agents and tools]
    Cascade -->|No| BlockDelete[Cannot delete FK exists]
    
    style Allow fill:#10B981
    style Reject fill:#EF4444
    style DeleteChildren fill:#F59E0B
```

### Data Validation

```javascript
// Validate before insert
function validateSession(session) {
  if (!session.session_id) throw new Error('session_id required');
  if (session.total_cost < 0) throw new Error('total_cost must be >= 0');
  if (!['active', 'completed'].includes(session.status)) {
    throw new Error('Invalid status');
  }
}
```

---

## Backup Strategies

### Online Backup (Recommended)

```sql
-- Using VACUUM INTO (SQLite 3.27+)
VACUUM INTO '/backups/dashboard_20240318.db';
```

### Offline Backup

```bash
#!/bin/bash
# Stop application
systemctl stop agenthub

# Copy database file
cp /var/lib/agenthub/dashboard.db /backups/dashboard_$(date +%Y%m%d).db

# Start application
systemctl start agenthub
```

### Backup Schedule

```mermaid
graph TB
    subgraph "Automated Backups"
        Daily[Daily Backup<br/>2 AM UTC]
        Weekly[Weekly Backup<br/>Sunday 2 AM]
        Monthly[Monthly Backup<br/>1st of month]
    end
    
    subgraph "Retention"
        Daily --> R7[Keep 7 days]
        Weekly --> R4[Keep 4 weeks]
        Monthly --> R12[Keep 12 months]
    end
    
    subgraph "Storage"
        R7 --> Local[Local Disk]
        R4 --> S3[AWS S3]
        R12 --> Glacier[AWS Glacier]
    end
    
    style Daily fill:#3B82F6
    style S3 fill:#FF9900
```

---

## Summary

The database schema provides:

- ✅ **Normalized design** - Minimal redundancy, clear relationships
- ✅ **Performance optimized** - Indexes, prepared statements, WAL mode
- ✅ **Data integrity** - Foreign keys, constraints, transactions
- ✅ **Migration support** - Schema versioning with PRAGMA user_version
- ✅ **Comprehensive indexing** - Fast queries for common access patterns
- ✅ **Backup strategies** - Online + offline backup options

For API usage, see [docs/API.md](./API.md).