fix(pricing): book daily cost to the day tokens were spent

token_usage holds a session's cumulative totals with no time, so
/api/pricing/cost booked a multi-day session's whole cost to its start
date and the status bar's Today read ~$0. TranscriptCache now also splits
usage by each message's UTC hour (tokensByHour); the hook writer stores it
in the new token_usage_hourly table (MAX per hour) and the server
back-fills existing sessions at start. The daily split books each total
over its hourly rows' local days and the rest to the start date, so the
days still add up to the total.
This commit is contained in:
2026-10-05 08:03:05 +07:00
parent ed1bf20e6b
commit 65fe680d38
11 changed files with 478 additions and 25 deletions
+1 -1
View File
File diff suppressed because one or more lines are too long
+12
View File
@@ -1864,6 +1864,18 @@ curl -X DELETE http://localhost:4820/api/pricing/gpt-5.1-codex
| 403 | Cannot delete default rule | | 403 | Cannot delete default rule |
| 500 | Database error | | 500 | Database error |
#### Total Cost
```http
GET /api/pricing/cost?tz_offset=<minutes>&sources=<csv>
```
The total cost with a per-model `breakdown`, plus `daily_costs: [{date, cost}]` by local date (`tz_offset` is the browser's `getTimezoneOffset()`; the status bar's **Today** is the entry for today). A day is the day the tokens were SPENT: each session's total is split over the local days of its `token_usage_hourly` rows (one per UTC hour, from each Claude transcript message's timestamp), and whatever the total holds beyond those rows — a session recorded before that table and whose transcript is gone, sub-agent files, Codex and Antigravity sessions — is booked to the session's start date. The days always add up to the total.
```bash
curl "http://localhost:4820/api/pricing/cost?tz_offset=-420"
```
--- ---
### Notifications ### Notifications
+6
View File
@@ -500,6 +500,12 @@ Managed through the `/api/remote-sources/*` routes; sync/status changes are broa
--- ---
### token_usage_hourly
The same buckets as `token_usage` (session, model, speed, inference_geo, service_tier and the eight count columns, no baselines), split by `hour` — the UTC hour `YYYY-MM-DDTHH` a Claude transcript message was written in. The hook writer fills it from `TranscriptCache`'s `tokensByHour` on every transcript-bearing event (`writeHourly` in `server/lib/hourly-usage.js`, `stmts.upsertHourlyUsage`, MAX per column so a rewritten transcript never takes an hour back), and the server back-fills sessions that have tokens but no hourly rows a few seconds after each start (`backfillHourly`, one transcript per tick). Only the daily split of `GET /api/pricing/cost` reads it; `token_usage` stays the total, and what it holds beyond these rows is booked to the session's start date. Cascades with its session.
---
### codex_rollouts ### codex_rollouts
How far `server/lib/codex-usage.js` has read each Codex rollout journal (`$CODEX_HOME/sessions/YYYY/MM/DD/rollout-<ts>-<thread>.jsonl`). A file whose `size` and `mtime_ms` are unchanged is not read again; a grown one is read from `offset` to its last newline, at most 16 MB per pass (`state.more` marks a file with more to read); a shrunken one from 0. There is one row per `thread_id`: the same journal met under a new path (CODEX_HOME moved, or reached through a symlink) replaces its old row instead of being counted twice. Because it lives in the DB, a Codex session the user deleted stays gone until Codex writes to that journal again, while a fresh database re-imports every journal. `POST /api/settings/clear-data` empties this table (and `antigravity_conversations`) with the sessions, so a cleared session is imported again whole instead of never, or half on its next write. How far `server/lib/codex-usage.js` has read each Codex rollout journal (`$CODEX_HOME/sessions/YYYY/MM/DD/rollout-<ts>-<thread>.jsonl`). A file whose `size` and `mtime_ms` are unchanged is not read again; a grown one is read from `offset` to its last newline, at most 16 MB per pass (`state.more` marks a file with more to read); a shrunken one from 0. There is one row per `thread_id`: the same journal met under a new path (CODEX_HOME moved, or reached through a symlink) replaces its old row instead of being counted twice. Because it lives in the DB, a Codex session the user deleted stays gone until Codex writes to that journal again, while a fresh database re-imports every journal. `POST /api/settings/clear-data` empties this table (and `antigravity_conversations`) with the sessions, so a cleared session is imported again whole instead of never, or half on its next write.
+203
View File
@@ -0,0 +1,203 @@
/**
* @file The daily cost split of `GET /api/pricing/cost`: a session's tokens are
* booked to the local days its `token_usage_hourly` rows fall on (the client's
* `tz_offset` moving an hour across midnight), the rest of its total to its
* start date, and the days add up to the total. Also `TranscriptCache`'s
* `tokensByHour` — split by each message's timestamp, kept across an
* incremental read — and the hook writer filling the table from it.
* @author Nguyễn Ngọc Trí Vĩ <vinnt@smartgift.vn>
*/
"use strict";
const { describe, it, before, after } = require("node:test");
const assert = require("node:assert/strict");
const fs = require("node:fs");
const os = require("node:os");
const path = require("node:path");
const http = require("node:http");
const ROOT = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), "daily-cost-")));
process.env.DASHBOARD_DB_PATH = path.join(ROOT, "t.db");
process.env.CLAUDE_HOME = path.join(ROOT, "claude");
process.env.DASHBOARD_DATA_DIR = path.join(ROOT, "data");
process.env.DASHBOARD_REMOTE_SYNC_MS = "0";
process.env.DASHBOARD_LIVENESS_PROBE = "0";
const { createApp } = require("../index");
const { db, stmts } = require("../db");
const TranscriptCache = require("../lib/transcript-cache");
const M = 1_000_000;
let server;
let base;
function call(method, url, body) {
return new Promise((resolve, reject) => {
const req = http.request(
`${base}${url}`,
{ method, headers: body ? { "content-type": "application/json" } : {} },
(res) => {
let text = "";
res.on("data", (c) => (text += c));
res.on("end", () => resolve({ status: res.statusCode, body: JSON.parse(text) }));
}
);
req.on("error", reject);
if (body) req.write(JSON.stringify(body));
req.end();
});
}
function session(id, startedAt) {
db.prepare(
"INSERT INTO sessions (id, name, status, started_at, updated_at) VALUES (?, ?, 'completed', ?, ?)"
).run(id, id, startedAt, startedAt);
}
const total = (id, input) =>
stmts.setTokenUsage.run(
id,
"daily-model",
"standard",
"global",
"standard",
input,
0,
0,
0,
0,
0,
0,
0
);
const hour = (id, h, input) =>
stmts.upsertHourlyUsage.run(
id,
h,
"daily-model",
"standard",
"global",
"standard",
input,
0,
0,
0,
0,
0,
0,
0
);
const byDate = (r) => Object.fromEntries(r.body.daily_costs.map((d) => [d.date, d.cost]));
const line = (ts, id, input) =>
JSON.stringify({
type: "assistant",
timestamp: ts,
message: {
id,
model: "daily-model",
role: "assistant",
usage: { input_tokens: input, output_tokens: 0 },
},
});
before(async () => {
server = http.createServer(createApp());
await new Promise((r) => server.listen(0, "127.0.0.1", r));
base = `http://127.0.0.1:${server.address().port}`;
// $1 per million input tokens, so a day's cost reads as millions of tokens.
const put = await call("PUT", "/api/pricing", {
model_pattern: "daily-model%",
display_name: "Daily",
input_per_mtok: 1,
output_per_mtok: 0,
cache_read_per_mtok: 0,
cache_write_per_mtok: 0,
});
assert.equal(put.status, 200);
});
after(() => {
server.close();
fs.rmSync(ROOT, { recursive: true, force: true });
});
describe("GET /api/pricing/cost daily split", () => {
it("books each hour to its local day and the rest to the start date", async () => {
session("multi-day", "2026-10-02T03:00:00.000Z");
total("multi-day", 7 * M); // 1 M recorded before hourly rows existed
hour("multi-day", "2026-10-03T05", 2 * M);
hour("multi-day", "2026-10-03T20", 1 * M); // 03:00 on the 4th at UTC+7
hour("multi-day", "2026-10-05T01", 3 * M);
const utc = byDate(await call("GET", "/api/pricing/cost?tz_offset=0"));
assert.deepEqual(utc, { "2026-10-02": 1, "2026-10-03": 3, "2026-10-05": 3 });
const ict = await call("GET", "/api/pricing/cost?tz_offset=-420");
assert.deepEqual(byDate(ict), {
"2026-10-02": 1,
"2026-10-03": 2,
"2026-10-04": 1,
"2026-10-05": 3,
});
assert.equal(ict.body.total_cost, 7);
});
it("scales hours above the total down to it, so the days still add up", async () => {
session("over", "2026-09-01T00:00:00.000Z");
total("over", 2 * M);
hour("over", "2026-09-02T00", 2 * M);
hour("over", "2026-09-03T00", 2 * M);
const r = await call("GET", "/api/pricing/cost?tz_offset=0");
const d = byDate(r);
assert.equal(d["2026-09-01"], undefined);
assert.equal(d["2026-09-02"], 1);
assert.equal(d["2026-09-03"], 1);
});
});
describe("TranscriptCache tokensByHour", () => {
it("splits by message hour, counts a message once, and survives an incremental read", () => {
const file = path.join(ROOT, "t.jsonl");
fs.writeFileSync(
file,
[
line("2026-10-02T10:15:00.000Z", "m1", 100),
line("2026-10-02T10:15:01.000Z", "m1", 100), // a second block line of m1
line("2026-10-02T10:40:00.000Z", "m2", 50),
].join("\n") + "\n"
);
const cache = new TranscriptCache();
const hours = (r) =>
Object.fromEntries(Object.values(r.tokensByHour).map((b) => [b.hour, b.input]));
assert.deepEqual(hours(cache.extract(file)), { "2026-10-02T10": 150 });
fs.appendFileSync(file, line("2026-10-04T23:59:00.000Z", "m3", 7) + "\n");
assert.deepEqual(hours(cache.extract(file)), { "2026-10-02T10": 150, "2026-10-04T23": 7 });
});
it("is written by the hook writer", async () => {
const file = path.join(ROOT, "hook.jsonl");
fs.writeFileSync(
file,
[
line("2026-10-02T10:00:00.000Z", "a", 1 * M),
line("2026-10-03T10:00:00.000Z", "b", 2 * M),
].join("\n") + "\n"
);
const r = await call("POST", "/api/hooks/event", {
hook_type: "Stop",
data: { session_id: "hooked", transcript_path: file },
});
assert.equal(r.status, 200);
const rows = db
.prepare(
"SELECT hour, input_tokens FROM token_usage_hourly WHERE session_id = 'hooked' ORDER BY hour"
)
.all();
assert.deepEqual(
rows.map((x) => [x.hour, x.input_tokens]),
[
["2026-10-02T10", 1 * M],
["2026-10-03T10", 2 * M],
]
);
});
});
+44
View File
@@ -219,6 +219,29 @@ db.exec(`
FOREIGN KEY (session_id) REFERENCES sessions(id) ON DELETE CASCADE FOREIGN KEY (session_id) REFERENCES sessions(id) ON DELETE CASCADE
); );
-- The same buckets split by the UTC hour (YYYY-MM-DDTHH) the tokens were
-- spent in, read from each transcript message's timestamp. Only the daily
-- cost split reads it (GET /api/pricing/cost): token_usage stays the total,
-- and whatever it holds beyond these rows is booked to the session's start.
CREATE TABLE IF NOT EXISTS token_usage_hourly (
session_id TEXT NOT NULL,
hour TEXT NOT NULL,
model TEXT NOT NULL,
speed TEXT NOT NULL DEFAULT 'standard',
inference_geo TEXT NOT NULL DEFAULT 'global',
service_tier TEXT NOT NULL DEFAULT 'standard',
input_tokens INTEGER NOT NULL DEFAULT 0,
output_tokens INTEGER NOT NULL DEFAULT 0,
cache_read_tokens INTEGER NOT NULL DEFAULT 0,
cache_write_tokens INTEGER NOT NULL DEFAULT 0,
cache_write_1h_tokens INTEGER NOT NULL DEFAULT 0,
web_search_requests INTEGER NOT NULL DEFAULT 0,
web_fetch_requests INTEGER NOT NULL DEFAULT 0,
code_execution_requests INTEGER NOT NULL DEFAULT 0,
PRIMARY KEY (session_id, hour, model, speed, inference_geo, service_tier),
FOREIGN KEY (session_id) REFERENCES sessions(id) ON DELETE CASCADE
);
CREATE TABLE IF NOT EXISTS model_pricing ( CREATE TABLE IF NOT EXISTS model_pricing (
model_pattern TEXT PRIMARY KEY, model_pattern TEXT PRIMARY KEY,
display_name TEXT NOT NULL, display_name TEXT NOT NULL,
@@ -1774,6 +1797,27 @@ const stmts = {
baseline_cache_write = 0, baseline_cache_write_1h = 0, baseline_cache_write = 0, baseline_cache_write_1h = 0,
baseline_web_search = 0, baseline_web_fetch = 0, baseline_code_execution = 0 baseline_web_search = 0, baseline_web_fetch = 0, baseline_code_execution = 0
`), `),
// One hour of one bucket, from a whole-transcript parse. MAX, not replace:
// a transcript that shrank (rewritten) must not take back an hour already
// counted. Args: session_id, hour, model, speed, inference_geo, service_tier,
// input, output, cache_read, cache_write, cache_write_1h, web_search,
// web_fetch, code_execution.
upsertHourlyUsage: db.prepare(`
INSERT INTO token_usage_hourly (session_id, hour, model, speed, inference_geo, service_tier,
input_tokens, output_tokens, cache_read_tokens, cache_write_tokens,
cache_write_1h_tokens, web_search_requests, web_fetch_requests,
code_execution_requests)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(session_id, hour, model, speed, inference_geo, service_tier) DO UPDATE SET
input_tokens = MAX(input_tokens, excluded.input_tokens),
output_tokens = MAX(output_tokens, excluded.output_tokens),
cache_read_tokens = MAX(cache_read_tokens, excluded.cache_read_tokens),
cache_write_tokens = MAX(cache_write_tokens, excluded.cache_write_tokens),
cache_write_1h_tokens = MAX(cache_write_1h_tokens, excluded.cache_write_1h_tokens),
web_search_requests = MAX(web_search_requests, excluded.web_search_requests),
web_fetch_requests = MAX(web_fetch_requests, excluded.web_fetch_requests),
code_execution_requests = MAX(code_execution_requests, excluded.code_execution_requests)
`),
getTokenTotals: db.prepare(` getTokenTotals: db.prepare(`
SELECT SELECT
COALESCE(SUM(input_tokens + baseline_input), 0) as total_input, COALESCE(SUM(input_tokens + baseline_input), 0) as total_input,
+13
View File
@@ -469,6 +469,19 @@ function startBackgroundServices() {
} }
} }
// Per-hour tokens for sessions recorded before `token_usage_hourly`, so the
// daily cost split covers them too. Picks only sessions with no hourly rows,
// so a later start re-reads nothing already done; non-blocking.
{
const t = setTimeout(() => {
require("./lib/hourly-usage")
.backfillHourly()
.then((n) => n > 0 && console.log(`Hourly token usage: back-filled ${n} session(s)`))
.catch((err) => console.warn("hourly token backfill failed:", err?.message || err));
}, 2_000);
if (t.unref) t.unref();
}
const { startUpdateScheduler } = require("./update-scheduler"); const { startUpdateScheduler } = require("./update-scheduler");
const { broadcast } = require("./websocket"); const { broadcast } = require("./websocket");
startUpdateScheduler({ broadcast }); startUpdateScheduler({ broadcast });
+72
View File
@@ -0,0 +1,72 @@
/**
* @file Per-hour token usage (`token_usage_hourly`): written from a Claude
* transcript's `tokensByHour` (`server/lib/transcript-cache.js`) by the hook
* writer, back-filled once for sessions recorded before the table existed, and
* read only by the daily cost split in `GET /api/pricing/cost` — so a session
* that runs for days books each day's spend to that day instead of its start.
* @author Nguyễn Ngọc Trí Vĩ <vinnt@smartgift.vn>
*/
const { db, stmts } = require("../db");
const TranscriptCache = require("./transcript-cache");
/** Write every hour of one transcript parse (MAX per hour, see the statement). */
function writeHourly(sessionId, tokensByHour) {
// ponytail: rewrites every hour of the session on each hook — ~1 row per
// active hour per model, cheap; write only the newest hours if it ever shows.
for (const t of Object.values(tokensByHour || {})) {
stmts.upsertHourlyUsage.run(
sessionId,
t.hour,
t.model,
t.speed,
t.geo,
t.tier,
t.input,
t.output,
t.cacheRead,
t.cacheWrite,
t.cacheWrite1h,
t.webSearch,
t.webFetch,
t.codeExec
);
}
}
/**
* Fill the table for Claude sessions that have tokens but no hourly rows yet,
* one transcript per tick so a large history never blocks the event loop.
* Resolves with the number of sessions written.
*/
function backfillHourly() {
const rows = db
.prepare(
`SELECT s.id, s.transcript_path FROM sessions s
WHERE s.transcript_path IS NOT NULL AND s.transcript_path != ''
AND EXISTS (SELECT 1 FROM token_usage tu WHERE tu.session_id = s.id)
AND NOT EXISTS (SELECT 1 FROM token_usage_hourly h WHERE h.session_id = s.id)`
)
.all();
const cache = new TranscriptCache(1);
let written = 0;
return new Promise((resolve) => {
const next = (i) => {
if (i >= rows.length) return resolve(written);
const { id, transcript_path } = rows[i];
try {
const result = cache.extract(transcript_path);
if (result?.tokensByHour && Object.keys(result.tokensByHour).length > 0) {
db.transaction(() => writeHourly(id, result.tokensByHour))();
written++;
}
} catch {
/* an unreadable transcript keeps its start-date booking */
}
setImmediate(() => next(i + 1));
};
next(0);
});
}
module.exports = { writeHourly, backfillHourly };
+34 -2
View File
@@ -1,5 +1,5 @@
/** /**
* @file TranscriptCache class for efficient extraction of token usage and compaction data from JSONL transcript files, with stat-based caching and incremental reads to handle append-only growth without re-reading the entire file. Also extracts API error entries and turn duration system messages for enhanced analytics. * @file TranscriptCache class for efficient extraction of token usage (in total and per UTC hour, `tokensByHour`) and compaction data from JSONL transcript files, with stat-based caching and incremental reads to handle append-only growth without re-reading the entire file. Also extracts API error entries and turn duration system messages for enhanced analytics.
* @author Nguyễn Ngọc Trí Vĩ <vinnt@smartgift.vn> * @author Nguyễn Ngọc Trí Vĩ <vinnt@smartgift.vn>
*/ */
@@ -16,6 +16,22 @@ const {
const MAX_CACHE_ENTRIES = 200; const MAX_CACHE_ENTRIES = 200;
/** `YYYY-MM-DDTHH` (UTC) of a transcript timestamp, or null when it has none. */
function utcHour(ts) {
if (ts === undefined || ts === null) return null;
const d = new Date(ts);
return Number.isNaN(d.getTime()) ? null : d.toISOString().slice(0, 13);
}
/** Add `tokens` into `map[key]`, creating the bucket (with its hour, if any). */
function addBucket(map, key, tokens) {
if (!map[key]) {
map[key] = emptyBucket(tokens.model, tokens.speed, tokens.geo, tokens.tier);
if (tokens.hour) map[key].hour = tokens.hour;
}
accumulateBucket(map[key], tokens);
}
// Marker text Claude Code writes into the transcript when a turn is cancelled // Marker text Claude Code writes into the transcript when a turn is cancelled
// by the user (Esc). The synthetic entry is `type:"user"` and also carries an // by the user (Esc). The synthetic entry is `type:"user"` and also carries an
// `interruptedMessageId` field; we accept either signal so detection survives // `interruptedMessageId` field; we accept either signal so detection survives
@@ -168,6 +184,7 @@ class TranscriptCache {
merged.usageExtras.inference_geos.length > 0); merged.usageExtras.inference_geos.length > 0);
const result = { const result = {
tokensByModel: hasTokens ? merged.tokensByModel : null, tokensByModel: hasTokens ? merged.tokensByModel : null,
tokensByHour: hasTokens ? merged.tokensByHour : null,
compaction: merged.compaction, compaction: merged.compaction,
errors: merged.errors, errors: merged.errors,
turnDurations: hasTurnDurations ? merged.turnDurations : null, turnDurations: hasTurnDurations ? merged.turnDurations : null,
@@ -363,6 +380,10 @@ class TranscriptCache {
_initParseState() { _initParseState() {
return { return {
tokensByModel: {}, tokensByModel: {},
// The same buckets split by the UTC hour of each message, so cost can be
// booked to the day it was spent (`token_usage_hourly`), not the day the
// session started. Key: `<hour><bucketKey>`.
tokensByHour: {},
compaction: null, compaction: null,
errors: [], errors: [],
turnDurations: [], turnDurations: [],
@@ -533,7 +554,12 @@ class TranscriptCache {
if (!state.tokensByModel[key]) { if (!state.tokensByModel[key]) {
state.tokensByModel[key] = emptyBucket(model, speed, geo, tier); state.tokensByModel[key] = emptyBucket(model, speed, geo, tier);
} }
accumulateBucket(state.tokensByModel[key], extractUsageFields(msg.usage)); const fields = extractUsageFields(msg.usage);
accumulateBucket(state.tokensByModel[key], fields);
const hour = utcHour(entry.timestamp);
if (hour) {
addBucket(state.tokensByHour, hour + key, { model, speed, geo, tier, hour, ...fields });
}
} }
if (msg.usage.service_tier) state.usageExtras.service_tiers.add(msg.usage.service_tier); if (msg.usage.service_tier) state.usageExtras.service_tiers.add(msg.usage.service_tier);
@@ -591,6 +617,7 @@ class TranscriptCache {
return { return {
tokensByModel: hasTokens ? state.tokensByModel : null, tokensByModel: hasTokens ? state.tokensByModel : null,
tokensByHour: hasTokens ? state.tokensByHour : null,
compaction: state.compaction, compaction: state.compaction,
errors: hasErrors ? state.errors : null, errors: hasErrors ? state.errors : null,
turnDurations: hasTurnDurations ? state.turnDurations : null, turnDurations: hasTurnDurations ? state.turnDurations : null,
@@ -642,6 +669,10 @@ class TranscriptCache {
accumulateBucket(tokensByModel[key], tokens); accumulateBucket(tokensByModel[key], tokens);
} }
} }
const tokensByHour = this._cloneTokens(cached.result?.tokensByHour) || {};
for (const [key, tokens] of Object.entries(incremental?.tokensByHour || {})) {
addBucket(tokensByHour, key, tokens);
}
let compaction = cached.result?.compaction let compaction = cached.result?.compaction
? this._cloneCompaction(cached.result.compaction) ? this._cloneCompaction(cached.result.compaction)
@@ -728,6 +759,7 @@ class TranscriptCache {
return { return {
tokensByModel, tokensByModel,
tokensByHour,
compaction, compaction,
errors, errors,
turnDurations, turnDurations,
+2
View File
@@ -9,6 +9,7 @@ const dbModule = require("../db");
const { stmts, db } = dbModule; const { stmts, db } = dbModule;
const { broadcast } = require("../websocket"); const { broadcast } = require("../websocket");
const TranscriptCache = require("../lib/transcript-cache"); const TranscriptCache = require("../lib/transcript-cache");
const { writeHourly } = require("../lib/hourly-usage");
const { scanAndImportSubagents } = require("../../scripts/import-history"); const { scanAndImportSubagents } = require("../../scripts/import-history");
const { evaluateEvent } = require("../lib/alerts"); const { evaluateEvent } = require("../lib/alerts");
const { ingestWorkflowsForSession } = require("../lib/workflow-ingest"); const { ingestWorkflowsForSession } = require("../lib/workflow-ingest");
@@ -1008,6 +1009,7 @@ const processEvent = db.transaction((hookType, data, tool = "claude") => {
tokens.codeExec tokens.codeExec
); );
} }
writeHourly(sessionId, result.tokensByHour);
} }
// Register API errors from transcript (quota limits, rate limits, overloaded, etc.) // Register API errors from transcript (quota limits, rate limits, overloaded, etc.)
+87 -22
View File
@@ -222,6 +222,92 @@ function calculateDailyCosts(dailyTokenRows, pricingRules) {
.map(([date, rows]) => ({ date, cost: calculateCost(rows, pricingRules, date).total_cost })); .map(([date, rows]) => ({ date, cost: calculateCost(rows, pricingRules, date).total_cost }));
} }
const USAGE_COLS = [
"input_tokens",
"output_tokens",
"cache_read_tokens",
"cache_write_tokens",
"cache_write_1h_tokens",
"web_search_requests",
"web_fetch_requests",
"code_execution_requests",
];
const BUCKET_COLS = ["model", "speed", "inference_geo", "service_tier"];
/**
* Token rows per (local date, bucket) for the daily cost split. A bucket's
* total (`token_usage`) is booked to the local days its `token_usage_hourly`
* rows fall on; whatever the total holds beyond them (sessions recorded before
* that table, sub-agent files, Codex/Antigravity) goes to the session's start
* date, as it always did. Hours above the total (never expected) are scaled
* down to it, so the days always add up to the total.
*/
function dailyTokenRows(tzModifier, whereClause, params) {
const totals = db
.prepare(
`SELECT tu.session_id, DATE(s.started_at, ?) as date,
${BUCKET_COLS.map((c) => `tu.${c}`).join(", ")},
tu.input_tokens + tu.baseline_input as input_tokens,
tu.output_tokens + tu.baseline_output as output_tokens,
tu.cache_read_tokens + tu.baseline_cache_read as cache_read_tokens,
tu.cache_write_tokens + tu.baseline_cache_write as cache_write_tokens,
tu.cache_write_1h_tokens + tu.baseline_cache_write_1h as cache_write_1h_tokens,
tu.web_search_requests + tu.baseline_web_search as web_search_requests,
tu.web_fetch_requests + tu.baseline_web_fetch as web_fetch_requests,
tu.code_execution_requests + tu.baseline_code_execution as code_execution_requests
FROM token_usage tu
JOIN sessions s ON s.id = tu.session_id
${whereClause}`
)
.all(tzModifier, ...params);
const hourly = db
.prepare(
`SELECT h.session_id, DATE(h.hour || ':00:00', ?) as date,
${BUCKET_COLS.map((c) => `h.${c}`).join(", ")},
${USAGE_COLS.map((c) => `SUM(h.${c}) as ${c}`).join(", ")}
FROM token_usage_hourly h
JOIN sessions s ON s.id = h.session_id
${whereClause}
GROUP BY h.session_id, 2, ${BUCKET_COLS.map((c) => `h.${c}`).join(", ")}`
)
.all(tzModifier, ...params);
const keyOf = (r) => [r.session_id, ...BUCKET_COLS.map((c) => r[c])].join("\u0001");
const hoursByBucket = new Map();
for (const h of hourly) {
const k = keyOf(h);
if (!hoursByBucket.has(k)) hoursByBucket.set(k, []);
hoursByBucket.get(k).push(h);
}
const out = new Map();
const add = (date, bucket, values) => {
const k = [date, ...BUCKET_COLS.map((c) => bucket[c])].join("\u0001");
let row = out.get(k);
if (!row) {
row = { date, ...Object.fromEntries(BUCKET_COLS.map((c) => [c, bucket[c]])) };
for (const c of USAGE_COLS) row[c] = 0;
out.set(k, row);
}
for (const c of USAGE_COLS) row[c] += values[c];
};
for (const t of totals) {
const days = hoursByBucket.get(keyOf(t)) || [];
const scale = {};
const rest = {};
for (const c of USAGE_COLS) {
const spent = days.reduce((n, d) => n + d[c], 0);
scale[c] = spent > t[c] ? t[c] / spent : 1;
rest[c] = Math.max(t[c] - spent, 0);
}
for (const d of days) {
add(d.date, t, Object.fromEntries(USAGE_COLS.map((c) => [c, d[c] * scale[c]])));
}
if (USAGE_COLS.some((c) => rest[c] > 0)) add(t.date, t, rest);
}
return [...out.values()];
}
// GET /api/pricing - List all pricing rules // GET /api/pricing - List all pricing rules
router.get("/", (_req, res) => { router.get("/", (_req, res) => {
const rules = stmts.listPricing.all(); const rules = stmts.listPricing.all();
@@ -387,28 +473,7 @@ router.get("/cost", (req, res) => {
const { clause, params } = sourceColumnClause(parseSources(req), "s.source"); const { clause, params } = sourceColumnClause(parseSources(req), "s.source");
const whereClause = clause ? `WHERE ${clause}` : ""; const whereClause = clause ? `WHERE ${clause}` : "";
const dailyTokens = db const dailyTokens = dailyTokenRows(tzModifier, whereClause, params);
.prepare(
`SELECT
DATE(s.started_at, ?) as date,
tu.model as model,
tu.speed as speed,
tu.inference_geo as inference_geo,
tu.service_tier as service_tier,
SUM(tu.input_tokens + tu.baseline_input) as input_tokens,
SUM(tu.output_tokens + tu.baseline_output) as output_tokens,
SUM(tu.cache_read_tokens + tu.baseline_cache_read) as cache_read_tokens,
SUM(tu.cache_write_tokens + tu.baseline_cache_write) as cache_write_tokens,
SUM(tu.cache_write_1h_tokens + tu.baseline_cache_write_1h) as cache_write_1h_tokens,
SUM(tu.web_search_requests + tu.baseline_web_search) as web_search_requests,
SUM(tu.web_fetch_requests + tu.baseline_web_fetch) as web_fetch_requests,
SUM(tu.code_execution_requests + tu.baseline_code_execution) as code_execution_requests
FROM token_usage tu
JOIN sessions s ON s.id = tu.session_id
${whereClause}
GROUP BY 1, tu.model, tu.speed, tu.inference_geo, tu.service_tier`
)
.all(tzModifier, ...params);
const rules = stmts.listPricing.all(); const rules = stmts.listPricing.all();
// Price the date-split rows so each day's usage bills at the rate effective on // Price the date-split rows so each day's usage bills at the rate effective on
// that date (e.g. Sonnet 5's intro discount before 2026-08-31, standard after). // that date (e.g. Sonnet 5's intro discount before 2026-08-31, standard after).
+4
View File
@@ -143,6 +143,7 @@ router.post("/clear-data", (_req, res) => {
const counts = getTableCounts(); const counts = getTableCounts();
db.pragma("foreign_keys = OFF"); db.pragma("foreign_keys = OFF");
db.prepare("DELETE FROM token_usage").run(); db.prepare("DELETE FROM token_usage").run();
db.prepare("DELETE FROM token_usage_hourly").run();
db.prepare("DELETE FROM events").run(); db.prepare("DELETE FROM events").run();
db.prepare("DELETE FROM agents").run(); db.prepare("DELETE FROM agents").run();
db.prepare("DELETE FROM sessions").run(); db.prepare("DELETE FROM sessions").run();
@@ -368,6 +369,9 @@ router.post("/cleanup", (req, res) => {
.prepare(`DELETE FROM agents WHERE session_id IN (${placeholders})`) .prepare(`DELETE FROM agents WHERE session_id IN (${placeholders})`)
.run(...ids).changes; .run(...ids).changes;
db.prepare(`DELETE FROM token_usage WHERE session_id IN (${placeholders})`).run(...ids); db.prepare(`DELETE FROM token_usage WHERE session_id IN (${placeholders})`).run(...ids);
db.prepare(`DELETE FROM token_usage_hourly WHERE session_id IN (${placeholders})`).run(
...ids
);
db.prepare(`DELETE FROM sessions WHERE id IN (${placeholders})`).run(...ids); db.prepare(`DELETE FROM sessions WHERE id IN (${placeholders})`).run(...ids);
result.purged_sessions = toDelete.length; result.purged_sessions = toDelete.length;
} }