feat(db): trim stored hook payloads; price the session list like a session

Analytics parity, phase 1 (data hygiene), from the analytics-parity session.

events.data is now stored through the SQL function trim_event_data(?)
(server/lib/event-payload.js, registered in server/db.js) on every event
INSERT and in the duplicate guard: Edit/MultiEdit/NotebookEdit/Write drop
the whole-file mirrors (originalFile, oldString/newString, Write content),
Read drops file.content/base64, and any string in tool_input/tool_response
over 8 KB keeps a 6 KB head + 2 KB tail. structuredPatch and every field
outside those two stay whole; _trimmed names what was dropped. The full
text stays in the transcript and its snapshot. Rows stored before this are
trimmed once in the background (sweepEventPayloads, 200 rows per write
transaction, user_version 4). No VACUUM, no row pruned.

The session list now prices every token column /api/pricing/cost/:id does
(speed, geo, tier, the 1h cache write, server-tool fees) through one
tokensForSessions helper, so a row's cost equals the session's own.
This commit is contained in:
2026-10-05 13:13:02 +07:00
parent 138e706dd1
commit 215cae7393
9 changed files with 368 additions and 46 deletions
+11 -3
View File
@@ -151,6 +151,14 @@ db.pragma("synchronous = NORMAL");
// (an import, a cleanup) left a WAL of hundreds of MB that never shrank while
// any dashboard kept the database open. See also `maintainDatabase`.
db.pragma("journal_size_limit = 67108864");
// Every event insert stores its payload through `trim_event_data(?)`
// (`server/lib/event-payload.js`): whole-file mirrors and long output stay in
// the transcript, not in `events.data`.
db.function(
"trim_event_data",
{ deterministic: true },
require("./lib/event-payload").trimEventData
);
db.exec(`
CREATE TABLE IF NOT EXISTS sessions (
@@ -1684,11 +1692,11 @@ const stmts = {
findRecentDuplicateEvent: db.prepare(
`SELECT 1 FROM (SELECT event_type, data, created_at FROM events
WHERE session_id = ? ORDER BY id DESC LIMIT 1)
WHERE event_type = ? AND data = ?
WHERE event_type = ? AND data = trim_event_data(?)
AND created_at >= strftime('%Y-%m-%dT%H:%M:%fZ', 'now', '-10 seconds')`
),
insertEvent: db.prepare(
"INSERT INTO events (session_id, agent_id, event_type, tool_name, summary, data, created_at) VALUES (?, ?, ?, ?, ?, ?, strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))"
"INSERT INTO events (session_id, agent_id, event_type, tool_name, summary, data, created_at) VALUES (?, ?, ?, ?, ?, trim_event_data(?), strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))"
),
// Same insert with an EXPLICIT created_at, for events recovered from a
// transcript: their real time is in the file, not the moment we happened to
@@ -1697,7 +1705,7 @@ const stmts = {
// re-parse re-insert the whole turn history (587k rows across this DB, 75%
// of the events table, one session holding 68,747 copies of the same turns).
insertEventAt: db.prepare(
"INSERT INTO events (session_id, agent_id, event_type, tool_name, summary, data, created_at) VALUES (?, ?, ?, ?, ?, ?, ?)"
"INSERT INTO events (session_id, agent_id, event_type, tool_name, summary, data, created_at) VALUES (?, ?, ?, ?, ?, trim_event_data(?), ?)"
),
listEvents: db.prepare("SELECT * FROM events ORDER BY created_at DESC, id DESC LIMIT ? OFFSET ?"),
listEventsBySession: db.prepare(