feat(db): trim stored hook payloads; price the session list like a session
Analytics parity, phase 1 (data hygiene), from the analytics-parity session. events.data is now stored through the SQL function trim_event_data(?) (server/lib/event-payload.js, registered in server/db.js) on every event INSERT and in the duplicate guard: Edit/MultiEdit/NotebookEdit/Write drop the whole-file mirrors (originalFile, oldString/newString, Write content), Read drops file.content/base64, and any string in tool_input/tool_response over 8 KB keeps a 6 KB head + 2 KB tail. structuredPatch and every field outside those two stay whole; _trimmed names what was dropped. The full text stays in the transcript and its snapshot. Rows stored before this are trimmed once in the background (sweepEventPayloads, 200 rows per write transaction, user_version 4). No VACUUM, no row pruned. The session list now prices every token column /api/pricing/cost/:id does (speed, geo, tier, the 1h cache write, server-tool fees) through one tokensForSessions helper, so a row's cost equals the session's own.
This commit is contained in:
@@ -479,6 +479,29 @@ function startBackgroundServices() {
|
||||
}
|
||||
}
|
||||
|
||||
// One-time (user_version 4) trim of the hook payloads stored before
|
||||
// `trim_event_data` existed — see `server/lib/event-payload.js`. Waits for
|
||||
// marker 3, so it never claims a DB whose token repair has not run yet.
|
||||
{
|
||||
const dbModule = require("./db");
|
||||
if (dbModule.db.pragma("user_version", { simple: true }) === 3) {
|
||||
const t = setTimeout(() => {
|
||||
require("./lib/event-payload")
|
||||
.sweepEventPayloads(dbModule.db)
|
||||
.then((r) => {
|
||||
if (r.rows > 0) {
|
||||
console.log(
|
||||
`Trimmed ${r.rows} stored event payload(s), ${(r.bytesSaved / 1048576).toFixed(0)} MB`
|
||||
);
|
||||
}
|
||||
dbModule.db.pragma("user_version = 4");
|
||||
})
|
||||
.catch((err) => console.warn("event payload trim failed:", err?.message || err));
|
||||
}, 5_000);
|
||||
if (t.unref) t.unref();
|
||||
}
|
||||
}
|
||||
|
||||
// Per-hour tokens for sessions recorded before `token_usage_hourly`, so the
|
||||
// daily cost split covers them too. Picks only sessions with no hourly rows,
|
||||
// so a later start re-reads nothing already done; non-blocking.
|
||||
|
||||
Reference in New Issue
Block a user