diff --git a/docs/API.md b/docs/API.md index 50e1dd93..3978811e 100644 --- a/docs/API.md +++ b/docs/API.md @@ -2963,7 +2963,7 @@ POST /api/run/drop-image Raw image body (png/jpeg/gif/webp, ≤20 M POST /api/run/clipboard-files → {files:[{path,size,dir}]}: the files a file manager's Copy put on THIS machine's clipboard (Windows GetFileDropList, macOS NSPasteboard, Linux text/uri-list via wl-paste/xclip), each that exists here (`size` null for a folder) — local mode + loopback only, else 403 `EREMOTE`; a pasted file whose name and size match is attached by that path, no upload and no size limit POST /api/run/selected-files → {files:[{path,size,dir}]}: the same for a DROP — the items selected in THIS machine's file manager, which is what a drag from it carries (Windows Explorer windows and the desktop via Shell.Application, macOS Finder's selection — macOS asks once to let AgentHub control Finder; Linux file managers publish none, so a Linux drop uses the browser's own `file://` list or uploads). Nothing is searched for on disk GET /api/run/native-paste → {native, key}: whether a `claude` pane can read this machine's clipboard itself (see drop-file below), and `key` — `ctrl+v`, or `alt+v` on Windows — the chord Claude Code binds its image paste to there, which the terminal sends -POST /api/run/drop-file?name=&laneId= Any raw file body (≤20 MB; ≤2 GB from a loopback browser outside server mode, streamed to disk — Chrome hands a dropped file no `file://` path) → {path}: same dir (S3: the handoff dir of the account workspace `laneId`'s panes run as, else the requester's), 0600, named -; what the terminal (drop or paste, Ctrl+V included), the prompt box and its paste use for every dropped file or image — except a browser on a loopback host dropping or pasting a file manager's files, whose `file://` paths are attached as they are, uploading nothing, once `POST /api/run/local-paths` finds the same files here, and an image PASTED into a `claude` pane by a loopback browser of a local-mode dashboard, which is sent as ^V (Alt+V on Windows, Claude Code's binding there) so `claude` reads the desktop clipboard itself (Settings → Appearance → **Paste images natively**, on by default, per browser — off for an SSH tunnel) and only while `GET /api/run/native-paste` answers `{native:true}`: local mode, a loopback request, and on Linux a reachable display plus `wl-paste` (Wayland) or `xclip` (X11) — without one the `^V` pasted nothing, so the image is uploaded instead. Both drop routes answer 413 `ETOOBIG` (with `max_bytes`, the limit the UI shows) to a `Content-Length` over their limit before reading the body, and close the connection; of several files only the refused ones are lost +POST /api/run/drop-file?name=&laneId= Any raw file body, streamed to disk as it arrives (S3: through `write-as` as the member) — ≤500 MB, like a Files upload; from a loopback browser outside server mode no cap, only the data dir's free space less a 1 GB reserve (Chrome hands a dropped file no `file://` path) → {path}: same dir (S3: the handoff dir of the account workspace `laneId`'s panes run as, else the requester's), 0600, named -; what the terminal (drop or paste, Ctrl+V included), the prompt box and its paste use for every dropped file or image — except a browser on a loopback host dropping or pasting a file manager's files, whose `file://` paths are attached as they are, uploading nothing, once `POST /api/run/local-paths` finds the same files here, and an image PASTED into a `claude` pane by a loopback browser of a local-mode dashboard, which is sent as ^V (Alt+V on Windows, Claude Code's binding there) so `claude` reads the desktop clipboard itself (Settings → Appearance → **Paste images natively**, on by default, per browser — off for an SSH tunnel) and only while `GET /api/run/native-paste` answers `{native:true}`: local mode, a loopback request, and on Linux a reachable display plus `wl-paste` (Wayland) or `xclip` (X11) — without one the `^V` pasted nothing, so the image is uploaded instead. Both drop routes answer 413 `ETOOBIG` (with `max_bytes`, the limit the UI shows) to a `Content-Length` over their limit before reading the body (a loopback drop: when it would not fit on the disk), and close the connection; of several files only the refused ones are lost POST /api/run/local-paths {paths:[…]} (1–100) → {sizes:[bytes|null…]}: each path's size when it is a regular file on this machine, else null. The composer and the terminal use a dropped file's own `file://` path only when every size matches the dropped bytes' — "localhost" in the address bar is no proof (an SSH tunnel's browser is on another machine). 403 EREMOTE in server mode (a pane runs as an account that cannot read the host's files) and to anything but a loopback socket without X-Forwarded-For/Forwarded; 400 EBADFIELD otherwise GET /api/run/drop/:name A dropped png/jpg/gif/webp, or txt/md/csv/tsv/log/json/xml/yaml as text/plain, served back (nosniff, sandbox CSP) from the requester's own drop dir (S3: their member account's, owner-checked — for an admin, then the host's, where their host-folder workspaces drop); only a name drop-image/drop-file made → 404 otherwise GET /api/run/:id/permission-mode A live claude pane's mode, read off its mode line → {mode} (null while a menu hides it) diff --git a/docs/SERVER-MODE.md b/docs/SERVER-MODE.md index ea1af2f9..734acd04 100644 --- a/docs/SERVER-MODE.md +++ b/docs/SERVER-MODE.md @@ -65,7 +65,7 @@ The proxy has to: - forward WebSocket upgrades (`/ws` for live updates, `/ws-pty/*` for terminals); - set `X-Forwarded-For` and `X-Forwarded-Proto`; - accept request bodies as large as AgentHub's own limits (Files uploads up to - 500 MB, prompt drops 20 MB). nginx's default `client_max_body_size` is 1 MB, + 500 MB, files dropped on a terminal or a prompt box 500 MB, dropped images 20 MB). nginx's default `client_max_body_size` is 1 MB, and its own 413 is an HTML page the UI can only report as a failed upload. nginx: diff --git a/server/__tests__/fixtures/fake-sudo.js b/server/__tests__/fixtures/fake-sudo.js index 2898fbdd..72b58388 100644 --- a/server/__tests__/fixtures/fake-sudo.js +++ b/server/__tests__/fixtures/fake-sudo.js @@ -1,32 +1,34 @@ /** * @file fake-sudo.js * @description Test helper for the S3 suites: an `execImpl` for - * `run-as.__setExecImpl` that plays `sudo -n -u [VAR=v…] …` - * without root — it records the argv (so a test can assert nothing secret + * `run-as.__setExecImpl` (and a `spawn` for `__setSpawnImpl`) that plays + * `sudo -n -u [VAR=v…] …` without root — it records the argv (so a test can assert nothing secret * crossed sudo) and runs the real `scripts/server/write-as.sh` for * `/opt/agenthub/libexec/write-as`, as this account. * @author Nguyễn Ngọc Trí Vĩ */ const path = require("node:path"); -const { execFileSync } = require("node:child_process"); +const { execFileSync, spawn: spawnChild } = require("node:child_process"); const WRITE_AS = path.join(__dirname, "..", "..", "..", "scripts", "server", "write-as.sh"); -/** `{ exec, calls }`: pass `exec` to `runAs.__setExecImpl`; `calls` = every sudo argv. */ +/** `{ exec, spawn, calls }`: pass `exec` to `runAs.__setExecImpl` and `spawn` + * to `runAs.__setSpawnImpl`; `calls` = every sudo argv. */ function fakeSudo() { const calls = []; - const exec = (cmd, argv, opts = {}) => { + // The command sudo would run, as [file, args]. + const unwrap = (cmd, argv) => { calls.push([cmd, ...argv]); - if (cmd !== "sudo") return execFileSync(cmd, argv, opts); + if (cmd !== "sudo") return [cmd, argv]; let i = argv.indexOf("-u") + 2; // past the login while (/^[A-Z_][A-Z0-9_]*=/.test(argv[i])) i++; const [bin, ...rest] = argv.slice(i); - if (bin === "/opt/agenthub/libexec/write-as") - return execFileSync("sh", [WRITE_AS, ...rest], opts); - return execFileSync(bin, rest, opts); + return bin === "/opt/agenthub/libexec/write-as" ? ["sh", [WRITE_AS, ...rest]] : [bin, rest]; }; - return { exec, calls }; + const exec = (cmd, argv, opts = {}) => execFileSync(...unwrap(cmd, argv), opts); + const spawn = (cmd, argv, opts = {}) => spawnChild(...unwrap(cmd, argv), opts); + return { exec, spawn, calls }; } module.exports = { fakeSudo }; diff --git a/server/__tests__/run.test.js b/server/__tests__/run.test.js index 780fa836..c74f2a06 100644 --- a/server/__tests__/run.test.js +++ b/server/__tests__/run.test.js @@ -334,12 +334,12 @@ describe("POST /api/run/drop-image", () => { }); it("refuses a file declared over the limit before its body arrives", async () => { - // 4 GB announced, 1 KB sent: the answer must come without the rest. + // A petabyte announced, 1 KB sent: the answer must come without the rest. const { status, body } = await new Promise((resolve, reject) => { const url = new URL("/api/run/drop-file?name=big.mp4", BASE); const req = http.request(url, { method: "POST", - headers: { "Content-Type": "video/mp4", "Content-Length": 4 * 1024 ** 3 }, + headers: { "Content-Type": "video/mp4", "Content-Length": 1024 ** 5 }, }); req.on("response", (res) => { let text = ""; @@ -354,8 +354,10 @@ describe("POST /api/run/drop-image", () => { }); assert.equal(status, 413); assert.equal(body.error.code, "ETOOBIG"); - // The UI shows the server's limit: a loopback browser's, here. - assert.equal(body.error.max_bytes, 2 * 1024 ** 3); + // A loopback browser has no cap, only the disk's free space (less a + // reserve) — and that is the limit the UI shows. + const st = fs.statfsSync(process.env.DASHBOARD_DATA_DIR); + assert.ok(body.error.max_bytes <= st.bavail * st.bsize, String(body.error.max_bytes)); }); it("streams a loopback drop over the remote limit to disk", async () => { diff --git a/server/__tests__/server-s3-review5.test.js b/server/__tests__/server-s3-review5.test.js index 34daa365..67701371 100644 --- a/server/__tests__/server-s3-review5.test.js +++ b/server/__tests__/server-s3-review5.test.js @@ -97,7 +97,9 @@ before(async () => { ); linux.__resetConfCache(); linux.__setRunImpl(() => "ok"); - runAs.__setExecImpl(fakeSudo().exec); + const sudo = fakeSudo(); + runAs.__setExecImpl(sudo.exec); + runAs.__setSpawnImpl(sudo.spawn); tmux.__setExecImpl(fakeTmux); server = await startServer(createApp(), 0); BASE = `http://127.0.0.1:${server.address().port}`; @@ -708,6 +710,8 @@ describe("a dropped file", () => { byAdmin.body.path.startsWith(path.join(MIA, ".agenthub", "handoff", "drops") + path.sep), byAdmin.body.path ); + // Streamed to the member's file through write-as, not held whole. + assert.equal(fs.readFileSync(byAdmin.body.path, "utf8"), "hello"); // A host workspace's pane runs as the host: the host's own 0700 drops dir. const hostLane = addLane(path.join(TMP, "hostdir", "d"), "boss"); const host = await drop("boss", hostLane); diff --git a/server/routes/run.js b/server/routes/run.js index 5fef6050..0b8fc277 100644 --- a/server/routes/run.js +++ b/server/routes/run.js @@ -25,6 +25,8 @@ const express = require("express"); const { Router } = express; const fs = require("node:fs"); const crypto = require("node:crypto"); +const { Transform } = require("node:stream"); +const { pipeline } = require("node:stream/promises"); const path = require("node:path"); const runs = require("../lib/pty-run"); const tmux = require("../lib/tmux"); @@ -114,21 +116,35 @@ const DROP_TYPES = { "image/webp": "webp", }; const DROP_MAX_BYTES = 20 * 1024 * 1024; -// A browser on this machine (loopback, never server mode) crosses no network: -// its drop is streamed to disk, so a video or a dump is no reason to refuse. -// Chrome hands a dropped file no `file://` path; the client first asks for -// the file manager's own (`/selected-files`, `/clipboard-files`) and only -// uploads what that cannot place. -const DROP_LOCAL_MAX_BYTES = 2 * 1024 ** 3; +// drop-file streams every body to disk, never holding it in memory: from +// another machine (server mode, a LAN browser) up to what a Files upload may +// carry, so a video or a dump dropped on a pane is no reason to refuse. +const DROP_REMOTE_MAX_BYTES = 500 * 1024 * 1024; +// A browser on this machine (loopback, never server mode) crosses no network, +// so its drop has no size cap — only what the disk can take: the copy must +// leave DROP_LOCAL_RESERVE_BYTES free. Chrome hands a dropped file no +// `file://` path; the client first asks for the file manager's own +// (`/selected-files`, `/clipboard-files`) and only uploads what that cannot +// place. +const DROP_LOCAL_RESERVE_BYTES = 1024 ** 3; const DROP_TTL_MS = 24 * 60 * 60_000; const localDrop = (req) => !require("../lib/auth").isServerMode() && isLocalRequest(req); -const dropLimit = (req) => (localDrop(req) ? DROP_LOCAL_MAX_BYTES : DROP_MAX_BYTES); +/** The free space a local drop may fill (a local drop lands in the data dir). */ +function localRoom() { + try { + const st = fs.statfsSync(require("../lib/claude-home").getDataDir()); + return Math.max(0, st.bavail * st.bsize - DROP_LOCAL_RESERVE_BYTES); + } catch { + return Infinity; // unknown: a full disk still fails the write, and leaves no file + } +} +const dropLimit = (req) => (localDrop(req) ? localRoom() : DROP_REMOTE_MAX_BYTES); function tooBig(res, max) { res.set("Connection", "close"); res.status(413).json({ error: { code: "ETOOBIG", - message: `a dropped file may be at most ${max / 1024 / 1024} MB`, + message: `a dropped file may be at most ${Math.floor(max / 1024 / 1024)} MB`, max_bytes: max, }, }); @@ -136,11 +152,11 @@ function tooBig(res, max) { /** A body declared over the limit is refused before a byte of it is read, and * the connection is closed: express.raw would read (and discard) all of it * first, so a 4 GB drop crossed the network in full just to be told 413. */ -function refuseOversize(req, res, next) { - const max = dropLimit(req); +const refuseOversize = (limit) => (req, res, next) => { + const max = limit(req); if (!(Number(req.headers["content-length"]) > max)) return next(); tooBig(res, max); -} +}; /** Where a drop goes: the handoff dir of the account the PANE runs as — the * workspace `?laneId=` names (scopeGuard already refused one this user may * not see), else the requester's own — written AS that member under S3, so @@ -195,7 +211,8 @@ function saveDrop(req, res, name, body) { router.post( "/drop-image", - refuseOversize, + // An image is held in memory: 20 MB, wherever the browser is. + refuseOversize(() => DROP_MAX_BYTES), express.raw({ type: Object.keys(DROP_TYPES), limit: DROP_MAX_BYTES }), (req, res) => { const ext = @@ -222,63 +239,63 @@ const dropName = (req) => .replace(/^\.+/, "") .slice(-100); -/** A local browser's drop is piped to its file as it arrives, never held in - * memory whole; a body that runs past the limit (or is empty) leaves no file. */ -function streamLocalDrop(req, res, next) { - if (!localDrop(req)) return next(); +/** A drop is piped to its file as it arrives, never held in memory whole — + * a member's (S3) through `write-as` as them, its bytes on stdin; a body that + * runs past the limit (or is empty, or is cut off) leaves no file. */ +async function streamDrop(req, res) { const safe = dropName(req); if (!safe) return res.status(400).json({ error: "?name= and a non-empty file body required" }); + const runAs = require("../lib/run-as"); + const max = dropLimit(req); let file; - let out; + let member; try { - const { dir, member } = prepareDropDir(req); - if (member) return next(); // a member's file is written as them, by saveDrop - file = path.join(dir, `${crypto.randomBytes(8).toString("hex")}-${safe}`); - out = fs.createWriteStream(file, { mode: 0o600, flags: "wx" }); + const dropped = prepareDropDir(req); + member = dropped.member; + file = path.join(dropped.dir, `${crypto.randomBytes(8).toString("hex")}-${safe}`); } catch (err) { return res.status(500).json({ error: err.message }); } let bytes = 0; - let failed = null; - const fail = (status, send) => { - if (failed) return; - failed = status; - req.unpipe(out); - out.destroy(); - fs.rm(file, { force: true }, () => {}); - if (!res.headersSent) send(); - }; - req.on("data", (c) => { - bytes += c.length; - if (bytes > DROP_LOCAL_MAX_BYTES) fail(413, () => tooBig(res, DROP_LOCAL_MAX_BYTES)); + const count = new Transform({ + transform(chunk, _enc, cb) { + bytes += chunk.length; + if (bytes > max) return cb(Object.assign(new Error("too big"), { code: "ETOOBIG" })); + cb(null, chunk); + }, }); - req.on("aborted", () => fail(400, () => {})); - out.on("error", (err) => fail(500, () => res.status(500).json({ error: err.message }))); - out.on("finish", () => { - if (failed) return; - if (bytes === 0) { - return fail(400, () => - res.status(400).json({ error: "?name= and a non-empty file body required" }) - ); + try { + if (member) { + await Promise.all([ + pipeline(req, count).catch((e) => { + count.destroy(e); + throw e; + }), + runAs.writeAsAsync(member, file, ["replace"], { input: count }), + ]); + } else { + await pipeline(req, count, fs.createWriteStream(file, { mode: 0o600, flags: "wx" })); } - res.json({ path: file }); - }); - req.pipe(out); -} - -router.post( - "/drop-file", - refuseOversize, - streamLocalDrop, - express.raw({ type: () => true, limit: DROP_MAX_BYTES }), - (req, res) => { - const safe = dropName(req); - if (!safe || !Buffer.isBuffer(req.body) || req.body.length === 0) { + if (bytes === 0) throw Object.assign(new Error("empty"), { code: "EEMPTY" }); + } catch (err) { + try { + if (member) runAs.removeAs(member, file); + else fs.rmSync(file, { force: true }); + } catch { + /* never written */ + } + // Past the limit mid-body, pipeline has already closed the socket. + if (res.headersSent) return; + if (err.code === "ETOOBIG") return tooBig(res, max); + if (err.code === "EEMPTY") { return res.status(400).json({ error: "?name= and a non-empty file body required" }); } - saveDrop(req, res, `${crypto.randomBytes(8).toString("hex")}-${safe}`, req.body); + return res.status(500).json({ error: err.message }); } -); + res.json({ path: file }); +} + +router.post("/drop-file", refuseOversize(dropLimit), streamDrop); // A browser on this machine is handed a file manager's files as `file://` // paths beside their bytes; the composer attaches such a path instead of