feat(clone): background clone jobs that survive a reload and a server restart
Both clone POSTs take background: true: every check answers synchronously,
then 202 {job}. git runs in a detached runner (scripts/clone-runner.js, its
own process group, stderr to a file — no SIGPIPE when the server restarts);
the server tails it for progress and runs the after-step (workspace / extra
folder) once when it exits. Jobs persist in <data dir>/runtime/clone-jobs.json
(0600, atomic, throttled progress writes); on start a live runner is adopted
(counted against the clone limit), an ended one finished, a queued one
requeued (interrupted if it carried a typed login), the rest interrupted.
GET /clone/jobs, DELETE /clone/jobs/:id, POST /clone/jobs/:id/cancel (queued:
dequeued; running: SIGTERM to the runner's group -> cancelled, no after-step)
and /retry (new job from the stored request, never a credential). The
held-open path is unchanged; worktree.gitCommand builds the runner's argv
and git-identity.cloneFailure the shared error classification.
This commit is contained in:
+52
-1
@@ -463,6 +463,57 @@ own config (`core.sshCommand` naming the key, or a member's
|
||||
`credential.helper=store --file=…`), so later fetch/pull/push — the dashboard's
|
||||
and an agent's — use it too; the user's global git config is never touched.
|
||||
|
||||
#### Background clone jobs
|
||||
|
||||
```
|
||||
POST /api/lanes/clone Body: { …as above, background: true } → 202 { job }
|
||||
POST /api/lanes/:id/sources/clone Body: { …, background: true } → 202 { job }
|
||||
GET /api/lanes/clone/jobs → 200 { jobs: CloneJob[] }
|
||||
DELETE /api/lanes/clone/jobs/:id → 200 { ok: true } (404 ENOTFOUND, 409 EBUSY while queued/running)
|
||||
POST /api/lanes/clone/jobs/:id/cancel → 200 { job } (404 ENOTFOUND, 409 EDONE once finished)
|
||||
POST /api/lanes/clone/jobs/:id/retry Body: { credentials? } → 202 { job } (404 ENOTFOUND, 409 EBUSY unless failed/interrupted/cancelled)
|
||||
|
||||
CloneJob = { id, kind: "workspace"|"source", state: "queued"|"running"|"done"|"failed"|"interrupted"|"cancelled",
|
||||
url, name, parent, target, laneId (source: its workspace), position (1-based while queued),
|
||||
phase, label, percent, detail, error: { code, message, auth? } | null,
|
||||
result: { laneId, path, created? } | null, createdAt, finishedAt,
|
||||
request: { url, parent, name, tool?, title?, partial? }, pid }
|
||||
```
|
||||
|
||||
`background: true` (`400 EBADFIELD` when not a boolean) runs every check of
|
||||
the clone POST synchronously — a refusal still answers 4xx at once — then
|
||||
answers `202 { job }` and clones in the background
|
||||
(`server/lib/clone-jobs.js`); without it both routes hold the request open as
|
||||
above. git runs in a **detached runner** (`scripts/clone-runner.js`: its own
|
||||
session and process group, the exact command `worktree.git()` would run —
|
||||
`sudo -n -u ah-<name> git …` for a member — with stderr appended to
|
||||
`<data dir>/runtime/clone-jobs/<id>/stderr.log` and `exit.json` written when
|
||||
git ends), so a clone survives a browser reload AND a server restart or
|
||||
update (the unit is `KillMode=process`; a git whose stderr was a pipe into the
|
||||
server died of SIGPIPE when it restarted). The server tails the log for
|
||||
`phase`/`percent`/`detail`, and when the runner ends it creates the workspace
|
||||
(or adds the extra folder) and broadcasts it exactly as the held-open route,
|
||||
once; a failure records that route's own `error.code`/`message`
|
||||
(`ECREDENTIALS` with `auth`, `ESSHAUTH`, `ENOREPO`, `EEXISTS`, `ECLONE`…). Jobs
|
||||
live in `<data dir>/runtime/clone-jobs.json` (0600, tmp + rename; progress at
|
||||
most every 2 s, every state change at once). On start the server **adopts** a
|
||||
job whose runner is still alive (counted against `AGENTHUB_CLONE_CONCURRENCY`,
|
||||
tailed, finished when it exits), finishes one whose `exit.json` is already
|
||||
there, requeues a queued one, and marks the rest `interrupted`
|
||||
(`EINTERRUPTED`, "AgentHub restarted while this clone ran") — a queued job
|
||||
that carried a typed login is interrupted too, so the login is typed again.
|
||||
`cancel` takes a queued job out of the queue or SIGTERMs the runner's process
|
||||
group (git removes its half-made folder; sudo relays the signal to a member's
|
||||
git); the job ends `cancelled` (`ECANCELLED`) and never makes its workspace or
|
||||
extra folder. `retry` starts a NEW job from the old one's `request` through
|
||||
every check of a fresh POST, then drops the old one. `request` never holds a
|
||||
credential: a typed `credentials` login is used for that one run (the host's
|
||||
travels in the runner's environment, as it would in git's) — a host
|
||||
`remember` is saved only when the same server process sees the clone succeed.
|
||||
Done jobs are kept 24 h, failed/interrupted/cancelled ones 7 days. A job is
|
||||
only ever looked up among the caller's own (`scope.ownerFor`; local mode:
|
||||
every job), 404 otherwise. The browser polls; there is no websocket message.
|
||||
|
||||
#### Update a workspace
|
||||
|
||||
```http
|
||||
@@ -1745,7 +1796,7 @@ POST /api/locks/:name/release
|
||||
GET /api/lanes/:id/sources
|
||||
PUT /api/lanes/:id/instructions {"instructions": "…" | null}
|
||||
POST /api/lanes/:id/sources/folder {"path": "/abs/dir"}
|
||||
POST /api/lanes/:id/sources/clone {"url": "https://host/o/r.git", "name"?: "r", "credentials"?: {username, password, remember?}}
|
||||
POST /api/lanes/:id/sources/clone {"url": "https://host/o/r.git", "name"?: "r", "credentials"?: {username, password, remember?}, "background"?: true}
|
||||
POST /api/lanes/:id/sources/file raw body, Content-Type: application/octet-stream, X-File-Name: <URI-encoded name>
|
||||
DELETE /api/lanes/:id/sources/:sourceId
|
||||
GET /api/lanes/:id/memory
|
||||
|
||||
Reference in New Issue
Block a user