fix(bootstrap): never stop the dashboard from inside its own systemd cgroup
A plugin self-update took the dashboard down and left it down. The chain
ran correctly up to the point where the bootstrap SIGTERMs the server it is
replacing — and that server was systemd-managed, with the bootstrap running
as its descendant, i.e. inside its cgroup. Two failures fired at once:
* the server exits 0, and `Restart=on-failure` reads a clean exit as
"meant to stop", so systemd did not bring it back;
* `KillMode=control-group` reaped the bootstrap itself, mid-run, before it
could repoint the unit at the new plugin root or write state.json.
Evidence from the incident: bootstrap.log ends at "stopped the server
started from the previous plugin version" with no systemd line after it,
.bootstrap.lock survived with a dead pid, state.json still named the
previous version, and the unit's ExecStart still pointed at the old cache
dir. The same truncation is visible on three earlier updates — those only
recovered because a later SessionStart bootstrapped again.
systemd-service.js now splits install into writeUserService (repoints the
unit, starts nothing) and restartUserService (--no-block). The bootstrap
skips the manual stop whenever a unit of ours owns the server, and defers
the restart past writeState and releaseLock, so being killed by its own
request costs nothing. The unit is Restart=always: a graceful SIGTERM from
anything must no longer read as "leave it stopped".
installUserService keeps its old semantics for `agenthub service install`,
which runs from a user's shell and is not in the cgroup.
This commit is contained in:
+40
-2
@@ -144,6 +144,9 @@ waits on an install:
|
||||
`client-dist` — scoped to this session, same as before this fell back to
|
||||
systemd.
|
||||
9. **Record state** — `runtime/state.json`.
|
||||
10. **Hand over to systemd** — only when a unit of ours already owns the server
|
||||
(see [Who may stop the server](#who-may-stop-the-server)). Deliberately
|
||||
after step 9.
|
||||
|
||||
The first run takes a few minutes — most of it is the UI build, which is what
|
||||
makes client-only routes (`http://localhost:4820/run`, and every other page)
|
||||
@@ -172,6 +175,41 @@ On macOS, Windows, or a Linux box without systemd, nothing changes: the
|
||||
server is a detached spawn started fresh on the next `SessionStart` after a
|
||||
reboot, same as always.
|
||||
|
||||
### Who may stop the server
|
||||
|
||||
**When a `agenthub.service` unit of ours exists, nothing else may stop the
|
||||
dashboard.** The bootstrap that performs a self-update is spawned by the
|
||||
running server, so it lives inside that server's systemd cgroup. SIGTERM-ing
|
||||
the server from there breaks two ways at once:
|
||||
|
||||
- the server exits **0**, and the unit's old `Restart=on-failure` read a clean
|
||||
exit as "meant to stop" — so systemd did not bring it back;
|
||||
- `KillMode=control-group` (systemd's default) reaps **every** process in the
|
||||
cgroup when the main process exits, including the bootstrap itself — mid-run,
|
||||
before it could repoint the unit at the new plugin root or write
|
||||
`runtime/state.json`.
|
||||
|
||||
The result was a dashboard that went down on every self-update and stayed down
|
||||
until some later Claude Code session happened to bootstrap again, leaving a
|
||||
stale `.bootstrap.lock` and a `state.json` still naming the previous version.
|
||||
|
||||
So the order is fixed, and `scripts/systemd-service.js` splits its API to
|
||||
enforce it:
|
||||
|
||||
| Situation | What the bootstrap does |
|
||||
|---|---|
|
||||
| A unit of ours owns the server (`unitIsOurs`) | Never stops it. `writeUserService` repoints the unit at the new plugin root **without starting anything**, the rest of the bootstrap runs, state is written, the lock is released — and only then `restartUserService` asks systemd to restart, with `--no-block` so being killed by its own request costs nothing. |
|
||||
| No unit, or a foreign one | Unchanged: stop the old server, then `installUserService` (or a detached spawn where there is no systemd). Nothing here is inside a unit's cgroup, so the stop is safe. |
|
||||
|
||||
The unit is written with **`Restart=always`**, not `on-failure`: a graceful
|
||||
SIGTERM from anything at all would otherwise leave the dashboard down for good.
|
||||
An explicit `systemctl stop` — which is what `agenthub service uninstall` and
|
||||
the Settings panel do — still stops it.
|
||||
|
||||
`agenthub service install` (the checkout path) keeps using `installUserService`:
|
||||
it runs from your own shell, outside the unit's cgroup, so it may block and
|
||||
start in one step.
|
||||
|
||||
### Commands
|
||||
|
||||
| Command | Does |
|
||||
@@ -268,8 +306,8 @@ not assumed. It does **not**:
|
||||
For a clean machine, do all of this yourself:
|
||||
|
||||
```bash
|
||||
# 1. Stop the server. On Linux, check for the systemd unit first — a bare
|
||||
# `kill` just gets restarted by it (Restart=on-failure).
|
||||
# 1. Stop the server. On Linux, disable the systemd unit FIRST — the unit is
|
||||
# Restart=always, so a bare `kill` is always undone by systemd.
|
||||
systemctl --user disable --now agenthub.service 2>/dev/null
|
||||
rm -f ~/.config/systemd/user/agenthub.service && systemctl --user daemon-reload 2>/dev/null
|
||||
# Fallback for a detached (non-systemd) server, or if the unit above was absent:
|
||||
|
||||
Reference in New Issue
Block a user