fix(bootstrap): never stop the dashboard from inside its own systemd cgroup

A plugin self-update took the dashboard down and left it down. The chain
ran correctly up to the point where the bootstrap SIGTERMs the server it is
replacing — and that server was systemd-managed, with the bootstrap running
as its descendant, i.e. inside its cgroup. Two failures fired at once:

  * the server exits 0, and `Restart=on-failure` reads a clean exit as
    "meant to stop", so systemd did not bring it back;
  * `KillMode=control-group` reaped the bootstrap itself, mid-run, before it
    could repoint the unit at the new plugin root or write state.json.

Evidence from the incident: bootstrap.log ends at "stopped the server
started from the previous plugin version" with no systemd line after it,
.bootstrap.lock survived with a dead pid, state.json still named the
previous version, and the unit's ExecStart still pointed at the old cache
dir. The same truncation is visible on three earlier updates — those only
recovered because a later SessionStart bootstrapped again.

systemd-service.js now splits install into writeUserService (repoints the
unit, starts nothing) and restartUserService (--no-block). The bootstrap
skips the manual stop whenever a unit of ours owns the server, and defers
the restart past writeState and releaseLock, so being killed by its own
request costs nothing. The unit is Restart=always: a graceful SIGTERM from
anything must no longer read as "leave it stopped".

installUserService keeps its old semantics for `agenthub service install`,
which runs from a user's shell and is not in the cgroup.
This commit is contained in:
2026-09-11 16:30:10 +07:00
parent 0335e0e6bc
commit de8d1079e0
6 changed files with 334 additions and 38 deletions
+4 -1
View File
@@ -456,7 +456,10 @@ Environment=NODE_ENV=production
Environment=PORT=4820
Environment=DASHBOARD_DB_PATH=/var/lib/agenthub/dashboard.db
ExecStart=/usr/bin/node server/index.js
Restart=on-failure
# `always`, not `on-failure`: a graceful SIGTERM exits 0, which `on-failure`
# reads as "meant to stop" — one stray kill and the dashboard stays down.
# `systemctl stop` still stops it. Same reasoning as the plugin's own unit.
Restart=always
RestartSec=10
StandardOutput=journal
StandardError=journal
+40 -2
View File
@@ -144,6 +144,9 @@ waits on an install:
`client-dist` — scoped to this session, same as before this fell back to
systemd.
9. **Record state** — `runtime/state.json`.
10. **Hand over to systemd** — only when a unit of ours already owns the server
(see [Who may stop the server](#who-may-stop-the-server)). Deliberately
after step 9.
The first run takes a few minutes — most of it is the UI build, which is what
makes client-only routes (`http://localhost:4820/run`, and every other page)
@@ -172,6 +175,41 @@ On macOS, Windows, or a Linux box without systemd, nothing changes: the
server is a detached spawn started fresh on the next `SessionStart` after a
reboot, same as always.
### Who may stop the server
**When a `agenthub.service` unit of ours exists, nothing else may stop the
dashboard.** The bootstrap that performs a self-update is spawned by the
running server, so it lives inside that server's systemd cgroup. SIGTERM-ing
the server from there breaks two ways at once:
- the server exits **0**, and the unit's old `Restart=on-failure` read a clean
exit as "meant to stop" — so systemd did not bring it back;
- `KillMode=control-group` (systemd's default) reaps **every** process in the
cgroup when the main process exits, including the bootstrap itself — mid-run,
before it could repoint the unit at the new plugin root or write
`runtime/state.json`.
The result was a dashboard that went down on every self-update and stayed down
until some later Claude Code session happened to bootstrap again, leaving a
stale `.bootstrap.lock` and a `state.json` still naming the previous version.
So the order is fixed, and `scripts/systemd-service.js` splits its API to
enforce it:
| Situation | What the bootstrap does |
|---|---|
| A unit of ours owns the server (`unitIsOurs`) | Never stops it. `writeUserService` repoints the unit at the new plugin root **without starting anything**, the rest of the bootstrap runs, state is written, the lock is released — and only then `restartUserService` asks systemd to restart, with `--no-block` so being killed by its own request costs nothing. |
| No unit, or a foreign one | Unchanged: stop the old server, then `installUserService` (or a detached spawn where there is no systemd). Nothing here is inside a unit's cgroup, so the stop is safe. |
The unit is written with **`Restart=always`**, not `on-failure`: a graceful
SIGTERM from anything at all would otherwise leave the dashboard down for good.
An explicit `systemctl stop` — which is what `agenthub service uninstall` and
the Settings panel do — still stops it.
`agenthub service install` (the checkout path) keeps using `installUserService`:
it runs from your own shell, outside the unit's cgroup, so it may block and
start in one step.
### Commands
| Command | Does |
@@ -268,8 +306,8 @@ not assumed. It does **not**:
For a clean machine, do all of this yourself:
```bash
# 1. Stop the server. On Linux, check for the systemd unit first — a bare
# `kill` just gets restarted by it (Restart=on-failure).
# 1. Stop the server. On Linux, disable the systemd unit FIRST — the unit is
# Restart=always, so a bare `kill` is always undone by systemd.
systemctl --user disable --now agenthub.service 2>/dev/null
rm -f ~/.config/systemd/user/agenthub.service && systemctl --user daemon-reload 2>/dev/null
# Fallback for a detached (non-systemd) server, or if the unit above was absent: