Skip to content

bug: Supervisor adopts any process with "openclaw" in its cmdline as the restarted gateway, so a crashed gateway can stay down #194

Open

Description

@eliavb43

Pre-flight checks

  • I updated to the latest add-on version and restarted it.
  • I checked the docs/troubleshooting section first.

What happened?

After the gateway process dies (OOM kill, crash, kill -9), the supervisor loop in openclaw_assistant/run.sh calls find_gateway_daemon_pid to detect a self-restart. Tier 3 returns the first /proc/*/cmdline that contains the substring openclaw and is not a known PID. Many long-lived processes match it:

  • CLI agent children the gateway spawns, e.g. claude ... --mcp-config /tmp/openclaw-cli-mcp-XXXX/mcp.json;
  • the codex app-server under /config/.openclaw/npm/...;
  • a managed Chromium with --user-data-dir=/config/.openclaw/browser/...;
  • any user script stored under /config/.openclaw/.

The loop then logs OpenClaw runtime active (PID <x>); monitoring. and polls that unrelated PID with kill -0 every 5 s. Nothing is listening on the gateway port, and nothing relaunches the gateway, until every matching process has exited. For a long-lived match that can be never. The add-on watchdog (watchdog: tcp://[HOST]:48099) checks the ingress nginx, which stays up, so Supervisor does not restart the add-on either.

Tier 2 has the same shape: pgrep -f openclaw-gateway also matches every openclaw gateway <subcommand> CLI call (OpenClaw sets process.title = "openclaw-" + <command>) and any shell whose command line contains that string.

Note: in a container, OpenClaw 2026.9.x restarts the gateway in-process and keeps its PID. So on current OpenClaw, tiers 2 and 3 only run after a real crash, which is exactly when they mis-adopt.

What did you expect to happen?

After a crash, the supervisor relaunches the gateway, or adopts only a process that really is a gateway and that owns the gateway port.

Steps to reproduce

No crash needed:

  1. With a few agent turns running (so claude CLI children exist), source find_gateway_daemon_pid from run.sh in a shell inside the add-on.
  2. Simulate a dead gateway: set GATEWAY_INTERNAL_PORT to an unused port, make pgrep skip the gateway PID, and set GW_PID to the gateway PID (as after wait returns).
  3. Call find_gateway_daemon_pid. It returns a claude PID. On our system 10 processes were adoptable, and the supervisor would monitor them in turn before any relaunch.

Live, with a controlled kill -KILL <gateway port owner> at 00:33:53 UTC:

00:33:53.63  INFO: OpenClaw runtime active (PID 10880); monitoring.
00:34:18.98  WARN: OpenClaw runtime exited with code 0. Restarting in 2s...
00:34:20.98  Starting OpenClaw Assistant runtime (openclaw)...

PID 10880 was not a gateway. It was an openclaw agent CLI client started 3 s before the kill. It exited on its own within about 2 s, so this time the gateway came back after about 50 s. An earlier OOM kill of the gateway on the same system stayed down until a manual add-on restart. For an adopted process that is not a child, the loop always reports exit code 0.

Add-on version

0.5.94

OpenClaw version (if known)

2026.9.3 (observed); run.sh on main (d3b414d, add-on 0.5.94 with OpenClaw 2026.9.9) has the same find_gateway_daemon_pid.

Access mode

tailnet_https

Add-on logs

00:33:53.63  INFO: OpenClaw runtime active (PID 10880); monitoring.
00:34:18.98  WARN: OpenClaw runtime exited with code 0. Restarting in 2s...
00:34:20.98  Starting OpenClaw Assistant runtime (openclaw)...

Additional context

Possible fix:

  1. Tier 3: match the gateway, not the substring. Accept a candidate only if its argv is the gateway entrypoint (openclaw gateway run, or the process title openclaw-gateway with no further arguments), and its parent is the old GW_PID or PID 1.
  2. Bound every adoption by the port. An adopted non-child must own GATEWAY_INTERNAL_PORT within a deadline (e.g. 120 s; a cold start on a Raspberry Pi takes 45-90 s). Otherwise stop tracking it and relaunch. This alone turns any mis-adoption into a delay instead of an outage. In the case above, tier 3 fired 0.1 s after the kill, which is too early for any real restart.
  3. Point the watchdog at the gateway (its port, or an nginx location that proxies /health), so Supervisor notices a dead gateway behind a live ingress.

Tying CLI children to the gateway does not close this: an agent's shell command, or a script it detaches on purpose, legitimately outlives the gateway and often has /config/.openclaw/... in its argv. Only the supervisor can tell a gateway from a process that mentions openclaw.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions