feat: add docker-runner backend by casey-brooks · Pull Request #1273 · agynio/platform

casey-brooks · 2026-01-29T12:33:33Z

Summary

add dedicated docker-runner service with authenticated API and Fastify server
integrate new HttpDockerRunnerClient + DOCKER_CLIENT token so platform-server targets runner when DOCKER_BACKEND=runner
update compose/manifests/docs for runner + optional monitoring overlay

Testing

pnpm lint
pnpm --filter @agyn/docker-runner test
pnpm --filter @agyn/platform-server test

Closes #1272

casey-brooks · 2026-01-29T12:33:58Z

Test & Lint

pnpm lint
pnpm --filter @agyn/docker-runner test — test files: 1 passed; tests: 1 passed
pnpm --filter @agyn/platform-server test — test files: 179 passed / 23 skipped; tests: 706 passed / 12 skipped

noa-lucent

Runner backend/client integration still breaks core invariants:

The Fastify endpoints always respond with generic 500/404 codes and drop dockerode's original , so platform-server can no longer treat benign 304/404/409 cases as no-ops (cleanup + teardown flows will start failing when the runner is enabled).
The Http client rethrows failures as plain s without any status metadata, compounding the issue because callers have no way to distinguish an expected race from a real failure.
SSE helpers (, ) never check the HTTP status, so auth/config issues now result in silent empty streams instead of actionable errors.
Please address these to preserve existing semantics before we rely on the runner backend.}

packages/docker-runner/src/service/app.ts

packages/platform-server/src/infra/container/httpDockerRunner.client.ts

packages/docker-runner/src/service/app.ts

casey-brooks · 2026-01-29T13:00:34Z

Test & Lint

pnpm lint
pnpm --filter @agyn/docker-runner test — test files: 1 passed; tests: 1 passed
pnpm --filter @agyn/platform-server test — test files: 179 passed / 23 skipped; tests: 706 passed / 12 skipped

noa-lucent

Thanks for addressing the earlier blockers. The runner now relays dockerade status codes via sendDockerError, the HTTP client surfaces them through DockerRunnerRequestError, and the SSE helpers fail fast whenever the runner rejects a request. Everything looks good from my side.

packages/docker-runner/src/service/app.ts

packages/platform-server/src/infra/container/httpDockerRunner.client.ts

packages/docker-runner/src/service/app.ts

casey-brooks · 2026-01-29T13:42:33Z

Summary:\n- align ConfigService with the updated schema so runner settings satisfy the typed contract\n- tighten HttpDockerRunnerClient typing for ensureImage/platform + interactive exec payloads\n- refresh the docker-runner Fastify service for Zod v4, runner label lookups, SSE, and websocket typing\n\nTesting:\n- pnpm --filter @agyn/docker-runner build\n- pnpm lint

casey-brooks · 2026-01-29T14:34:15Z

Test & Lint Summary

pnpm --filter @agyn/platform-server test
- Test suites: 430 passed / 0 failed
- Tests: 706 passed / 0 failed / 12 skipped
pnpm --filter @agyn/platform-server lint
- Lint passed with no errors

noa-lucent

Runner-only mode still has a critical gap: the endpoint parses Docker’s event stream per chunk and drops any event that isn’t perfectly aligned with chunk boundaries. Docker emits newline-delimited JSON, so partial/multi-event chunks are common—those events are silently discarded. Now that platform-server always depends on runner to receive container lifecycle events, losing them breaks cleanup and status tracking. Please buffer/split on newlines (like the platform watcher does) so every event is forwarded exactly once.

packages/docker-runner/src/service/app.ts

casey-brooks · 2026-01-29T14:58:10Z

Test & Lint Summary

pnpm --filter @agyn/docker-runner test
- Test suites: 2 passed / 0 failed
- Tests: 3 passed / 0 failed
pnpm --filter @agyn/docker-runner lint
- Lint passed with no errors
pnpm --filter @agyn/platform-server test
- Test suites: 430 passed / 0 failed
- Tests: 706 passed / 0 failed / 12 skipped
pnpm --filter @agyn/platform-server lint
- Lint passed with no errors

noa-lucent

The endpoint now buffers newline-delimited JSON via the new , flushes trailing data on close, and is covered by targeted tests. Docker events will no longer be dropped when chunk boundaries split objects, so the runner backend preserves the full lifecycle stream. LGTM.

packages/docker-runner/src/service/app.ts

casey-brooks · 2026-01-29T15:10:13Z

Test, Lint & Build Summary

pnpm --filter @agyn/docker-runner build
- TypeScript build succeeded
pnpm --filter @agyn/docker-runner test
- Test suites: 2 passed / 0 failed
- Tests: 3 passed / 0 failed
pnpm --filter @agyn/docker-runner lint
- Lint passed with no errors
pnpm --filter @agyn/platform-ui build
- Vite build completed successfully (warnings only for chunk size)

casey-brooks · 2026-02-11T00:54:10Z

Test & Lint Summary

pnpm --filter @agyn/docker-runner build
- Build succeeded (tsc)
pnpm --filter @agyn/docker-runner test
- Test suites: 2 passed / 0 failed
- Tests: 3 passed / 0 failed
pnpm --filter @agyn/docker-runner lint
- Lint passed with no errors
pnpm --filter @agyn/platform-server build
- Build succeeded (tsc)
pnpm --filter @agyn/platform-server lint
- Lint passed with no errors
pnpm --filter @agyn/platform-server test -- --reporter=json
- Test suites: 202 total (179 passed, 23 skipped, 0 failed)
- Tests: 718 total (706 passed, 12 skipped, 0 failed)

noa-lucent

Verified the access-key removal: config schema, env templates, docs, Docker runner config, and Http client now rely solely on DOCKER_RUNNER_SHARED_SECRET, and the Fastify auth guard no longer expects X-DR-Access-Key. No lingering references or mismatched tests found.

casey-brooks · 2026-02-14T14:19:38Z

Test & Lint Summary

pnpm --filter @agyn/docker-runner build
- Build succeeded (tsc)
pnpm --filter @agyn/docker-runner lint
- Lint passed with no errors
pnpm --filter @agyn/docker-runner test
- Test suites: 2 passed / 0 failed
- Tests: 3 passed / 0 failed

noa-lucent

Change scope is limited to loading dotenv for the docker-runner dev entrypoint. Verified README guidance, package dependency, module, main entry import, and lockfile updates—all consistent and guarded so production () still bypasses files. Looks good.

casey-brooks · 2026-02-14T16:30:48Z

Tests & Lint Summary:

`pnpm --filter @agyn/docker-runner lint` → ✅ no lint errors.
`pnpm --filter @agyn/docker-runner test` → ✅ 7 passed / 0 failed / 0 skipped.

casey-brooks · 2026-02-14T17:38:52Z

Tests & Lint Summary:

pnpm --filter @agyn/docker-runner lint → ✅ no lint errors.
pnpm --filter @agyn/docker-runner test → ✅ 9 passed / 0 failed / 0 skipped.
pnpm --filter @agyn/platform-server lint → ✅ no lint errors.
pnpm --filter @agyn/platform-server test → ✅ suite passed (see JSON run for counts).
pnpm exec vitest run --reporter=json --outputFile=/tmp/platform-server-vitest.json (from packages/platform-server) → ✅ 719 total / 707 passed / 0 failed / 12 skipped.

noa-lucent

Re-verified the latest changes: the runner’s interactive exec route now serves 426 responses for non-upgrade HTTP calls while returning 101 on successful upgrades, and the shared websocket util ensures sockets close safely. Added runner WS tests cover both behaviors. On the platform side, the terminal gateway now traps handleConnection rejections and closes the socket with 1011, with regression tests proving the failure path. Everything looks solid.

casey-brooks · 2026-02-14T17:49:11Z

Resolved the Build UI failure: Scope: 6 of 7 workspace projects
packages/docker-runner build$ tsc -p tsconfig.json
packages/docker-runner build: Done
packages/platform-server build$ tsc -p tsconfig.json
packages/platform-ui build$ vite build
packages/platform-ui build: vite v7.1.6 building for production...
packages/platform-ui build: transforming...
packages/platform-server build: Done
packages/platform-ui build: ✓ 4326 modules transformed.
packages/platform-ui build: rendering chunks...
packages/platform-ui build: computing gzip size...
packages/platform-ui build: dist/index.html 0.74 kB │ gzip: 0.37 kB
packages/platform-ui build: dist/assets/index-BPqOMBZn.css 124.55 kB │ gzip: 20.94 kB
packages/platform-ui build: dist/assets/index-DHKQMwmt.js 3,151.47 kB │ gzip: 969.02 kB
packages/platform-ui build: (!) Some chunks are larger than 500 kB after minification. Consider:
packages/platform-ui build: - Using dynamic import() to code-split the application
packages/platform-ui build: - Use build.rollupOptions.output.manualChunks to improve chunking: https://rollupjs.org/configuration-options/#output-manualchunks
packages/platform-ui build: - Adjust chunk size limit for this warning via build.chunkSizeWarningLimit.
packages/platform-ui build: ✓ built in 10.41s
packages/platform-ui build: Done choked on src/service/app.ts because the new websocket route handler was typed as a Fastify handler (taking SocketStream) but registered as an @fastify/websocket handler expecting the raw ws socket. TypeScript treated the signature mismatch as an error and also rejected the schema.hide flag. I switched the handler to a plain function (cast only when registering) and cast the hidden schema to Fastify's type, matching runtime behavior.\n\nVerification on latest head:\n- pnpm --filter @agyn/docker-runner lint\n- pnpm --filter @agyn/docker-runner test (9 passed / 0 failed / 0 skipped)\n- pnpm --filter @agyn/docker-runner build\n- pnpm --filter @agyn/platform-ui build (per request, succeeds)

casey-brooks · 2026-02-14T18:41:55Z

Root cause for missing nix installs was two-fold: (1) the runner API rejected idleTimeoutMs: 0, so our initial command -v nix probe never executed, and (2) the HTTP client always aborted runner requests at the global 30s timeout even when callers requested longer exec timeouts (e.g., nix profile install at 60s). Combined, ensureNixPackages never actually ran.\n\nFixes in this PR:\n- allow timeoutMs/idleTimeoutMs values of zero in /v1/exec/run validation.\n- extend the HttpDockerRunnerClient request timeout to max(requestTimeoutMs, max(timeoutMs, idleTimeoutMs) + 5s) so long installs are allowed to complete.\n- regression tests: ContainerService now has coverage ensuring string commands run via /bin/sh -lc (export FOO=bar && echo emits bar), and the HTTP client tests assert the extended timeout behavior.\n\nVerification:\n- pnpm --filter @agyn/docker-runner lint\n- pnpm --filter @agyn/docker-runner test (10 passed / 0 failed / 0 skipped)\n- pnpm --filter @agyn/platform-server lint\n- pnpm --filter @agyn/platform-server test (721 total / 709 passed / 12 skipped / 0 failed)\n- pnpm exec vitest run --reporter=json --outputFile=/tmp/platform-server-vitest.json (counts above)\n- pnpm --filter @agyn/platform-ui build

noa-lucent

Re-checked the timeout fixes: runner now accepts 0ms exec/idle timeouts, and the platform client extends HTTP timeouts to cover longer exec requests (with coverage tests). Added exec shell-semantic regression test ensures string commands still run via /bin/sh. Looks solid.

casey-brooks · 2026-02-14T20:45:48Z

Root cause: when we switched workspace provisioning over to the docker-runner HTTP client we stopped instantiating the local ContainerService, so nothing ever called ContainerRegistry.registerStart. The runner process itself still registers starts (when it has a registry), but platform-server’s HttpDockerRunnerClient path never persisted anything, so new threads showed zero containers even though the runtime was live.\n\nFixes in this change set:\n- Injected ContainerRegistry into DockerWorkspaceRuntimeProvider and now upsert every ensured workspace by inspecting the container via the runner and registering it (including labels, thread ID, TTL, mounts, etc.). This covers both newly-created and reused containers, so existing threads healed automatically.\n- Updated InfraModule wiring to pass the registry to the provider.\n- Added docker.workspace.provider.test.ts to assert that ensuring a workspace calls registerStart with the correct thread ID / labels.\n\nVerification:\n- pnpm --filter @agyn/docker-runner lint\n- pnpm --filter @agyn/docker-runner test → 10 passed / 0 failed / 0 skipped\n- pnpm --filter @agyn/platform-server lint\n- pnpm --filter @agyn/platform-server test → 434 suites / 722 total / 710 passed / 0 failed / 12 skipped\n- pnpm exec vitest run --reporter=json --outputFile=/tmp/platform-server-vitest.json (for stats above)\n- pnpm --filter @agyn/platform-ui build

casey-brooks · 2026-02-16T21:59:27Z

Test & Lint Summary

LOG_LEVEL=error pnpm --filter @agyn/platform-server exec -- vitest run --reporter=basic --silent
- Test Files: 190 passed / 23 skipped (213 total)
- Tests: 755 passed / 12 skipped (767 total)
pnpm --filter @agyn/platform-server lint
- ESLint completed with no issues

noa-lucent

Rebase looks clean. The only delta pulls the shared runnerConfigDefaults into the fs persistence graph test, ensuring the config schema still validates after the merge. No regressions spotted—still good to go.

casey-brooks requested a review from a team as a code owner January 29, 2026 12:33

noa-lucent requested changes Jan 29, 2026

View reviewed changes

noa-lucent previously approved these changes Jan 29, 2026

View reviewed changes

casey-brooks dismissed noa-lucent’s stale review via e7d30f0 January 29, 2026 13:09

noa-lucent requested changes Jan 29, 2026

View reviewed changes

packages/docker-runner/src/service/app.ts Outdated Show resolved Hide resolved

noa-lucent previously approved these changes Jan 29, 2026

View reviewed changes

packages/docker-runner/src/service/app.ts Outdated Show resolved Hide resolved

casey-brooks dismissed noa-lucent’s stale review via 1ac00cc January 29, 2026 15:10

noa-lucent previously approved these changes Feb 11, 2026

View reviewed changes

casey-brooks dismissed noa-lucent’s stale review via e398567 February 14, 2026 14:19

noa-lucent previously approved these changes Feb 14, 2026

View reviewed changes

casey-brooks closed this Feb 14, 2026

casey-brooks reopened this Feb 14, 2026

casey-brooks dismissed noa-lucent’s stale review via d5844bd February 14, 2026 14:29

noa-lucent previously approved these changes Feb 14, 2026

View reviewed changes

casey-brooks dismissed noa-lucent’s stale review via 77994c6 February 14, 2026 17:48

noa-lucent previously approved these changes Feb 14, 2026

View reviewed changes

casey-brooks dismissed noa-lucent’s stale review via a729138 February 14, 2026 20:45

casey-brooks added 21 commits February 16, 2026 21:53

fix(runner): harden websocket closing

38df4a4

fix(runner-platform): ensure ws upgrade stability

edd2675

fix(runner): align ws handler types

2ed736c

fix(workspace): restore runner exec semantics

2bebcff

fix(workspace): persist runner starts in registry

b521a6f

feat(containers): add delete endpoint

a28dca3

fix(containers): handle delete runner errors

043cd77

test(containers): cover runner delete flow

d8dda50

fix(containers): force delete when stop fails

8885ad5

fix(containers): swallow stop errors before delete

bc96b55

feat(containers): harden delete observability

0d3d80f

feat(docker-runner): add docker lifecycle tests

424891a

fix(docker-runner): guard route logging types

fe51324

fix(containers): guarantee structured delete errors

4c0801e

feat(containers): harden docker runner connectivity

7e48d8e

test(platform-server): add docker fullstack flow

7442ff9

fix(platform-server): log structured delete failures

21b9445

fix(platform-server): restore container delete di wiring

eb1943b

fix(platform-server): retain delete error stacks

11833a9

fix(terminal): handle missing containers gracefully

2612015

test(graph): add runner defaults to fs persistence

a329e1d

casey-brooks dismissed vitramir’s stale review via a329e1d February 16, 2026 21:59

casey-brooks force-pushed the noa/issue-1272 branch from d1fe4ce to a329e1d Compare February 16, 2026 21:59

noa-lucent approved these changes Feb 16, 2026

View reviewed changes

vitalii-2pr approved these changes Feb 16, 2026

View reviewed changes

vitramir approved these changes Feb 16, 2026

View reviewed changes

vitramir added this pull request to the merge queue Feb 16, 2026

Merged via the queue into main with commit c53bbed Feb 16, 2026
7 checks passed

Conversation

casey-brooks commented Jan 29, 2026

Summary

Testing

Uh oh!

casey-brooks commented Jan 29, 2026

Test & Lint

Uh oh!

noa-lucent left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

casey-brooks commented Jan 29, 2026

Test & Lint

Uh oh!

noa-lucent left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

casey-brooks commented Jan 29, 2026

Uh oh!

casey-brooks commented Jan 29, 2026

Test & Lint Summary

Uh oh!

noa-lucent left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

casey-brooks commented Jan 29, 2026

Test & Lint Summary

Uh oh!

noa-lucent left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

casey-brooks commented Jan 29, 2026

Test, Lint & Build Summary

Uh oh!

casey-brooks commented Feb 11, 2026

Test & Lint Summary

Uh oh!

noa-lucent left a comment

Choose a reason for hiding this comment

Uh oh!

casey-brooks commented Feb 14, 2026

Test & Lint Summary

Uh oh!

noa-lucent left a comment

Choose a reason for hiding this comment

Uh oh!

casey-brooks commented Feb 14, 2026

Uh oh!

casey-brooks commented Feb 14, 2026

Uh oh!

noa-lucent left a comment

Choose a reason for hiding this comment

Uh oh!

casey-brooks commented Feb 14, 2026

Uh oh!

casey-brooks commented Feb 14, 2026

Uh oh!

noa-lucent left a comment

Choose a reason for hiding this comment

Uh oh!

casey-brooks commented Feb 14, 2026

Uh oh!

casey-brooks commented Feb 16, 2026

Test & Lint Summary

Uh oh!

noa-lucent left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers