Skip to content

Stop standalone smoke CI from creating throwaway workspaces #1565

Description

@khaliqgant

Parent incident: #1562

Package Validation runs scripts/ci-standalone-smoke.sh without a workspace key, so every standalone lifecycle smoke provisions a Relaycast workspace that is abandoned after seconds. The test asserts local broker status/down/up/readiness behavior; fresh-workspace provisioning appears incidental, not an explicit acceptance property.

Scope:

  • reuse a dedicated CI workspace without committing its credential;
  • fail closed if the credential is unavailable so CI never silently resumes minting;
  • verify shared-workspace concurrency and per-run local-state isolation;
  • demonstrate a green Package Validation run and quantify avoided creations.

No workspace delete endpoint exists today (relaycast#336), so create-and-clean-up is not available. This issue does not authorize a merge.

Activity

  1. khaliqgant commented on Aug 18, 2026

    @khaliqgant
    MemberAuthor

    Checkpoint — implementation started (2026-08-18 Europe/Oslo).

    Environment:

    • host: Khaliqs-MacBook-Pro.local
    • GitHub auth account: khaliqgant
    • token scopes: admin:public_key, delete_repo, gist, read:org, repo, workflow
    • isolated worktree: /tmp/wt-relay-ci-ws-reuse-0818
    • branch: codex/ci-workspace-reuse-0818 (from current origin/main)

    Repository secret inventory currently has no dedicated CI workspace key. I am reading both Package Validation and publish smoke call sites plus the workspace-selection path before editing. I will verify concurrency/isolation rather than assume a single shared workspace is safe, and I will not merge.

  2. khaliqgant commented on Aug 18, 2026

    @khaliqgant
    MemberAuthor

    Implementation checkpoint — local proof complete.

    Decision: fresh workspace creation is incidental. The smoke asserts local status/down/up/readiness and never asserts first-run provisioning. The available safe choice is one dedicated secret-backed workspace; a committed pin would expose a credential, and create/delete is impossible because relaycast#336 has no delete endpoint.

    Implemented:

    • created exactly one long-lived workspace named relay-ci-standalone-smoke and stored its key as the Actions secret RELAY_CI_WORKSPACE_KEY (the value was never printed or put on argv);
    • both Package Validation and publish smoke call sites inject that secret;
    • the script fails closed when the key is absent, clears higher-precedence RELAY_WORKSPACES_JSON, and asserts environment selection + Workspace: joined while rejecting Workspace: created new workspace;
    • every run uses a unique broker name; temporary HOME/project directories already isolate connection/state files;
    • fork and Dependabot PRs skip the live-cloud job because GitHub withholds Actions secrets there, so they cannot silently fall back to minting.

    Concurrency/isolation evidence with real CLI + Rust broker artifacts against one existing shared workspace:

    • two simultaneous lifecycle smokes: exit 0 / exit 0;
    • one follow-up lifecycle after both teardowns: exit 0.
      No agents or channels are created by this smoke, and no inventory assertion depends on workspace emptiness.

    Validation:

    • focused contract: 4/4 passed;
    • full Vitest suite: 144 files, 2,068 passed, 23 skipped;
    • npm build: exit 0;
    • bash -n + shellcheck: exit 0;
    • Prettier: exit 0.

    Reduction: Package Validation had 24 workflow runs and 30 run attempts in the preceding 24 hours; at current volume this removes roughly 30 throwaway workspaces/day from this workflow (including reruns). CI branch proof is next. No merge.

  3. khaliqgant commented on Aug 18, 2026

    @khaliqgant
    MemberAuthor

    Done checkpoint: PR #1567 is open and Package Validation is green on final SHA 5cf2326 (run 32124891250). Standalone macOS Smoke job 95673166518 logged Workspace reuse verified: joined the dedicated CI workspace and Standalone smoke passed; no workspace-creation line appeared. No merge performed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions