Skip to content

[Bug]: SSH link mode fails as "ssh output was not valid UTF-8" when the remote's default shell is PowerShell (real error masked) #6088

Closed

Description

@ZhenyuXiao

Summary

Machine-link mode (ocx link, dashboard → "Remote connection") cannot add a Windows host as a Child when that host's OpenSSH DefaultShell is PowerShell. The remote command never runs: PowerShell rejects the POSIX-quoted sh -c … argv, and the resulting error text (written in the console code page) trips the runner's strict UTF-8 decode, which then masks the real failure as ssh output was not valid UTF-8.

Environment

opencodex 2.68.0, same version on both sides
Local side Windows; ssh is OpenSSH; ssh -G / ssh-keygen -l output verified valid UTF-8
Remote side Windows 10.0.26200 with an MSYS2 environment; ocx installed and callable
Critical detail the remote's OpenSSH DefaultShell is PowerShell (pwsh), not a POSIX shell
Remote code page CP936 (Chinese-locale Windows)

Reproduction

  1. Dashboard → Remote connection → connect an SSH host so it becomes a Child (the path that runs ocx connect --link --key-stdin --tunnel-port <port> --link-id <id> on the remote).
  2. The UI reports: Could not run ocx on the remote computer over SSH. Check SSH access and retry. Reason: ssh output was not valid UTF-8.
  3. crash.log records an unhandledRejection:
SshRunnerError: ssh output was not valid UTF-8
    at readOutput (<pkg>\src\link\ssh-runner.ts:123:15)
  cause: TypeError: The encoded data was not valid for encoding utf-8
  code: decode
  activity: GET /api/link/status

Verified NOT affected (byte-checked on the same remote)

Remote ocx --version, ocx link port, ocx link status --json, ocx link revoke --link-id <uuid> — all valid UTF-8. Locally, ssh -G <alias> and ssh-keygen -l -f <file> are also valid UTF-8.

Reason: PowerShell runs those as plain native commands, so nothing is parsed as a PowerShell expression.

Root cause

src/link/ssh-argv.ts builds a POSIX command and quotes it POSIX-style:

export const REMOTE_OCX_SCRIPT = 'PATH="$PATH:$HOME/.bun/bin:$HOME/.local/bin:/opt/homebrew/bin:/usr/local/bin"; exec ocx "$@"';
export function remoteOcxArgv(args: readonly string[]): string[] { return ["sh", "-c", REMOTE_OCX_SCRIPT, "ocx", ...args]; }

buildExecArgv single-quotes each element, so the remote receives:

'sh' '-c' 'PATH="$PATH:…"; exec ocx "$@"' 'ocx' 'connect' '--link' …

With PowerShell as the remote default shell, that string is parsed as a PowerShell expression and rejected:

ParserError:
Line |
   1 |  'sh' '-c' 'PATH="$PATH:$HOME/.bun/bin:…'
     |       ~~~~
     | Unexpected token ''-c'' in expression or statement.

PowerShell encodes that stderr with the console code page. The only non-ASCII bytes in the captured 382-byte stderr are A1 AD — CP936's … (U+2026), where UTF-8 would be E2 80 A6:

… 6f 70 74 2f 68 6f 6d 65 62 72 65 77 2f 20 a1 ad 1b 5b 30 6d 0d 0a …

Then the strict decode fails:

// src/link/ssh-runner.ts
return new TextDecoder("utf-8", { fatal: true }).decode(bytes);   // throws

→ SshRunnerError("decode", "ssh output was not valid UTF-8"), and the useful remote error is lost.

Expected

Either the remote call works, or the surfaced error describes the remote failure (for example "the remote SSH default shell is not POSIX"), never a decode failure.

Suggested fixes

  1. Don't fail the pipeline on decoding. Decode leniently (fatal: false) and, when invalid bytes appear, fall back to the platform code page (CP936/CP1252) or attach a hex / first-line excerpt to the error hint. A single ellipsis byte should not hide the real message.
  2. Probe the remote shell instead of assuming sh. A cheap capability probe, then adapt the invocation (bash, or a PowerShell-safe form) or fail with an explicit message.
  3. Give the decode path a hint too — sshFailureHint already exists but is only used for non-zero exits.

Workaround (remote side, administrator)

New-ItemProperty 'HKLM:\SOFTWARE\OpenSSH' -Name DefaultShell -PropertyType String `
  -Value 'C:\msys64\usr\bin\bash.exe' -Force
New-ItemProperty 'HKLM:\SOFTWARE\OpenSSH' -Name DefaultShellCommandOption -PropertyType String `
  -Value '-c' -Force
Restart-Service sshd

(Or use hub pairing instead of the SSH link mode.)

Activity

  1. ZhenyuXiao commented on Sep 27, 2026

    @ZhenyuXiao
    Author

    One clarification that matters for triage: PowerShell is not generally unable to run sh.

    PS> sh -c 'echo 1'
    1
    

    The trigger is specifically the single-quoted command name in the generated argv. buildExecArgv quotes every remote argv element, including the first one, so PowerShell receives:

    'sh' '-c' 'PATH="$PATH:…"; exec ocx "$@"' 'ocx' 'connect' '--link' …
    

    In PowerShell a statement that starts with a quoted string is parsed as an expression, not as a command invocation, so 'sh' '-c' … becomes "string literal followed by string literal" and fails:

    ParserError:
    Line |
       1 |  'sh' '-c' 'PATH="$PATH:$HOME/.bun/bin:…'
         |       ~~~~
         | Unexpected token ''-c'' in expression or statement.
    

    That is exactly the ParserError captured in the report — the caret sits under -c (4 columns = the length of '-c').

    Two consequences worth noting for a fix:

    • Quoting the command name is what makes the difference. A POSIX-shaped invocation with an unquoted command name (sh -c '…' ocx …) would also have run under PowerShell; only the fully-quoted form fails. So "quote only the arguments that need it" may be enough for the simple cases, if the maintainers prefer to keep a single code path.
    • The $PATH=… and $@ inside the script argument survive as-is, because they sit inside a single-quoted string and PowerShell does not interpolate single-quoted strings — so no extra escaping is needed there.

    This does not change the root cause: the strict TextDecoder("utf-8", { fatal: true }) still turns one CP936 ellipsis byte pair (A1 AD) in PowerShell's error output into a generic decode failure that hides all of the above.

  2. ZhenyuXiao commented on Sep 27, 2026

    @ZhenyuXiao
    Author

    Update: verified fix (applied in a real install and confirmed end-to-end)

    Emitting the command name bare instead of single-quoting it makes link mode work against a remote whose sshd DefaultShell is PowerShell, with no change on the remote.

    Why it works:

    • PowerShell parses a bare first word as a command name and the following token as an argument. A statement that starts with a quoted string is parsed as an expression, which is exactly what produced the Unexpected token ''-c'' ParserError in the report.
    • POSIX shells are unaffected: a bare sh and a quoted 'sh' are the same word, so Linux/macOS remotes keep behaving identically.

    Patch (src/link/ssh-argv.ts, quoteRemote):

     export function quoteRemote(argv: readonly string[]): string {
    -  return argv.map(arg => {
    +  return argv.map((arg, index) => {
         if (arg.includes("\0")) throw new LinkSshArgumentError("remote argument contains NUL");
    +    // The command name stays bare: PowerShell parses a leading quoted string as an
    +    // expression (`'sh' '-c' …` → "Unexpected token ''-c''"), while a bare word is a
    +    // command in both PowerShell and POSIX shells. Restricted to a conservative shape.
    +    if (index === 0) {
    +      if (!/^[A-Za-z0-9._-]+$/.test(arg)) {
    +        throw new LinkSshArgumentError(`remote command name is not a safe bare token: ${JSON.stringify(arg)}`);
    +      }
    +      return arg;
    +    }
         return `'${arg.replaceAll("'", `'"'"'`)}'`;
       }).join(" ");
     }

    Resulting wire form (printed by calling the patched function through the bundled Bun runtime):

    sh '-c' 'PATH="$PATH:$HOME/.bun/bin:$HOME/.local/bin:/opt/homebrew/bin:/usr/local/bin"; exec ocx "$@"' 'ocx' 'link' 'port'
    

    Verification performed:

    1. Reproduced the failure locally by running the current form through pwsh ('sh' '-c' … → the same ParserError, caret under -c), then confirmed the bare form parses fine (sh '-c' …).
    2. Confirmed PowerShell's native argument passing keeps the single-quoted elements as separate, quote-stripped arguments and does not interpolate the script argument — so $PATH and $@ still expand in the remote sh, and --key-stdin still receives stdin.
    3. Applied the patched quoteRemote in a real install, restarted the service, and completed the link: a Windows host whose OpenSSH DefaultShell is PowerShell now joins successfully as a Child. ocx --version, ocx link port, ocx link status --json, ocx link revoke and the tunnel all keep working.

    Notes for a maintainer-side fix:

    • This keeps a single code path — no remote-shell detection is needed for this failure mode. If you prefer detection over bare-quoting, the observable check is simply whether the command name survives as a command on the remote (sh -c true-style probe).
    • The strict TextDecoder("utf-8", { fatal: true }) still hides this whole class of failure whenever a non-UTF-8 byte appears (here: a single CP936 …). Decoding leniently, or attaching a hex / first-line excerpt to the error, would have surfaced the PowerShell error immediately instead of "ssh output was not valid UTF-8".
  3. added 4 commits that reference this issue on Sep 27, 2026
    039dae3
    bf9de81
    6a6abe3
    9f50bdd
  4. lidge-jun commented on Sep 27, 2026

    @lidge-jun
    Owner

    Fixed on dev by #6113 (merge 2275680ab3, commit 9f50bddfe8). Thanks @ZhenyuXiao for the precise diagnosis and the end-to-end check.

    • The remote command now emits sh bare. Only sh is accepted in command position, and every argument stays single-quoted, so a PowerShell OpenSSH DefaultShell dispatches it as a command.
    • The Child's SSH runner caps stderr bytes, then decodes with replacement, so a CP936 error line reaches the redacted, length-bounded hint instead of ssh output was not valid UTF-8. Structured stdout stays strict UTF-8.

    Verification: a Windows-only test runs powershell.exe's parser on the generated command and asserts a single sh command invocation. It ran and passed on the Windows 6/9 shard of Cross-platform CI run 36331108394. POSIX tests execute the generated string through /bin/sh and check that argument bytes arrive intact. Runner tests cover invalid-UTF-8 stderr, strict stdout and the byte cap.

    Known limit: your end-to-end check used pwsh 7. Windows PowerShell 5.1's legacy native-argument passing does not escape embedded double quotes, and the -c script contains some, so that combination is untested. If you hit it, please open a new issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions