Question: how can the values provided by process.platform and process.arch better reflect the underlying system

[lovell]

Thank you for all your work on WebContainers, they are really exciting technology. My thoughts below are about (hopefully) making the use of native modules slightly easier for the multiple parties involved.

The value currently provided by process.arch is always x64 and the value provided by process.platform is always linux.

According to lovell/sharp#3750 (comment) "WebContainers specifically aims to comply with Linux behavior", however from what can tell, WebContainers do not expose or provide any underlying Linux APIs.

It looks like at least one other person has been confused and has questioned this, e.g. #1095 states unexpected behaviour: "...reports Linux x64 rather than the expected WASM"

Many native dependencies are either using or are migrating to the use of optionalDependencies with os and cpu filters to tell package managers to download only the relevant binaries. In the case of WebContainers, the current values on the process global would imply the installation of a package and therefore native binary that defines the following filters:

  "os": [
    "linux"
  ],
  "cpu": [
    "x64"
  ]

The current approach to deal with this appears to be that package maintainers should include custom logic via @webcontainer/env or that end-users must add Wasm-specific dependencies, or that StackBlitz will automagically polyfill certain dependencies.

I feel like some consensus could be reached here around more appropriate values for process.platform and process.arch to allow Node.js compatible WebAssembly runtimes to work with existing package manager logic rather than against it, and that perhaps StackBlitz could be able to lead this.

Here's a very quick worked example (and it is only an example). Let's say process.platform is wasm and process.arch is 32. A maintainer of a native package might then be able to include the following in a published package.json file to make it explicit that this is the relevant package to install and use with a Wasm32 runtime.

  "os": [
    "wasm"
  ],
  "cpu": [
    "32"
  ]

The underlying aim for this question/discussion is an attempt to make life easier for StackBlitz (you can start to remove polyfills), make life easier for end-users (no manually-added Wasm-specific dependencies) and make life easier for package maintainers (the method already used for platform-specific binaries will "just work"). Thank you for reading.

[RReverser]

FWIW I'm personally +1 on having different values there, and brought something similar up when first encountered too.

Just one adjustment - the standard triplets for Wasm are wasm32-unknown-unknown or variations (wasm64-..., -wasi, ...-emscripten). So I think the arch would ideally be reported as wasm32 to match the standard target.

os should probably still report Linux as it's in principle most similar to Linux in terms of provided tools and this way would be compatible with more packages that don't care about CPU part (anything without native deps). Or it could report stackblitz or webcontainers or even unknown. I think it's a bit less important out of those 2, at least for native deps.

But I'm not a Stackblitz employee, just an opinionated Wasm developer :)

[Nemikolh]

Hey @lovell and @RReverser ! Thanks for starting this discussion! 😃

The original motivation for having process.platform set to linux and process.arch to x64 has been for better interoperability with existing packages in the node ecosystem. A typical example would be a package checking process.platform to be one of linux, win32 or darwin and throw an error if the value is different.

I feel like some consensus could be reached here around more appropriate values for process.platform and process.arch to allow Node.js compatible WebAssembly runtimes to work with existing package manager logic rather than against it, and that perhaps StackBlitz could be able to lead this.

I totally agree. I don't see us changing process.platform anytime soon because, as @RReverser pointed out, we try to mimic a Linux environment in terms of tooling (and if we don't it's a bug and we'll try to fix it) and we don't want to break packages that rely on that value.

I really like the idea of having process.arch set to wasm32 or wasm32-wasi (although wasi support comes through node so maybe the engines field is enough to detect wasi support). I'm not exactly sure of the impact it would have but I would guess there isn't much code out there that test that value unless it has native deps.

This is something we (StackBlitz) should investigate.

[RReverser]

I really like the idea of having process.arch set to wasm32 or wasm32-wasi (although wasi support comes through node so maybe the engines field is enough to detect wasi support).

Note that wasi in wasm32-wasi is OS part of the triplet as wasm32-wasi is basically shortening for wasm32-unknown-wasi which is how that target was called in the past (similarly to how there is wasm32-unknown-emscripten and wasm32-unknown-unknown). If it would be exposed, it would need to go into process.platform not process.arch, but as you said that wouldn't make much sense for common users.

[RReverser]

I'm not exactly sure of the impact it would have but I would guess there isn't much code out there that test that value unless it has native deps.

Yeah I'd expect that only native modules care about process.arch as it's important to know which instruction set the CPU supports. In that regard it's the most appropriate field for Wasm detection, as Wasm is about instruction set not OS APIs.

[lovell]

Brilliant, thanks both, I agree that setting process.arch to wasm32 and therefore relying upon the npm cpu filter value within the WebContainer runtime is the right thing to do here. 🎉