Regarding ESBUILD_BINARY_PATH and stuff

[HanabishiRecca]

Fix esbuild installation on Arch Linux (#2785, #2812, #2865)

Someone made an unofficial esbuild package for Linux that adds the ESBUILD_BINARY_PATH=/usr/bin/esbuild environment variable to the user's default environment. This breaks all npm installations of esbuild for users with this unofficial Linux package installed, which has affected many people.

It was me, didn't realized that this variable overrides it so hard.
I will explain why I did that a little bit later, but first, I am sorry for the inconvenience. The package is actually used as a dependency almost exclusively by other packages also maintained by me (webcord, kwin-bismuth), so I fairly surprized that it affected many people.
Little grunt: they are probably don't use AUR properly and blindly run AUR helpers, not even watching what they are installing, which is strongly discouraged by Arch Linux devs.

Note: The ESBUILD_BINARY_PATH variable is an undocumented way to override the location of esbuild's binary when esbuild's npm package is installed, which is necessary to substitute your own locally-built esbuild binary when debugging esbuild's npm package. It's only meant for very custom situations and should absolutely not be forced on others by default, especially without their knowledge. I may remove the code in esbuild's installer that reads ESBUILD_BINARY_PATH in the future to prevent these kinds of issues. It will unfortunately make debugging esbuild harder. If ESBUILD_BINARY_PATH is ever removed, it will be done in a "breaking change" release.

Yeah, I should've think about it earlier.
Anyway, as ESBUILD_BINARY_PATH is not really an option, can we get an alternative? This can conviniently be used with the fact that binary npm packages are optional dependencies.

Here is explanation why: I don't trust binaries from npm.
It is not about esbuild itself, npm is just exploited by hackers constantly (hacking dev's accounts etc.) and have zero checks.
Also they are built not according to Arch Linux guidelines.
So I'd prefer to use the binary built from the source. And maybe some other people too.

[evanw]

Thanks for commenting on this. I appreciate it. Caveat: I'm very unfamiliar with Arch Linux. I have never used it myself and all I have heard about this issue has come from reports from other users.

I don't trust binaries from npm. ... So I'd prefer to use the binary built from the source.

That's fine! Instructions for doing that are here: https://esbuild.github.io/getting-started/#build-from-source. But I don't understand why you would need to inject ESBUILD_BINARY_PATH into the user's default environment to do that. The esbuild binary executable is completely self-contained and running it doesn't involve npm at all (it also doesn't even involve node). You can use ./esbuild to run esbuild after building it from source. The only reason you'd need ESBUILD_BINARY_PATH is if you are using esbuild's JavaScript API from npm and trying to override which binary executable it uses. But if you're doing that then you'd be trusting content on npm, so I'm assuming that's not what you're trying to do.

[HanabishiRecca]

But I don't understand why you would need to inject ESBUILD_BINARY_PATH into the user's default environment to do that.

Well, I just underestimated the effect of this action. And never noticed that npm script actually expects the binary version to be exactly the same.

The esbuild binary executable is completely self-contained and running it doesn't involve npm at all (it also doesn't even involve node).

This is true and totally works in AUR build scripts. But I'm addressing a different scenario here.

The only reason you'd need ESBUILD_BINARY_PATH is if you are using esbuild's JavaScript API from npm and trying to override which binary executable it uses. But if you're doing that then you'd be trusting content on npm, so I'm assuming that's not what you're trying to do.

I'm talking about that actually. Regarless of invocation method, via CLI or JS API, in NodeJS projects you link esbuild as a dependency in package.json. Especially if such project are shared/public. And yes, I know that npm is intentionally designed this way to pull every dependency independently for every project. So I understand if you will be against this idea.

But I think this is not too hard to implement. E.g. when the main script detects that the binary package is not installed, try to fallback to esbuild binary in current $PATH.
Again, there can be some problems with version compatibility. But I think people intentionally omitting the binary npm package are aware of their actions.

[evanw]

I'm still confused. If you are pulling anything from npm then you're trusting the content on npm. It doesn't matter if it's encoded as ASCII or binary. You wrote this in your original post:

Here is explanation why: I don't trust binaries from npm.
It is not about esbuild itself, npm is just exploited by hackers constantly (hacking dev's accounts etc.) and have zero checks.

If that's your threat model, then nothing you install from npm is safe. If my npm account credentials were to be hacked somehow, that could be used to upload malicious JavaScript code in exactly the same way as it could be used to upload a malicious binary executable.

If your approach is to trust JavaScript code from npm but to not trust anything encoded as binary from npm, then here's something you could consider doing that is similar to the workflow you want (manually building and injecting esbuild) but without having to use an unusual installation method to install esbuild: you can verify the contents of the esbuild binary executable published to npm before installing the package.

I have done a lot of work to ensure that esbuild's build process generates reproducible builds. There is automation that runs after every esbuild release which verifies that the binary executables that were published to npm are bitwise identical with ones built in an isolated CI environment (the source code is here). You could consider doing something similar: build esbuild locally, then validate that the package archive on npm has the same build. If that passes then it should be fine to just install that specific esbuild version with npm normally without messing with the installer.

But I think this is not too hard to implement. E.g. when the main script detects that the binary package is not installed, try to fallback to esbuild binary in current $PATH.

I'm not planning on doing this. The only supported workflow for installing esbuild with npm is the one described in esbuild's documentation.

[HanabishiRecca]

Ok, I got your point. Thanks.