Evaluate use of pkg to make single-binary server

[sr229]

pkg is a application that compiles the project into a single executable, which we currently do, but pkg seems to be flexible as you can provide runtime version and such with the sacrifice of requiring some of the requires to be __dirname'd

[Zeit pkg repo]

[kylecarbs]

We've attempted to use pkg. Using native modules is a large problem we've found, along with differentiating filesystems between inside and outside the binary.

Using nexe has been alright, but our usage requires a lot of jankiness. Optimally, we'll eventually create our own project to compile the binary.

We're able to provide the runtime version with nexe.

[sr229]

have you tried to open this to Zeit's devs about the problem with native modules @kylecarbs?

[kylecarbs]

Seems its an open feature request: vercel/pkg#329

[walking-octopus]

The feature request has been fulfilled. Can anyone investigate if it's possible now?

[jsjoeio]

I wasn't around when this was being evaluated. @walking-octopus what benefits do you see in us using pkg?

[walking-octopus]

I feel it could be smaller or more convenient than an existing archive. If this was found important enough to hack around the limitations the last time, why not try again?

[jsjoeio]

Interesting! Smaller is definitely appealing. I don't know the full history around some of the hacks or limitations from the early years of the project.

@code-asher do you have any context? Is this worth considering?

[code-asher]

To go back a bit first, I think the main draws were that a single binary is easy to run (just download one file and execute it), easy to install (just copy one file), and easy to clean up (just delete one file). Unfortunately the single binary ended up not being straightforward because there are extensions that make file reads to some of the files distributed with code-server which fail because those files are not actually individually on the filesystem; they are in the binary. Another example is the git extension; it was attempting to spawn a shell script distributed with code-server which does not work for the same reason. There may be other broken scenarios we did not discover. To deal with this I believe there were three ideas: 1. Binary extracts parts of itself then we patch reads/spawns/etc to read from the extracted location (we did this for a while). 2. Patch reads/spawns/etc to read directly from the binary (I think we tried this but ran into issues). 3. Binary extracts itself entirely and we run from the extracted location. In practice the first two strategies sort of worked but they were brittle (basically a game of whack-a-mole) and the patching only works for JavaScript code. Self-extraction deviates from why we wanted a single binary in the first place since it is not especially different from an archive except less flexible since you cannot choose the extraction location. If there was a general and robust way of solving these issues then I feel like going back to the single binary would be a good option.