You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Reproducible build issues: Does not use sh on path #1100
node-gyp's generated make file is using /bin/sh rather than the version of sh on the PATH. This makes it impossible to be certain builds are reproducible.
I could be wrong, but doesn't this mean that builds are more reproducible? On any machine, the version of sh in the path is more likely to change that the one at /bin/sh right?
Could you be more specific about the benefits of using #!/usr/bin/env sh over #!/bin/sh?
First up, the benefits of using #!/usr/bin/env sh over #!/bin/sh is a slightly different issue.
This is about the shell used in the make file. And you're right - the shell in the PATH may not be the one at /bin/sh. That's the point - you're respecting the user's wishes. In this case, the user doing the compiler. They known their machine; upstream does not. In fact, it's even more subtle than that, because there are actually two shells in play; the one you want to compile things against (the build machine) and the one you want to use when running your final output (eg shell scripts used as init scripts). This subtle distinction is one that many builds get wrong - including the git source - because most projects only think about cross-compiling some time after developing code - whereas thinking that actually all builds should be cross builds, even for the native machine using a system compiler, is much more likely to create truly reproducible build solutions.
Using /bin/sh means you've just forced a dependency on whatever shell, and whatever version of a shell, is at a particular location that will be outside of any build toolchain. On Libertine Linux, a not-quite-there-yet project of mine, we build a complete toolchain, including exact versions of all dependencies, right down to core utils, the shell, and even make and autohell. So now the build is a function of the system toolchain, not the build toolchain.
Let me try to persuade you further. C's a language, and it's quite natural to provide a different compiler, both for build-time (eg node's mksnapshot, linux's 'scripts') and cross-compile time. So is C++. We control which version of Python we use. And we ruthlessly try to version those. Likewise, shell is a language. It's a bit naff in places, it's a bit old, but it's a language. It even has a standard.
Back to /usr/bin/env. It's not perfect - as one is still using a system component - but we're now down to what is probably a lowest common denominator. Ordinarily I can work around the sorts of issues we're talking about, but because the make file is generated (and is deeply nested in a knitting ball of yarn), I can't.
I don't think gyp or node-gyp explicitly set the shell anywhere. If I had to venture a guess, it's that you need to invoke node-gyp with env SHELL=/path/to/sh node-gyp rebuild to make make(1) pick up the right shell.
Activity
gibfahn commentedon Jan 31, 2017
I could be wrong, but doesn't this mean that builds are more reproducible? On any machine, the version of
shin the path is more likely to change that the one at/bin/shright?Could you be more specific about the benefits of using
#!/usr/bin/env shover#!/bin/sh?raphaelcohn commentedon Jan 31, 2017
First up, the benefits of using
#!/usr/bin/env shover#!/bin/shis a slightly different issue.This is about the shell used in the make file. And you're right - the shell in the PATH may not be the one at
/bin/sh. That's the point - you're respecting the user's wishes. In this case, the user doing the compiler. They known their machine; upstream does not. In fact, it's even more subtle than that, because there are actually two shells in play; the one you want to compile things against (the build machine) and the one you want to use when running your final output (eg shell scripts used as init scripts). This subtle distinction is one that many builds get wrong - including the git source - because most projects only think about cross-compiling some time after developing code - whereas thinking that actually all builds should be cross builds, even for the native machine using a system compiler, is much more likely to create truly reproducible build solutions.Using
/bin/shmeans you've just forced a dependency on whatever shell, and whatever version of a shell, is at a particular location that will be outside of any build toolchain. On Libertine Linux, a not-quite-there-yet project of mine, we build a complete toolchain, including exact versions of all dependencies, right down to core utils, the shell, and even make and autohell. So now the build is a function of the system toolchain, not the build toolchain.Let me try to persuade you further. C's a language, and it's quite natural to provide a different compiler, both for build-time (eg node's mksnapshot, linux's 'scripts') and cross-compile time. So is C++. We control which version of Python we use. And we ruthlessly try to version those. Likewise, shell is a language. It's a bit naff in places, it's a bit old, but it's a language. It even has a standard.
Back to
/usr/bin/env. It's not perfect - as one is still using a system component - but we're now down to what is probably a lowest common denominator. Ordinarily I can work around the sorts of issues we're talking about, but because the make file is generated (and is deeply nested in a knitting ball of yarn), I can't.raphaelcohn commentedon Jan 31, 2017
PS, That's not Hursley, in the background, by any chance?
bnoordhuis commentedon Feb 1, 2017
I don't think gyp or node-gyp explicitly set the shell anywhere. If I had to venture a guess, it's that you need to invoke node-gyp with
env SHELL=/path/to/sh node-gyp rebuildto make make(1) pick up the right shell.gibfahn commentedon Feb 1, 2017
@raphaelcohn It's actually King John's House, but close!
bnoordhuis commentedon May 12, 2017
Closing, answered.