Error installing node.js: The Node.js binary could not be verified.

[jarrodek]

When I run the build I receive the following error:

Sending build context to Docker daemon     75MB
Step 1/15 : FROM launcher.gcr.io/google/nodejs
 ---> d8132d024114
Step 2/15 : RUN install_node v12.16.2
 ---> Running in 80fefff9837f
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 22.0M  100 22.0M    0     0  8969k      0  0:00:02  0:00:02 --:--:-- 8969k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  4029  100  4029    0     0  11964      0 --:--:-- --:--:-- --:--:-- 11991
gpg: Signature made Wed Apr 8 19:22:22 2020 UTC using RSA key ID 5599653C gpg: Can't check signature: public key not found
The Node.js binary could not be verified.
This means it may not be an officially released Node.js binary
or may have been tampered with.

Aborting the installation.

The installation can be forced using the --ignore-verification-failure
flag. However, it is strongly recommended that you install a version
of Node.js that can be verified.

Node installation failed: /opt/gcp/runtime/bootstrap_node exited with a non-zero exit code: 1
The command '/bin/sh -c install_node v12.16.2' returned a non-zero code: 1

I have the following configuration:

# Dockerfile extending the generic Node image with application files for a
# single application.
FROM launcher.gcr.io/google/nodejs

# Check to see if the the version included in the base runtime satisfies
# '>=8.12.0', if not then do an npm install of the latest available
# version that satisfies it.
# RUN /usr/local/bin/install_node '>=12.13.0'
RUN install_node v12.16.2

When I do what is suggested it installs the node:

RUN install_node v12.16.2 --ignore-verification-failure

but clearly there is a problem somewhere.

[plegner]

Any update on this? I'm seeing the same error with the latest Google Cloud SDK (309.0.0)

[aiimtechdev]

seems the issue with node 14.10.0. you can try node version 14.9.0 then it works!

[prowwid]

Hello @plegner

in our case, it was the same as for @aiimtechdev.

We had broken deployment to App Engine because it installed Node 14.10.0 as we had "node": "14.x" in the package.json. Seems like the latest release has some issues.

I fixed deploy by changing the engine to a fixed version:

"engines": {
    "node": "14.9"
}

If you are using 12.x - try some of the previous version that worked.

Obviously it does not solve the overall issue with the latest release but at least get things working.

Hope it will help!

[plegner]

Thanks, @prowwid! We were using ">=12.0.0" and had lots of successfully released with that until this week. Fixing the version also worked for us 😄

[MatthieuLemoine]

This error also occurs with the latest release of Node 10 10.23.0 which was released by @richardlau. His signing key is missing in the KEYS file https://github.com/GoogleCloudPlatform/nodejs-docker/blob/master/runtime-image/contents/KEYS, it was added back in july in the Node repo https://github.com/nodejs/node/pull/34397/files

[richardlau]

In addition to my signing key, whoever maintains this repository may also want to add the keys for @ruyadorno (nodejs/node#34628) and @danielleadams (nodejs/node#35545) who have also been recently onboarded as releasers and may sign future releases: https://github.com/nodejs/node/blob/master/README.md#release-keys

[donmccasland]

Taking a look at this.

[donmccasland]

Okay, fixed in staging:
onmccasland-macbookpro2:nodejs-docker donmccasland$ docker run -it gcr.io/google-appengine/nodejs@sha256:058266594604d7967a371094f0631c32f93908c37e20c3354d22f1b3a63c9a17 /bin/bash
root@d9eee9e2f7c8:/app# install_node v10
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 20.2M 100 20.2M 0 0 10.6M 0 0:00:01 0:00:01 --:--:-- 10.6M
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 4232 100 4232 0 0 14452 0 --:--:-- --:--:-- --:--:-- 14493
node-v10.23.0-linux-x64.tar.gz: OK
Installed Node.js v10.23.0

Starting the release process..

[donmccasland]

Fixed.

[elyobo]

Occurring in GAE flex again with 14.x set.

Step #1: INFO[0047] RUN /usr/local/bin/install_node '14.x'       
Step #1: INFO[0047] cmd: /bin/sh                                 
Step #1: INFO[0047] args: [-c /usr/local/bin/install_node '14.x'] 
Step #1:   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
Step #1:                                  Dload  Upload   Total   Spent    Left  Speed
100 32.3M  100 32.3M    0     0  43.1M      0 --:--:-- --:--:-- --:--:-- 43.2M
Step #1:   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
Step #1:                                  Dload  Upload   Total   Spent    Left  Speed
100  3835  100  3835    0     0  24299      0 --:--:-- --:--:-- --:--:-- 24426
Step #1: gpg: Signature made Tue May 11 23:13:29 2021 UTC using RSA key ID 643B6201 gpg: Can't check signature: public key not found
Step #1: The Node.js binary could not be verified.
Step #1: This means it may not be an officially released Node.js binary
Step #1: or may have been tampered with.
Step #1: 
Step #1: Aborting the installation.
Step #1: 
Step #1: The installation can be forced using the --ignore-verification-failure
Step #1: flag. However, it is strongly recommended that you install a version
Step #1: of Node.js that can be verified.
Step #1: 
Step #1: Node installation failed: /opt/gcp/runtime/bootstrap_node exited with a non-zero exit code: 1
Step #1: error building image: error building stage: waiting for process to exit: exit status 1
Finished Step #1
ERROR
ERROR: build step 1 "gcr.io/kaniko-project/executor@sha256:f87c11770a4d3ed33436508d206c584812cd656e6ed08eda1cff5c1ee44f5870" failed: step exited with non-zero status: 1

Edit: setting the package engine version to the last successfully deployed version (14.6.1) instead of 14.x works around the issue for now.

[alexnault]

I confirm: Node 14.17.0 is having that same issue. Note: Node 14.16.0 works fine.

[cewald]

I've the same problem using Google App Engine Flex & Standard: if I downgrade to 14.16.x using "engines": { "node": "14.16.x" } in my package.json I'd work but if I deploy the package in another environment using the standard environment it would again result in an error-message because of the wrong version:

The 'package.json' configuration file must specify a supported nodejs14 version that is compatible with the runtime specified in the deployment. Please pin your application to a compatible major version of the runtime via the 'engines.node' property. Supported 'engines.node' values for the runtime 'nodejs14' are: [14.x.x, 14.X.X, 14.x, 14.X, 14.*.*, 14.*, ~14, ~14.0, ~14.0.0, 14, ^14, ^14.0, ^14.0.0, >=14, >=14.0, >=14.0.0, >14.0, >14.0.0, >=1, >=1.0, >=1.0.0, >1, >1.0, >1.0.0, >=2, >=2.0, >=2.0.0, >2, >2.0, >2.0.0, >=3, >=3.0, >=3.0.0, >3, >3.0, >3.0.0, >=4, >=4.0, >=4.0.0, >4, >4.0, >4.0.0, >=5, >=5.0, >=5.0.0, >5, >5.0, >5.0.0, >=6, >=6.0, >=6.0.0, >6, >6.0, >6.0.0, >=7, >=7.0, >=7.0.0, >7, >7.0, >7.0.0, >=8, >=8.0, >=8.0.0, >8, >8.0, >8.0.0, >=9, >=9.0, >=9.0.0, >9, >9.0, >9.0.0, >=10, >=10.0, >=10.0.0, >10, >10.0, >10.0.0, >=11, >=11.0, >=11.0.0, >11, >11.0, >11.0.0, >=12, >=12.0, >=12.0.0, >12, >12.0, >12.0.0, >=13, >=13.0, >=13.0.0, >13, >13.0, >13.0.0].

So this should be updated in both environments.

[pete-beresford-bzy]

Seems even v14.16 now has key issues (expired):

gpg: Signature made 02/23/21 13:00:40 GMT Standard Time using RSA key ID A1AB005C
gpg: Good signature from "Beth Griggs <bgriggs@redhat.com>" [expired]
gpg:                 aka "Beth Griggs <Bethany.Griggs@uk.ibm.com>" [expired]
gpg: Note: This key has expired!
Primary key fingerprint: 4ED7 78F5 39E3 634C 779C  87C6 D706 2848 A1AB 005C

[BethGriggs]

@pete-beresford-bzy, Hmm...I extended my key back in March but it seems it hasn't properly propagated/synced with the servers - I'll try resyncing/resubmitting.

[plegner]

Why can't AppEngine just pick the last supported version of Node if a range like 14.x is specified in package.json?

Even if we manually specify a supported version like 14.16, the engines key is automatically updated by scripts like Renovate, and it is quite annoying if that leads to unpredictable deployment failures…

[danielleadams]

Are there still issues with 14.7.0? Like @BethGriggs my key was synced a few months ago (Back in January, it looks like nodejs/node#36793), but I've just recently resubmitted/resynched it again to confirm.

[brunogabuzomeu]

Same issue with version 16.11.0

[chiefy]

Same as what @brunogabuzomeu said, we tempoarily solved it by using =16.10.0 which still works.

[guavadevelopment]

FYI 14.18.1 was released 12-Oct-2021 and is having this issue, 14.18.0 can be set manually to deploy

[BethGriggs]

The issue is possibly that the KEYS file in this repository has not been updated to contain @danielleadams's new key (updated in January, nodejs/node#36793).

The Node.js project has recently started to maintain https://github.com/nodejs/release-keys/tree/main/keys as a source of truth for our release keys, which might be a useful resource for keeping this repository in sync with Node.js key updates.

[vvision]

Same issue here, GAE Flex with "engines": { "node": "14.x"},. Was working on monday (2021-10-11).
I guess KEYS file need to be updated again.
I will try downgrading to unblock deployment, but I'm definitvely not a fan of this solution.
Edit: Downgrading to "engines": {"node": "14.18.0"}, fixed it.

Could KEY file updates be automated to prevent such issue to periodically happen ?

[chiefy]

@vvision see @BethGriggs comment - yes it could be automated but I just tried forking + running the tests (they are very broken) and this repo appears unmaintained 👎

@donmccasland any help here please?

[wokkaflokka]

Setting 14.18.0 works as a short term solution -- unfortunately, if you are deploying, say, cloud functions from the same project, "engines": {"node": "14.18.0"} is invalid according to a separate Google technology platform.

Error: package.json in functions directory has an engines field which is unsupported. Valid choices are: {"node": 10|12|14|16}

[takayukioda]

I had same issue for node12.22.7 today.

I have specified node: ^12.22.1 in package.json and failed.
Next I ran with node: 12.22.7 and it failed.
Finally I ran with node: 12.22.6 and it worked.

according to the nodejs releases, 12.22.7 has been released at 2021/10/12, so I guess that's where it got broken.

[BethGriggs]

according to the nodejs releases, 12.22.7 has been released at 2021/10/12, so I guess that's where it got broken.

Any releases signed by @danielleadams's key will not verify until the keys are updated in this repository, which include the latest security releases released on 2021-10-12 (v12.22.7, v14.18.1, and v16.11.1).

[louisgv]

I checked the key list, it seems Danielle's key is in:
https://github.com/GoogleCloudPlatform/nodejs-docker/blob/master/runtime-image/bin/gen_keys.sh#L23

https://github.com/nodejs/node#release-keys

[OPNutrition]

@jinglundong here is my config

package.json:

  "engines": {
    "node": "14.x",
    "npm": ">=6.0.0"
  }

app.yaml:

runtime: nodejs

env: flex

manual_scaling:
  instances: 1

resources:
  cpu: 1
  memory_gb: 0.5
  disk_size_gb: 10

error:

Step #1: [1/5] Validating package.json...
Step #1: error onpoint-nutrition@0.1.0: The engine "node" is incompatible with this module. Expected version "14.x". Got "12.19.0"

[jinglundong]

I couldn't reproduce the error with

  "engines": {
    "node": "14.x",
    "npm": ">=6.0.0"
  }

When I deploy, the version picked were: Installed Node.js v14.19.1 and Installed "npm@8.6.0" with binaries. Could you try this combination. Another idea is to use node >= 14.x.

[ishak-lm]

@jinglundong I have almost the same configuration:

  "engines": {
    "node": "14.x",
    "npm": ">=6.0.0"
  }

runtime: nodejs

env: flex

# Resources configuration (for each server)
resources:
  cpu: 1
  memory_gb: 2
  disk_size_gb: 10

# Configuration of automatic scaling
# block max_num_instance (3)
automatic_scaling:
  min_num_instances: 1
  max_num_instances: 1
  cool_down_period_sec: 180
  cpu_utilization:
    target_utilization: 0.6
# [END app_yaml]

Step #1: yarn install v1.22.18
Step #1: [1/5] Validating package.json...
Step #1: error *************@1.5.30: The engine "node" is incompatible with this module. Expected version "14.x". Got "12.19.0"
Step #1: error Found incompatible module.
Step #1: info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
Step #1: error building image: error building stage: waiting for process to exit: exit status 1
Finished Step #1

gcloud app deploy app-dev.yaml --project=*******

[jinglundong]

@ishak-lm I think this is a different issue. "The engine "node" is incompatible with this module. Expected version "14.x". Got "12.19.0" is not complaining about the pgp key. Could you file another issue and try the specific version that I suggested? node: 14.19.1 and npm: 8.6.0?

I'm going to close this issue for now. If anyone's still seeing Can't check signature: public key not found The Node.js binary could not be verified. please reopen.

[chiefy]

Just to note, this is going to have to be re-opened the next time a new GPG key is added, the problem wasn't solved.

[jinglundong]

That is correct. I'm listing a few ideas below following my preferred order, but I haven't had a chance to figure out the details. Please advice if anyone has better solutions.

1. We have a plan to simplify how we build Flex applications. I believe it's going to solve this problem, or at least be able to patch it much faster. This is the actual fix, but it will likely take around two quarters to develop.
2. If new keys are added and announced before they are being used (which would cause the GPG check failure), we are happy to monitor that list of keys more closely. This is a short term solution.
3. We may have a way to detach from latest, and pin to a relatively new node version. Not always using the latest version of node may cause problems for some users too.

[jinglundong]

A heads-up, nodejs 18.0.0 is out today. Our internal test seems to be telling me that we don't support it automatically. A mitigation could be limit the nodejs version to < 18.0.0.

I opened #232 for this issue.

[justisGipson]

Experiencing this error today with Node 18.5.0.

[jinglundong]

Sorry to hear that. We plan to support Node 18 before year end 2022. The solution was to modernize how we support and build Flex node applications. Our goal is to accelerate the new version support in the future. We will keep this issue updated, especially when Node 18 is ready for private/public preview.

[abentpole]

Is there an explicit list of which specific minor versions of Node aren't going to trigger this problem?

[joshualyon]

I experienced this issue today. I couldn't see which version the system was attempting to install anywhere in the logs.

In the failing case, I saw:

Step #2: INFO[0105] RUN /usr/local/bin/install_node '14.x'
Step #2: INFO[0105] cmd: /bin/sh
Step #2: INFO[0105] args: [-c /usr/local/bin/install_node '14.x']
Step #2:   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
Step #2:                                  Dload  Upload   Total   Spent    Left  Speed
100 32.9M  100 32.9M    0     0  77.6M      0 --:--:-- --:--:-- --:--:-- 77.7M
Step #2:   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
Step #2:                                  Dload  Upload   Total   Spent    Left  Speed
100  3835  100  3835    0     0  35435      0 --:--:-- --:--:-- --:--:-- 35841
Step #2: gpg: Signature made Fri Sep 23 16:05:56 2022 UTC using RSA key ID E57C7D57 gpg: Can't check signature: public key not found
Step #2: The Node.js binary could not be verified.
Step #2: This means it may not be an officially released Node.js binary
Step #2: or may have been tampered with.

Looking at the signature date of Sep 23rd and our restriction to Node 14.x, we can infer that it's 14.20.1 as it was released on that day, but it would have been helpful if the logs showed which version was being attempted. Better yet, it would be helpful if the system didn't try to use unsupported versions at all.

For reference, in a successful installation (package.json engine version restricted to 14.19), the immediate next step shows the full installed version number:

Step #2: INFO[0102] RUN /usr/local/bin/install_node '14.19'
Step #2: INFO[0102] cmd: /bin/sh
Step #2: INFO[0102] args: [-c /usr/local/bin/install_node '14.19']
Step #2:   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
Step #2:                                  Dload  Upload   Total   Spent    Left  Speed
100 32.9M  100 32.9M    0     0  59.2M      0 --:--:-- --:--:-- --:--:-- 59.1M
Step #2:   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
Step #2:                                  Dload  Upload   Total   Spent    Left  Speed
100  3835  100  3835    0     0  19242      0 --:--:-- --:--:-- --:--:-- 19271
Step #2: node-v14.19.3-linux-x64.tar.gz: OK
Step #2: Installed Node.js v14.19.3

[wokkaflokka]

This started failing again today for node 16.x. Observed that node published a release circa 11AM today.

Step #2: gpg: Signature made Wed Oct 12 14:48:44 2022 UTC using RSA key ID 94EC4689 gpg: Can't check signature: public key not found
Step #2: The Node.js binary could not be verified.
Step #2: This means it may not be an officially released Node.js binary
Step #2: or may have been tampered with.
Step #2: 
Step #2: Aborting the installation.
Step #2: 
Step #2: The installation can be forced using the --ignore-verification-failure
Step #2: flag. However, it is strongly recommended that you install a version
Step #2: of Node.js that can be verified.
Step #2: 
Step #2: Node installation failed: /opt/gcp/runtime/bootstrap_node exited with a non-zero exit code: 1
Step #2: error building image: error building stage: waiting for process to exit: exit status 1
Finished Step #2

What can we do to be insulated from this error without having to pin versions in our application itself?

[jinglundong]

tl;dr: I can take a look at the new key and probably update it. To mitigate the problem, please pin it to the previous patch version of Node.js 16.17.x or Node.js 16.17.1. https://nodejs.org/en/download/releases/

I haven't looked into the exact problem, but I'd suspect that it's broken because there's a new pgp key in Node.js 16.18. This means our builder (or nodejs itself) is not entirely backward compatible between minor version upgrades.

This is a known issue where we don't really have a short term automation on our side. The best option in to pin to a patch version of nodejs version. Because new pgp keys are getting added along with new patch version of nodejs. We don't have a way to detect that, or block anyone from picking it up immediately when they have something like 16.x.

We are still working on a project to improve our build system. Once that's complete, we should be able to control this behavior - we will qualify the new minor or patch version before the builder using it.

[wokkaflokka]

@jinglundong

Thanks for sharing.

We've used that pinned solution in the past and have, indeed, already deployed such a fix today.

I recognize that this isn't your problem per se, but I do want to explicitly point out the fact that a recommendation to fix a deployment issue in one Google Infrastructure component actually introduces a new issue in a separate Google infrastructure component.

Google Cloud Functions deployments will raise an error if a package.json when engine is not a whole number (anything other than 12,14,16, etc..) ... but the solution recommended here requires specifying an engine value other than the above (e.g. 16.17.1).

The result is that, without additional work, app engine deploys succeed and function deployments fail. This is in a monorepo where splitting the systems is impractical, even if possible.

To support both, our build systems basically do a sed to modify the pinned version for AppEngine to just a whole number for Functions.

In any case, thansk for hearing me out.

We really appreciate the work y'all do to keep GCP stable and performant.

[nataliemeurer]

Hi folks - We hit this issue as well with our NodeJS app engine deployment. We were running 16.x. Pinning did end up working at a pin of 16.17.x. For future devs that hit this issue, make sure you pin to the precise version - 16.17.x NOT ^16.17.x

Here's a trace:

Step #2: INFO[0017] Taking snapshot of full filesystem...
Step #2: INFO[0024] Using files from context: [/workspace]       
Step #2: INFO[0024] COPY . /app/                                 
Step #2: INFO[0024] Taking snapshot of files...
Step #2: INFO[0026] RUN /usr/local/bin/install_node '^16.17.x'   
Step #2: INFO[0026] cmd: /bin/sh                                 
Step #2: INFO[0026] args: [-c /usr/local/bin/install_node '^16.17.x'] 
Step #2:   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
Step #2:                                  Dload  Upload   Total   Spent    Left  Speed
100 32.3M  100 32.3M    0     0  90.4M      0 --:--:-- --:--:-- --:--:-- 90.6M
Step #2:   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
Step #2:                                  Dload  Upload   Total   Spent    Left  Speed
100  4035  100  4035    0     0  46219      0 --:--:-- --:--:-- --:--:-- 46379
Step #2: gpg: Signature made Wed Oct 12 14:48:44 2022 UTC using RSA key ID 94EC4689 gpg: Can't check signature: public key not found
Step #2: The Node.js binary could not be verified.
Step #2: This means it may not be an officially released Node.js binary
Step #2: or may have been tampered with.
Step #2: 
Step #2: Aborting the installation.
Step #2: 
Step #2: The installation can be forced using the --ignore-verification-failure
Step #2: flag. However, it is strongly recommended that you install a version
Step #2: of Node.js that can be verified.
Step #2: 
Step #2: Node installation failed: /opt/gcp/runtime/bootstrap_node exited with a non-zero exit code: 1
Step #2: error building image: error building stage: waiting for process to exit: exit status 1
Finished Step #2
ERROR
ERROR: build step 2 "gcr.io/kaniko-project/executor@sha256:f87c11770a4d3ed33436508d206c584812cd656e6ed08eda1cff5c1ee44f5870" failed: step exited with non-zero status: 1

[jinglundong]

We updated the key at #254 and #256. I verified this in prod with a 16.x app that got resolved to 16.18.1. Please let us know if you have any other questions.

To be clear, this doesn't solve the problem of 18.x. Node.js 18 support requires a bigger effort. We have it planned in the first half of 2023.

[m-s-morgan]

@jinglundong is there any reason why it would not even try and run an install of the node version specifed in the package.json? We have our engine specified and have tried pinning to 16.17.1 but it doesn't even attempt an install. Please see the output below:

FETCHSOURCE
BUILD
Starting Step #0 - "fetcher"
Step #0 - "fetcher": Already have image (with digest): gcr.io/cloud-builders/gcs-fetcher
Step #0 - "fetcher": Fetching manifest gs://staging.****************.appspot.com/ae/5d15c126-2759-4b47-926a-5ee0ae1a4f93/manifest.json.
Step #0 - "fetcher": Processing 112 files.
Step #0 - "fetcher": ******************************************************
Step #0 - "fetcher": Status:                      SUCCESS
Step #0 - "fetcher": Started:                     2022-11-08T04:47:33Z
Step #0 - "fetcher": Completed:                   2022-11-08T04:47:33Z
Step #0 - "fetcher": Requested workers:    200
Step #0 - "fetcher": Actual workers:       112
Step #0 - "fetcher": Total files:          112
Step #0 - "fetcher": Total retries:          0
Step #0 - "fetcher": GCS timeouts:           0
Step #0 - "fetcher": MiB downloaded:         0.49 MiB
Step #0 - "fetcher": MiB/s throughput:       2.57 MiB/s
Step #0 - "fetcher": Time for manifest:    104.01 ms
Step #0 - "fetcher": Total time:             0.30 s
Step #0 - "fetcher": ******************************************************
Finished Step #0 - "fetcher"
Starting Step #1
Step #1: Pulling image: gcr.io/gcp-runtimes/nodejs/gen-dockerfile@sha256:8e2b8eef87fae1bc4b80206cc84139a52973433daafef5ba30ae84f2f85d01b6
Step #1: gcr.io/gcp-runtimes/nodejs/gen-dockerfile@sha256:8e2b8eef87fae1bc4b80206cc84139a52973433daafef5ba30ae84f2f85d01b6: Pulling from gcp-runtimes/nodejs/gen-dockerfile
Step #1: Digest: sha256:8e2b8eef87fae1bc4b80206cc84139a52973433daafef5ba30ae84f2f85d01b6
Step #1: Status: Downloaded newer image for gcr.io/gcp-runtimes/nodejs/gen-dockerfile@sha256:8e2b8eef87fae1bc4b80206cc84139a52973433daafef5ba30ae84f2f85d01b6
Step #1: gcr.io/gcp-runtimes/nodejs/gen-dockerfile@sha256:8e2b8eef87fae1bc4b80206cc84139a52973433daafef5ba30ae84f2f85d01b6
Step #1: Checking for Node.js.
Finished Step #1
Starting Step #2
Step #2: Already have image (with digest): gcr.io/kaniko-project/executor@sha256:f87c11770a4d3ed33436508d206c584812cd656e6ed08eda1cff5c1ee44f5870
Step #2: INFO[0000] Removing ignored files from build context: [node_modules .dockerignore Dockerfile npm-*****.log yarn-error.log .git .hg .svn app.yaml] 
Step #2: INFO[0000] Downloading base image gcr.io/google-appengine/nodejs@sha256:f4979bc91ed28ae6fc489b5377fb87d628263fe7a7da971b134c03aff47327b2
Step #2: INFO[0017] Taking snapshot of full filesystem...
Step #2: INFO[0024] Using files from context: [/workspace]       
Step #2: INFO[0024] COPY . /app/                                 
Step #2: INFO[0024] RUN NODE_ENV=*********** yarn install ||   ((if [ -f yarn-error.log ]; then       cat yarn-error.log;     fi) && false) 
Step #2: INFO[0024] cmd: /bin/sh                                 
Step #2: INFO[0024] args: [-c NODE_ENV=*********** yarn install ||   ((if [ -f yarn-error.log ]; then       cat yarn-error.log;     fi) && false)]
Step #2: yarn install v1.22.19
Step #2: [1/5] Validating package.json...
Step #2: error *************-api@1.0.0: The engine "node" is incompatible with this module. Expected version "16.17.1". Got "12.19.0"
Step #2: error Found incompatible module

[jinglundong]

@m-s-morgan Hi Matt, if my following analysis is moving towards the right direction, could you file a different issue?

I'm seeing the error at step #2, which looks like a version conflict between yarn v1.22.19 and nodejs 16.17.1.

Step #2: yarn install v1.22.19
Step #2: [1/5] Validating package.json...
Step #2: error *************-api@1.0.0: The engine "node" is incompatible with this module. Expected version "16.17.1". Got "12.19.0"

A few actionable ideas:

• Would you be able to start the app locally? Like yarn start.
• Do you mind sharing the related section of your package.json and/or the lock file?
• Do you know where is this version "12.19.0" coming from?

[m-s-morgan]

@m-s-morgan Hi Matt, if my following analysis is moving towards the right direction, could you file a different issue?

I'm seeing the error at step #2, which looks like a version conflict between yarn v1.22.19 and nodejs 16.17.1.

Step #2: yarn install v1.22.19
Step #2: [1/5] Validating package.json...
Step #2: error *************-api@1.0.0: The engine "node" is incompatible with this module. Expected version "16.17.1". Got "12.19.0"

A few actionable ideas:

• Would you be able to start the app locally? Like yarn start.

@jinglundong I can start the app locally just fine. The version of yarn is incompatible with this version of node. In our CI environment our tests run just fine using node 16.17.1 and yarn 1.22.19.

• Do you mind sharing the related section of your package.json and/or the lock file?

The related section of our package.json is below:

  "private": true,
  "main": "lib/index.js",
  "engines": {
    "node": "16.17.1",
    "yarn": ">=1.22.10"
  },

We've also tried locking the version of yarn to an earlier version and to 1.22.19.

• Do you know where is this version "12.19.0" coming from?

12.19.0 is the version that Google Cloud is using. I have no idea why it's using 12.19.0 and not 16.17.1 like I've specified. That is why I'm asking what I could be doing wrong to not even attempt to install what we have listed in our packge.json. Generally I'd expect to see a line like:

RUN /usr/local/bin/install_node '16.17.1' 

But that never appears. We are not using a cloudbuild.yaml nor using our own Dockerfile so this is the default App Engine build

[kennethye1]

@m-s-morgan I wasn't able to reproduce with a basic hello world. However, the most likely suspect is

<% if (config.canInstallDeps) { %>
  <% if (config.useYarn) { %>
RUN NODE_ENV=development yarn install || \
  ((if [ -f yarn-error.log ]; then \
      cat yarn-error.log; \
    fi) && false)
  <% } else { %>
# You have to specify "--unsafe-perm" with npm install
# when running as root.  Failing to do this can cause
# install to appear to succeed even if a preinstall
# script fails, and may have other adverse consequences
# as well.
# This command will also cat the npm-debug.log file after the
# build, if it exists.
RUN NODE_ENV=development npm install --unsafe-perm || \
  ((if [ -f npm-debug.log ]; then \
      cat npm-debug.log; \
    fi) && false)
  <% } %>
<% } %>
RUN npm run gcp-build
  <% if (config.canInstallDeps) { %>
RUN rm -Rf node_modules
  <% } %>

in here is being called somehow before the RUN /usr/local/bin/install_node '16.17.1' .

[jinglundong]

@m-s-morgan I think #233 (comment) is the tracking bug for the issue you are experiencing. I shared more details from my investigation with Kenneth.

[m-s-morgan]

@jinglundong @kennethye1 I'm assuming the yarn install here

  <% if (config.useYarn) { %>
RUN NODE_ENV=development yarn install || \

is causing the issue. The first thing that command does is check the package.json. The node binary installed by the runtime is 12.19.0 and the package.json has the node version pinned to 16.17.1. This of course is going to cause a failure. I believe the docker file needs to install the specified config node version before running the above code as it does a few steps later.

[jinglundong]

Let's track this issue at #233 if you don't mind.

[WonderfulDev0810]

you can change engine's sub node version to fix it.
in my case, I fixed node: 14.16

[REPTILEHAUS]

How is this still an issue, trying to use node v16 and getting this, trying to use node LTS v18 and getting glib errors GoogleCloudPlatform/buildpacks#198

almost impossible to use app engine for anything critical these days

[kennethye1]

Hi @REPTILEHAUS, I've updated the KEYS. For nodejs 18 and later, please follow https://cloud.google.com/appengine/docs/flexible/nodejs/runtime#newversions.