Fix the assembly of the Docker image in CI #3584

danicheg · 2025-02-10T10:14:24Z

Currently, we're experiencing this issue in CI workflows:

[info] ERROR: failed to solve: process "/bin/sh -c npm install --global yarn" did not complete successfully: exit code: 127
[error] java.lang.RuntimeException: Nonzero exit value: 1
[error] 	at com.typesafe.sbt.packager.docker.DockerPlugin$.publishLocalDocker(DockerPlugin.scala:745)

It turns out, that we run npm install --global yarn within the docker settings defined in the SBT build. So, it's an attempt to fix the issue.

exoego · 2025-02-10T10:22:47Z

build.sbt

+      name = Some("Setup NodeJS v18 LTS"),
+      params = Map("node-version" -> "18")


Suggested change

name = Some("Setup NodeJS v18 LTS"),

params = Map("node-version" -> "18")

name = Some("Setup NodeJS LTS"),

params = Map("node-version" -> "22")

v18 is already at maintenance phase and EOL in three months.
v22 is two years longer, so I think it would be a better choice.

It was a deliberate choice, since our upstream (core library actually), Cats-Effect, is on v18 in their CI. If it's not a thing, we can bump it.

exoego · 2025-02-10T10:27:05Z

Maybe apk add --update npm is missing in

scala-steward/build.sbt

Lines 363 to 410 in 0171537

    
           lazy val dockerSettings = Def.settings( 
        
             dockerBaseImage := Option(System.getenv("DOCKER_BASE_IMAGE")) 
        
               .getOrElse("eclipse-temurin:11-alpine"), 
        
             dockerCommands ++= { 
        
               val curl = "curl -fL --output" 
        
               val binDir = "/usr/local/bin" 
        
               val sbtVer = sbtVersion.value 
        
               val sbtTgz = s"sbt-$sbtVer.tgz" 
        
               val installSbt = Seq( 
        
                 s"$curl $sbtTgz https://github.com/sbt/sbt/releases/download/v$sbtVer/$sbtTgz", 
        
                 s"tar -xf $sbtTgz", 
        
                 s"rm -f $sbtTgz" 
        
               ).mkString(" && ") 
        
               val millVer = Dependencies.millMain.revision 
        
               val millBin = s"$binDir/mill" 
        
               val installMill = Seq( 
        
                 s"$curl $millBin https://repo1.maven.org/maven2/com/lihaoyi/mill-dist/$millVer/mill", 
        
                 s"chmod +x $millBin" 
        
               ).mkString(" && ") 
        
               val csBin = s"$binDir/cs" 
        
               val installCoursier = Seq( 
        
                 s"$curl $csBin.gz https://github.com/coursier/coursier/releases/download/v${Dependencies.coursierCore.revision}/cs-x86_64-pc-linux-static.gz", 
        
                 s"gunzip $csBin.gz", 
        
                 s"chmod +x $csBin" 
        
               ).mkString(" && ") 
        
               val scalaCliBin = s"$binDir/scala-cli" 
        
               val installScalaCli = Seq( 
        
                 s"$curl $scalaCliBin.gz https://github.com/Virtuslab/scala-cli/releases/latest/download/scala-cli-x86_64-pc-linux-static.gz", 
        
                 s"gunzip $scalaCliBin.gz", 
        
                 s"chmod +x $scalaCliBin" 
        
               ).mkString(" && ") 
        
               Seq( 
        
                 Cmd("USER", "root"), 
        
                 Cmd( 
        
                   "RUN", 
        
                   "apk --no-cache add bash git gpg ca-certificates curl maven openssh nodejs npm ncurses" 
        
                 ), 
        
                 Cmd("RUN", installSbt), 
        
                 Cmd("RUN", installMill), 
        
                 Cmd("RUN", installCoursier), 
        
                 Cmd("RUN", installScalaCli), 
        
                 Cmd("RUN", s"$csBin install --install-dir $binDir scalafix scalafmt"), 
        
                 Cmd("RUN", "npm install --global yarn"), 
        
                 // Ensure binaries are in PATH 
        
                 Cmd("RUN", "echo $PATH"), 
        
                 Cmd("RUN", "which cs mill mvn node npm sbt scala-cli scalafix scalafmt yarn") 
        
               ) 
        
             },

danicheg · 2025-02-10T10:38:27Z

Now, CI fails with

[info]   29 |     RUN /usr/local/bin/cs install --install-dir /usr/local/bin scalafix scalafmt
[info]   30 |     RUN echo $PATH
[info]   31 | >>> RUN which cs mill mvn node npm sbt scala-cli scalafix scalafmt yarn
[info]   32 |     
[info] --------------------
[info] ERROR: failed to solve: process "/bin/sh -c which cs mill mvn node npm sbt scala-cli scalafix scalafmt yarn" did not complete successfully: exit code: 1

Which is a bit more ambiguous. Is it missing yarn? But we have installed it already 🤔

mzuehlke · 2025-02-10T10:59:50Z

The error is happening when building the Docker image
so installing npm or yarn on the node won't be of any help (my guess)
why this stopped working now, no clue at the moment....

danicheg · 2025-02-10T11:09:37Z

The error is happening when building the Docker image
so installing npm or yarn on the node won't be of any help (my guess)
why this stopped working now, no clue at the moment....

It's true that installing nodejs/npm/yarn globally on a ci machine seemingly shouldn't affect the image build, but the only change that caused this issue, without modifying our build definition, might be related to changes in the running environment, IMHO. So that's why I tried this out.

danicheg · 2025-02-10T12:07:48Z

We have this

[info] #18 [mainstage  7/17] RUN apk --no-cache add nodejs npm
[info] #18 0.212 fetch https://dl-cdn.alpinelinux.org/alpine/v3.21/main/x86_64/APKINDEX.tar.gz
[info] #18 0.314 fetch https://dl-cdn.alpinelinux.org/alpine/v3.21/community/x86_64/APKINDEX.tar.gz
[info] #18 0.591 (1/11) Installing libgcc (14.2.0-r4)
[info] #18 0.601 (2/11) Installing libstdc++ (14.2.0-r4)
[info] #18 0.638 (3/11) Installing ada-libs (2.9.2-r1)
[info] #18 0.650 (4/11) Installing c-ares (1.34.3-r0)
[info] #18 0.659 (5/11) Installing icu-data-en (74.2-r0)
[info] #18 0.689 Executing icu-data-en-74.2-r0.post-install
[info] #18 0.693 *
[info] #18 0.693 * If you need ICU with non-English locales and legacy charset support, install
[info] #18 0.693 * package icu-data-full.
[info] #18 0.693 *
[info] #18 0.694 (6/11) Installing icu-libs (74.2-r0)
[info] #18 0.737 (7/11) Installing nghttp2-libs (1.64.0-r0)
[info] #18 0.745 (8/11) Installing simdjson (3.10.1-r0)
[info] #18 0.753 (9/11) Installing simdutf (5.6.3-r0)
[info] #18 0.762 (10/11) Installing nodejs (22.13.1-r0)
[info] #18 1.036 (11/11) Installing npm (10.9.1-r0)
[info] #18 1.259 Executing busybox-1.37.0-r9.trigger
[info] #18 1.267 OK: 104 MiB in 82 packages
[info] #18 DONE 1.4s

but

ERROR: failed to solve: process "/bin/sh -c node -v && npm -v && which node && which npm" did not complete successfully: exit code: 127

that's quite puzzling.

From the last successful build, we got these versions of nodejs/npm:

[info] #17 2.588 (41/50) Installing nodejs (22.11.0-r1)
[info] #17 2.860 (42/50) Installing npm (10.9.1-r0)

while now, we're getting these:

[info] #18 0.762 (10/11) Installing nodejs (22.13.1-r0)
[info] #18 1.036 (11/11) Installing npm (10.9.1-r0)

Also, 4 days ago ubuntu-22.04 was updated (JFTR, much likely does not relate).
And 10 days ago eclipse-temurin:11-alpine was updated (however, the sha remained the same 8631bdced20560ea97c24dc5ed6551a5c663155ef3d6d9e0458f52250c2d4d77 across the workflow runs)

I have no idea how all of these can result in our suffering. But that is it.

codecov · 2025-02-10T13:36:45Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 89.80%. Comparing base (acc41d8) to head (155e914).
Report is 5 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #3584   +/-   ##
=======================================
  Coverage   89.80%   89.80%           
=======================================
  Files         174      174           
  Lines        5031     5031           
  Branches      445      493   +48     
=======================================
  Hits         4518     4518           
  Misses        513      513

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

danicheg · 2025-02-10T13:37:12Z

Okay, it seems I found something.

mzuehlke · 2025-02-10T14:52:11Z

Okay, it seems I found something.

What did you found ?
We need sqlite in the Docker image ? That sound wired....

danicheg · 2025-02-10T15:03:52Z

@mzuehlke 🤷🏻

[info]  > [mainstage 12/14] RUN npm install --global yarn:
[info] 0.215 Error relocating /usr/bin/node: sqlite3session_attach: symbol not found
[info] 0.215 Error relocating /usr/bin/node: sqlite3changeset_apply: symbol not found
[info] 0.215 Error relocating /usr/bin/node: sqlite3session_create: symbol not found
[info] 0.215 Error relocating /usr/bin/node: sqlite3session_changeset: symbol not found
[info] 0.215 Error relocating /usr/bin/node: sqlite3session_patchset: symbol not found
[info] 0.215 Error relocating /usr/bin/node: sqlite3session_delete: symbol not found

It also amazes me, initially I thought the problem was with improper installation of nodejs and/or npm and/or yarn, but ultimately, simply adding sqlite modules resolved the issue (per what was printed during yarn installation).

Perhaps, the newer version of nodejs (which is now v22.13.1), requires the presence of these sqlite modules. I dug into the v22.13.0 release notes, and there are some references to sqlite, which might shed light on our issue. But I don't feel myself too excited to go deeper with it, honestly :)

fthomas

Many thanks for fixing the build, @danicheg!

danicheg force-pushed the fix-ci branch from 84c196b to 8cc3a16 Compare February 10, 2025 10:21

exoego reviewed Feb 10, 2025

View reviewed changes

danicheg force-pushed the fix-ci branch from 8cc3a16 to 0972b66 Compare February 10, 2025 10:27

danicheg force-pushed the fix-ci branch from 0972b66 to ec7a326 Compare February 10, 2025 10:44

danicheg force-pushed the fix-ci branch 2 times, most recently from 44ce10b to cb15590 Compare February 10, 2025 11:57

danicheg force-pushed the fix-ci branch 3 times, most recently from 886359e to 477eace Compare February 10, 2025 12:52

Fix the assembly of the Docker image in CI

155e914

danicheg force-pushed the fix-ci branch from ced6477 to 155e914 Compare February 10, 2025 13:45

danicheg changed the title ~~Use setup-node action in CI~~ Fix the assembly of the Docker image in CI Feb 10, 2025

danicheg marked this pull request as ready for review February 10, 2025 13:46

danicheg requested a review from exoego February 10, 2025 13:50

mzuehlke approved these changes Feb 10, 2025

View reviewed changes

fthomas approved these changes Feb 10, 2025

View reviewed changes

fthomas added this to the 0.33.0 milestone Feb 10, 2025

fthomas added the build label Feb 10, 2025

fthomas merged commit 231fe8f into scala-steward-org:main Feb 10, 2025
10 checks passed

danicheg deleted the fix-ci branch February 10, 2025 15:27

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix the assembly of the Docker image in CI #3584

Fix the assembly of the Docker image in CI #3584

danicheg commented Feb 10, 2025

exoego Feb 10, 2025 •

edited

Loading

danicheg Feb 10, 2025

exoego commented Feb 10, 2025

danicheg commented Feb 10, 2025 •

edited

Loading

mzuehlke commented Feb 10, 2025

danicheg commented Feb 10, 2025

danicheg commented Feb 10, 2025 •

edited

Loading

codecov bot commented Feb 10, 2025 •

edited

Loading

danicheg commented Feb 10, 2025

mzuehlke commented Feb 10, 2025

danicheg commented Feb 10, 2025 •

edited

Loading

fthomas left a comment

		name = Some("Setup NodeJS v18 LTS"),
		params = Map("node-version" -> "18")

Fix the assembly of the Docker image in CI #3584

Fix the assembly of the Docker image in CI #3584

Conversation

danicheg commented Feb 10, 2025

exoego Feb 10, 2025 • edited Loading

Choose a reason for hiding this comment

danicheg Feb 10, 2025

Choose a reason for hiding this comment

exoego commented Feb 10, 2025

danicheg commented Feb 10, 2025 • edited Loading

mzuehlke commented Feb 10, 2025

danicheg commented Feb 10, 2025

danicheg commented Feb 10, 2025 • edited Loading

codecov bot commented Feb 10, 2025 • edited Loading

Codecov Report

danicheg commented Feb 10, 2025

mzuehlke commented Feb 10, 2025

danicheg commented Feb 10, 2025 • edited Loading

fthomas left a comment

Choose a reason for hiding this comment

exoego Feb 10, 2025 •

edited

Loading

danicheg commented Feb 10, 2025 •

edited

Loading

danicheg commented Feb 10, 2025 •

edited

Loading

codecov bot commented Feb 10, 2025 •

edited

Loading

danicheg commented Feb 10, 2025 •

edited

Loading