Use VS Code packaging for releases (#7721)

* Allow setting the VS Code build target

For the NPM package (and tests, at least for now), we will still use
linux-x64, but this is going to allow using the platform build targets
for our standalone releases so we can avoid having to copy all the
packaging steps (like cleaning up modules).

This does mean that the NPM package when installed will be missing those
cleanup steps.  Possibly we can try to break out the packaging step into
a something that can be ran standalone (which will also require
installing dev dependencies like gulp) but not sure how much work this
would be.

* Preserve dependencies for e2e tests

To avoid having to install them again.

Also moved an env block to the root of the  job.

* Refactor releases to use VS Code packaging

Instead of building the linux-x64 package, stripping the modules, then
installing them again, we build the correct target and use the modules
as they are.

This means we do not have to copy all the post-processing steps like the
ones that delete unnecessary modules.

For the NPM package we still publish the linux-x64 package (without
modules of course).  This means npm installations do not get that same
post-processing.

Another advantage of this is that we can run the release immediately
without having to wait for the build step, or on a commit that no longer
has a build artifact, since they all build individually now.  We could
try sharing the core-ci build step, but leaving that alone for now.

I also converted the macOS jobs into a matrix.

Deleted the CI readme because it was out of date and seemed to just
repeat what should be described in the scripts anyway.

Removed a section about Homebrew since we do not maintain that anymore.

It looks like there is no need to symlink node_modules.asar anymore.

Author: code-asher

.github/workflows/build.yaml

@@ -15,12 +15,6 @@ concurrency:
   group: ${{ github.workflow }}-${{ github.ref }}
   cancel-in-progress: ${{ github.event_name == 'pull_request' }}
 
-# Note: if: success() is used in several jobs -
-# this ensures that it only executes if all previous jobs succeeded.
-
-# if: steps.cache-node-modules.outputs.cache-hit != 'true'
-# will skip running `npm install` if it successfully fetched from cache
-
 jobs:
   changes:
     runs-on: ubuntu-latest
@@ -31,10 +25,8 @@ jobs:
       docs: ${{ steps.filter.outputs.docs }}
       helm: ${{ steps.filter.outputs.helm }}
     steps:
-      - name: Checkout repo
-        uses: actions/checkout@v6
-      - name: Check changed files
-        uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3
+      - uses: actions/checkout@v6
+      - uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3
         id: filter
         with:
           filters: |
@@ -157,20 +149,25 @@ jobs:
           token: ${{ secrets.CODECOV_TOKEN }}
 
   build:
-    name: Build code-server
+    name: linux-x64
     runs-on: ubuntu-22.04
     env:
       CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
       DISABLE_V8_COMPILE_CACHE: 1
+      VERSION: 0.0.0
+      VSCODE_TARGET: linux-x64
+      GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+
     steps:
-      - uses: actions/checkout@v6
-        with:
-          submodules: true
       - run: sudo apt update && sudo apt install -y libkrb5-dev
       - uses: awalsh128/cache-apt-pkgs-action@2c09a5e66da6c8016428a2172bd76e5e4f14bb17 # latest
         with:
           packages: quilt
           version: 1.0
+
+      - uses: actions/checkout@v6
+        with:
+          submodules: true
       - run: quilt push -a
       - uses: actions/setup-node@v6
         with:
@@ -181,51 +178,43 @@ jobs:
             test/package-lock.json
       - run: SKIP_SUBMODULE_DEPS=1 npm ci
       - run: npm run build
-        env:
-          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
       # Get Code's git hash.  When this changes it means the content is
       # different and we need to rebuild.
       - name: Get latest lib/vscode rev
         id: vscode-rev
         run: echo "rev=$(git rev-parse HEAD:./lib/vscode)" >> $GITHUB_OUTPUT
-      # We need to rebuild when we have a new version of Code, when any of
-      # the patches changed, or when the code-server version changes (since
-      # it gets embedded into the code).  Use VSCODE_CACHE_VERSION to
-      # force a rebuild.
-      - name: Fetch prebuilt Code package from cache
+      # We need to rebuild when we have a new version of Code, when any of the
+      # patches changed, or when the code-server version changes (since it gets
+      # embedded into the code).  Use VSCODE_CACHE_VERSION to force a rebuild.
+      - name: Fetch prebuilt linux-x64 Code package from cache
         id: cache-vscode
         uses: actions/cache@v4
         with:
-          path: lib/vscode-reh-web-*
-          key: vscode-reh-package-${{ secrets.VSCODE_CACHE_VERSION }}-${{ steps.vscode-rev.outputs.rev }}-${{ hashFiles('patches/*.diff', 'ci/build/build-vscode.sh') }}
+          path: lib/vscode-reh-web-linux-x64
+          key: vscode-linux-x64-package-${{ secrets.VSCODE_CACHE_VERSION }}-${{ steps.vscode-rev.outputs.rev }}-${{ hashFiles('patches/*.diff', 'ci/build/build-vscode.sh') }}
       - name: Build vscode
-        env:
-          VERSION: "0.0.0"
         if: steps.cache-vscode.outputs.cache-hit != 'true'
         run: |
           pushd lib/vscode
           npm ci
           popd
           npm run build:vscode
-      # The release package does not contain any native modules
-      # and is neutral to architecture/os/libc version.
-      - run: npm run release
-        if: success()
-      # https://github.com/actions/upload-artifact/issues/38
+      # Push up an artifact containing the linux-x64 release.
+      - run: KEEP_MODULES=1 npm run release
       - run: tar -czf package.tar.gz release
       - uses: actions/upload-artifact@v7
         with:
-          name: npm-package
+          name: linux-x64-package
           path: ./package.tar.gz
 
   test-e2e:
     name: Run e2e tests
     runs-on: ubuntu-22.04
     needs: [changes, build]
-    if: needs.changes.outputs.code == 'true' || needs.changes.outputs.deps == 'true'
+    if: needs.changes.outputs.code == 'true' || needs.changes.outputs.deps == 'true' || needs.changes.outputs.ci == 'true'
+
     steps:
       - uses: actions/checkout@v6
-      - run: sudo apt update && sudo apt install -y libkrb5-dev
       - uses: actions/setup-node@v6
         with:
           node-version-file: .node-version
@@ -234,31 +223,47 @@ jobs:
             package-lock.json
             test/package-lock.json
       - run: SKIP_SUBMODULE_DEPS=1 npm ci
-      - uses: actions/download-artifact@v8
-        with:
-          name: npm-package
-      - run: tar -xzf package.tar.gz
-      - run: cd release && npm install --unsafe-perm --omit=dev
       - name: Install Playwright OS dependencies
         run: |
           ./test/node_modules/.bin/playwright install-deps
           ./test/node_modules/.bin/playwright install
+
+      - uses: actions/download-artifact@v8
+        with:
+          name: linux-x64-package
+      - run: tar -xzf package.tar.gz
+
       - run: CODE_SERVER_TEST_ENTRY=./release npm run test:e2e
       - uses: actions/upload-artifact@v7
         if: always()
         with:
           name: failed-test-videos
           path: ./test/test-results
-      - run: rm -rf ./release ./test/test-results
 
   test-e2e-proxy:
     name: Run e2e tests behind proxy
     runs-on: ubuntu-22.04
+    env:
+      GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
     needs: [changes, build]
-    if: needs.changes.outputs.code == 'true' || needs.changes.outputs.deps == 'true'
+    if: needs.changes.outputs.code == 'true' || needs.changes.outputs.deps == 'true' || needs.changes.outputs.ci == 'true'
+
     steps:
+      - name: Cache Caddy
+        uses: actions/cache@v4
+        id: caddy-cache
+        with:
+          path: |
+            ~/.cache/caddy
+          key: cache-caddy-2.5.2
+      - name: Install Caddy
+        if: steps.caddy-cache.outputs.cache-hit != 'true'
+        run: |
+          gh release download v2.5.2 --repo caddyserver/caddy --pattern "caddy_2.5.2_linux_amd64.tar.gz"
+          mkdir -p ~/.cache/caddy
+          tar -xzf caddy_2.5.2_linux_amd64.tar.gz --directory ~/.cache/caddy
+
       - uses: actions/checkout@v6
-      - run: sudo apt update && sudo apt install -y libkrb5-dev
       - uses: actions/setup-node@v6
         with:
           node-version-file: .node-version
@@ -267,35 +272,20 @@ jobs:
             package-lock.json
             test/package-lock.json
       - run: SKIP_SUBMODULE_DEPS=1 npm ci
-      - uses: actions/download-artifact@v8
-        with:
-          name: npm-package
-      - run: tar -xzf package.tar.gz
-      - run: cd release && npm install --unsafe-perm --omit=dev
       - name: Install Playwright OS dependencies
         run: |
           ./test/node_modules/.bin/playwright install-deps
           ./test/node_modules/.bin/playwright install
-      - name: Cache Caddy
-        uses: actions/cache@v4
-        id: caddy-cache
+
+      - uses: actions/download-artifact@v8
         with:
-          path: |
-            ~/.cache/caddy
-          key: cache-caddy-2.5.2
-      - name: Install Caddy
-        env:
-          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-        if: steps.caddy-cache.outputs.cache-hit != 'true'
-        run: |
-          gh release download v2.5.2 --repo caddyserver/caddy --pattern "caddy_2.5.2_linux_amd64.tar.gz"
-          mkdir -p ~/.cache/caddy
-          tar -xzf caddy_2.5.2_linux_amd64.tar.gz --directory ~/.cache/caddy
+          name: linux-x64-package
+      - run: tar -xzf package.tar.gz
+
       - run: ~/.cache/caddy/caddy start --config ./ci/Caddyfile
       - run: CODE_SERVER_TEST_ENTRY=./release npm run test:e2e:proxy
       - run: ~/.cache/caddy/caddy stop --config ./ci/Caddyfile
         if: always()
-
       - uses: actions/upload-artifact@v7
         if: always()
         with:

.github/workflows/publish.yaml

@@ -6,7 +6,6 @@ on:
   workflow_dispatch:
     inputs:
       version:
-        description: The version to publish (include "v", i.e. "v4.9.1").
         type: string
         required: true
 
@@ -23,50 +22,45 @@ concurrency:
 jobs:
   npm:
     runs-on: ubuntu-latest
+    env:
+      TAG: ${{ inputs.version || github.ref_name }}
+      GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
+      NPM_ENVIRONMENT: "production"
+
     steps:
-      - name: Checkout code-server
-        uses: actions/checkout@v6
+      - name: Set version to tag without leading v
+        run: |
+          echo "VERSION=${TAG#v}" >> $GITHUB_ENV
 
-      - name: Install Node.js
-        uses: actions/setup-node@v6
+      - uses: actions/checkout@v6
+      - uses: actions/setup-node@v6
         with:
           node-version-file: .node-version
 
-      - name: Download npm package from release artifacts
-        uses: robinraju/release-downloader@daf26c55d821e836577a15f77d86ddc078948b05 # v1.12
+      - uses: robinraju/release-downloader@daf26c55d821e836577a15f77d86ddc078948b05 # v1.12
         with:
           repository: "coder/code-server"
           tag: ${{ inputs.version || github.ref_name }}
           fileName: "package.tar.gz"
           out-file-path: "release-npm-package"
 
-      # Strip out the v (v4.9.1 -> 4.9.1).
-      - name: Get and set VERSION
-        run: |
-          echo "VERSION=${TAG#v}" >> $GITHUB_ENV
-
-        env:
-          TAG: ${{ inputs.version || github.ref_name }}
-      - run: npm run publish:npm
-        env:
-          VERSION: ${{ env.VERSION }}
-          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-          NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
-          NPM_ENVIRONMENT: "production"
+      - run: tar -xzf release-npm-package/package.tar.gz
+      - run: |
+          pushd release
+          npm publish --access public
 
   aur:
     runs-on: ubuntu-latest
     timeout-minutes: 10
     env:
       GH_TOKEN: ${{ secrets.HOMEBREW_GITHUB_API_TOKEN }}
+      TAG: ${{ inputs.version || github.ref_name }}
 
     steps:
-      # We need to checkout code-server so we can get the version
-      - name: Checkout code-server
-        uses: actions/checkout@v6
-        with:
-          fetch-depth: 0
-          path: "./code-server"
+      - name: Set version to tag without leading v
+        run: |
+          echo "VERSION=${TAG#v}" >> $GITHUB_ENV
 
       - name: Checkout code-server-aur repo
         uses: actions/checkout@v6
@@ -86,27 +80,14 @@ jobs:
           git config --global user.name cdrci
           git config --global user.email opensource@coder.com
 
-      # Strip out the v (v4.9.1 -> 4.9.1).
-      - name: Get and set VERSION
-        run: |
-          echo "VERSION=${TAG#v}" >> $GITHUB_ENV
-
-        env:
-          TAG: ${{ inputs.version || github.ref_name }}
       - name: Validate package
         uses: heyhusen/archlinux-package-action@c9f94059ccbebe8710d31d582f33ef4e84fe575c # v3.0.0
-        env:
-          VERSION: ${{ env.VERSION }}
         with:
           pkgver: ${{ env.VERSION }}
           updpkgsums: true
           srcinfo: true
 
       - name: Open PR
-        # We need to git push -u otherwise gh will prompt
-        # asking where to push the branch.
-        env:
-          VERSION: ${{ env.VERSION }}
         run: |
           git checkout -b update-version-${{ env.VERSION }}
           git add .
@@ -116,54 +97,40 @@ jobs:
 
   docker:
     runs-on: ubuntu-latest
-    steps:
-      - name: Checkout code-server
-        uses: actions/checkout@v6
+    env:
+      GITHUB_TOKEN: ${{ github.token }}
+      TAG: ${{ inputs.version || github.ref_name }}
 
-      - name: Set up QEMU
-        uses: docker/setup-qemu-action@c7c53464625b32c7a7e944ae62b3e17d2b600130 # v3
+    steps:
+      - name: Set version to tag without leading v
+        run: |
+          echo "VERSION=${TAG#v}" >> $GITHUB_ENV
 
-      - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3
+      - uses: actions/checkout@v6
+      - uses: docker/setup-qemu-action@c7c53464625b32c7a7e944ae62b3e17d2b600130 # v3
+      - uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3
 
-      - name: Login to Docker Hub
-        uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
+      - uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
         with:
           username: ${{ secrets.DOCKER_USERNAME }}
           password: ${{ secrets.DOCKER_PASSWORD }}
-
-      - name: Login to GHCR
-        uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
+      - uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
         with:
           registry: ghcr.io
           username: ${{ github.actor }}
           password: ${{ secrets.GITHUB_TOKEN }}
 
-      # Strip out the v (v4.9.1 -> 4.9.1).
-      - name: Get and set VERSION
-        run: |
-          echo "VERSION=${TAG#v}" >> $GITHUB_ENV
-
-        env:
-          TAG: ${{ inputs.version || github.ref_name }}
-      - name: Download deb artifacts
-        uses: robinraju/release-downloader@daf26c55d821e836577a15f77d86ddc078948b05 # v1.12
+      - uses: robinraju/release-downloader@daf26c55d821e836577a15f77d86ddc078948b05 # v1.12
         with:
           repository: "coder/code-server"
           tag: v${{ env.VERSION }}
           fileName: "*.deb"
           out-file-path: "release-packages"
-
-      - name: Download rpm artifacts
-        uses: robinraju/release-downloader@daf26c55d821e836577a15f77d86ddc078948b05 # v1.12
+      - uses: robinraju/release-downloader@daf26c55d821e836577a15f77d86ddc078948b05 # v1.12
         with:
           repository: "coder/code-server"
           tag: v${{ env.VERSION }}
           fileName: "*.rpm"
           out-file-path: "release-packages"
 
-      - name: Publish to Docker
-        run: ./ci/steps/docker-buildx-push.sh
-        env:
-          VERSION: ${{ env.VERSION }}
-          GITHUB_TOKEN: ${{ github.token }}
+      - run: npm run publish:docker