Skip to content

Release CI x64 reconfiguration #4390

Description

@richardlau

Currently on the release CI we have three x64 machines:

These three machines service two builds:

We have observed long build times, usually due to ccache invalidation/not being populated. For the latter the theory is that having three machines may be actively working against ccache since for a given build only two of the three machines would be used (so the third will not have an updated ccache cache).

I'm going to take the second IBM hosted release machine offline, leaving one Digital Ocean hosted and one IBM hosted machine. For now we'll keep both labels on both machines, but reserve the option to pin each of the remaining machines to one of the build types.

Activity

  1. richardlau commented on Jul 2, 2026

    @richardlau
    MemberAuthor

    release-ibm-rhel8-x64-2 now marked offline.

  2. pinned this issue on Jul 29, 2026
  3. sxa commented on Jul 29, 2026

    @sxa
    Member

    release-ibm-rhel8-x64-2 now marked offline.

    I've been having a look at utilising rthis system for hosting an Alpine container. I'm in two minds between doing this with podman on the existing RHEL host (which will require changes to the playbooks and probably building/starting as a non-root user such as iojs) or reinstalling with Ubuntu and using docker.

    At the moment the machine is quite restricted on disk space (particularly with the ccache) so we may need to rebuild anyway, or at least add additional space for the containers, so I'm perhaps leaning down the reinstall route to retain consistency with the other dockerhost systems that we have.

  4. sxa commented on Aug 5, 2026

    @sxa
    Member

    I am currently provisioning release-ibm-ubuntu2404_docker-x64-1 to host an Alpine 3.23 container release-alpine323_container-x64-1.
    This will likely mean that we can just delete the release-ibm-rhel8-x64-2 once everything is verifed as working. Noting that we got through the last set of releases last week, including the security releases, without the release-ibm-rhel8-x64-2 system.

    Related PR: #4414

  5. sxa commented on Aug 12, 2026

    @sxa
    Member

    With the addition of xz to the Dockerfile the new Alpine release machine looks ok and gets as far as trying to copy to node-www which it can't because the firewall rules aren't set up yet.

  6. sxa commented on Aug 27, 2026

    @sxa
    Member

    Alpine machine now fully set up, working and has performed a release. release-ibm-rhel8-x64-2 can likely be decommissioned now and then I think we're done with this issue.

  7. unpinned this issue on Aug 28, 2026
  8. richardlau commented on Sep 23, 2026

    @richardlau
    MemberAuthor

    I'm going to replace release-ibm-rhel8-x64-2 with a RHEL 9 x64 release machine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions