Add Multiplatform Nightly musl Images #4731
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ponylang-main
added
the
discuss during sync
|
This will fail until:
both will be happening soon. |
SeanTAllen
force-pushed
the
multiplatform-musl-nightly
branch
from
fd919e8 to
f2b9855
Compare
SeanTAllen
commented
Jun 4, 2025
|
|
||
| concurrency: | ||
| group: build-nightly-musl-image | ||
| cancel-in-progress: false |
There was a problem hiding this comment.
cancelling in progress could result in unrelated sync events killing off both of our expected events and resulting in no image.
This introduces a number of changes for our container image building. It introduces the eventual deprecation of `latest` as a tag. At the moment, our use of `latest` is misleading. That is supposed to be "most recent" but we don't use it as such. For us it is "most recently nightly". This commit will be the start of us having "nightly" images that are explicitly a nightly build. Additionally, our tags will have a qualifier for the libc version. In the case of this musl image the tag would be "musl-nightly". When we do the same for GNU libc, the tag would be "gnu-nightly". This same name scheme will end up being carried over to releases: "musl-release" and "gnu-release" as well as "by version" tags "musl-0.60.0" and "gnu-0.60.0". The older tags will be dropped as this rolls out. We can if we want then start using a variation on latest with "musl-latest" and "gnu-latest" should we want, or for ponyc images we can ditch the entire concept of "latest" as "nightly" and "release" versions will always be for a given library the latest nightly and latest release. In addition the tag name changes, this also is our first multiplatform image. The build process for this is as simple as I could make it. We have a new workflow for building the nightly image. It is set to have a concurrency of 1 so, as events arrive from Cloudsmith, we will build a new version of the image with whatever is the latest nightly arm64 and latest nightly am64 version of ponyc that is available in Cloudsmith. This means that as we upload to Cloudsmith each night we will create the image twice. If the arm64 synchronized message arrives first and then the amd64 one we will have the following happen: - arm64 message arrives - Build a new "nightly image" with today's arm64 and yesterday's amd64 - amd64 message arrives - Build a new "nightly image" with today's arm64 and today's arm64 The same pattern would be applied to releases. At the time the message arrives, the image will be built using the most recent version of ponyc for each platform that is available. This has a couple nice qualities to it: - If one failed to build, we still make the one that succeeded available. - It makes our configuration relatively simple Without this "cross over", we would need to wait for each message to have arrived and then build the final image. That is some tricky yaml that we should want to avoid doing if possible.
SeanTAllen
force-pushed
the
multiplatform-musl-nightly
branch
from
f2b9855 to
25396bb
Compare
SeanTAllen
commented
Jun 4, 2025
| topic: ${{ github.repository }} scheduled job failure | ||
| content: ${{ github.server_url}}/${{ github.repository }}/actions/runs/${{ github.run_id }} failed. | ||
|
|
||
| send-repository-dispatch-event: |
There was a problem hiding this comment.
The downside to this approach is that we will send this event twice and double the downstream work of "verify we work with latest" and also "rebuild shared docker images" or whatever else the event triggers.
We could add the complicated "only if both logic" as a gate in front of this event but that feels like work for another day if we decide it is a problem. This starts at the simplest solution.
There was a problem hiding this comment.
The downside to not coalescing to a single dispatch event here is that downstream could end up with a race condition unless it also has the workflow concurrency set to only handle a single event at a time.
|
Looks good to me. |
SeanTAllen
removed
the
discuss during sync
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This introduces a number of changes for our container image building.
It introduces the eventual deprecation of
latestas a tag. At the moment, our use oflatestis misleading. That is supposed to be "most recent" but we don't use it as such. For us it is "most recently nightly".This commit will be the start of us having "nightly" images that are explicitly a nightly build. Additionally, our tags will have a qualifier for the libc version. In the case of this musl image the tag would be "musl-nightly". When we do the same for GNU libc, the tag would be "gnu-nightly".
This same name scheme will end up being carried over to releases: "musl-release" and "gnu-release" as well as "by version" tags "musl-0.60.0" and "gnu-0.60.0". The older tags will be dropped as this rolls out.
We can if we want then start using a variation on latest with "musl-latest" and "gnu-latest" should we want, or for ponyc images we can ditch the entire concept of "latest" as "nightly" and "release" versions will always be for a given library the latest nightly and latest release.
In addition the tag name changes, this also is our first multiplatform image. The build process for this is as simple as I could make it. We have a new workflow for building the nightly image. It is set to have a concurrency of 1 so, as events arrive from Cloudsmith, we will build a new version of the image with whatever is the latest nightly arm64 and latest nightly am64 version of ponyc that is available in Cloudsmith.
This means that as we upload to Cloudsmith each night we will create the image twice. If the arm64 synchronized message arrives first and then the amd64 one we will have the following happen:
The same pattern would be applied to releases.
At the time the message arrives, the image will be built using the most recent version of ponyc for each platform that is available. This has a couple nice qualities to it:
Without this "cross over", we would need to wait for each message to have arrived and then build the final image. That is some tricky yaml that we should want to avoid doing if possible.