Repository navigation
Upload dist assets to a staging R2 bucket before a release is promoted #3602
Description
Activity
Upload the release binaries and docs to the dist-staging bucket as soon as they're uploaded to the DO server
I suggest doing this between the "upload to DO" and "touch .done file" steps in the existing scripts (present in nodejs/node's Makefile and vcbuild.bat files). We can do the upload from the DO server via SSH.
Reacted by flakey5 and Claudio WunderI created the
dist-stagingbucket.Reacted by Claudio WunderGive each releaser their own S3 api token to write to the staging bucket (unless we want this step to happen from the DO server as well)
It will be easier to do it from the DO server, as the releaser already has access to it and already copies the shasums to it.
Reacted by flakey5, Claudio Wunder and Richard LauPr opened here in the node repo to copy both release assets and the signed shasum files to the staging R2 bucket: nodejs/node#51394
Reacted by Michaël Zasso- added 2 commits that reference this issue
on Jan 28, 2024 - added a commit that references this issue
on May 6, 2024 - added 2 commits that reference this issue
on May 15, 2024 - added a commit that references this issue
on Jul 3, 2024 Nightly build was promoted from the looks of it!
https://r2.nodejs.org/download/nightly/v23.0.0-nightly2024070366b76e24e2/
https://nodejs.org/download/nightly/v23.0.0-nightly2024070366b76e24e2/There is a weird thing with the dates going on in the directory listing, but should be an easy fix. Will take a look later.
Reacted by Trivikram Kamat
For anyone unfamiliar with what we're trying to do with the R2 migration, see #3461
Due to some issues we've experienced with syncing release assets from the DO server to the
dist-prodR2 bucket, we want to look into having adist-stagingR2 bucket that will have the assets before the release is promoted. What we think the issue is is that the DO server simply doesn't have enough resources to handle all of the traffic that comes after a release and handle uploading assets to the prod R2 bucket at the same time. By uploading to a staging bucket before the release is promoted (and thus before cache is purged), we should be able to solve this issue.Note this will not replace uploading the release assets to the DO server. The assets will still be uploaded there like usual.
TODO:
dist-stagingbucket on Node.js's Cloudflare accountdist-stagingbucket as soon as they're uploaded to the DO serverdist-stagingtodist-prodinstead of from the dist folder in the DO server.cc @MoLow @ovflowd