Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reduce Cloud Storage / Container Registry Spending #2157

Open
4 tasks
afrittoli opened this issue Sep 6, 2024 · 2 comments
Open
4 tasks

Reduce Cloud Storage / Container Registry Spending #2157

afrittoli opened this issue Sep 6, 2024 · 2 comments
Assignees
@afrittoli

Comments

@afrittoli
Copy link
Member

afrittoli commented Sep 6, 2024 •
edited
Loading

Steps to adopt ghcr.io

  • Make the list of regions configurable
  • Setup auth for ghcr.io
  • Add logic to link container images to projects
  • Update runtime resource with image registry values for ghcr.io
@afrittoli afrittoli mentioned this issue Sep 6, 2024
Open
@afrittoli afrittoli self-assigned this Sep 9, 2024
@afrittoli
Copy link
Member Author

afrittoli commented Sep 9, 2024 •
edited
Loading

This will have to be done for various Tekton projects.
The set relevant for the tekton-releases project is:

For tekton-nightly there are more projects that publish images to gcr.io, however, the egress bandwidth for those is not significant (< 10% of tekton-releases).

This was referenced Sep 23, 2024
Merged
Merged
afrittoli added a commit to afrittoli/triggers that referenced this issue Sep 26, 2024
@afrittoli
Migrate nightly builds and releases to ghcr.io
05fea3f 
As part of the work to reduce Tekton's infra spend, we are migrating
nightly builds and new releases from gcr.io to ghcr.io, to reduce the
expensive egress bandwith utilisation
(tektoncd/plumbing#2157).

This PR introduces support for publishing container images to ghcr.io
(YAML files are still published to Google Cloud storage). It also
replicates a few improvements that have been done in the pipeline
release pipeline before, such us:

- Support for hashing the image path (for nicer path in GitHub)
- Support for skipping build and test tasks, to be enabled for nightly
  builds to save CPU time

Signed-off-by: Andrea Frittoli <andrea.frittoli@uk.ibm.com>
@afrittoli afrittoli mentioned this issue Sep 26, 2024
Merged
7 tasks
afrittoli added a commit to afrittoli/triggers that referenced this issue Sep 26, 2024
@afrittoli
Migrate nightly builds and releases to ghcr.io
e311691 
As part of the work to reduce Tekton's infra spend, we are migrating
nightly builds and new releases from gcr.io to ghcr.io, to reduce the
expensive egress bandwith utilisation
(tektoncd/plumbing#2157).

This PR introduces support for publishing container images to ghcr.io
(YAML files are still published to Google Cloud storage). It also
replicates a few improvements that have been done in the pipeline
release pipeline before, such us:

- Support for hashing the image path (for nicer path in GitHub)
- Support for skipping build and test tasks, to be enabled for nightly
  builds to save CPU time

Signed-off-by: Andrea Frittoli <andrea.frittoli@uk.ibm.com>
tekton-robot pushed a commit to tektoncd/triggers that referenced this issue Sep 26, 2024
@afrittoli @tekton-robot
Migrate nightly builds and releases to ghcr.io
96723b3 
As part of the work to reduce Tekton's infra spend, we are migrating
nightly builds and new releases from gcr.io to ghcr.io, to reduce the
expensive egress bandwith utilisation
(tektoncd/plumbing#2157).

This PR introduces support for publishing container images to ghcr.io
(YAML files are still published to Google Cloud storage). It also
replicates a few improvements that have been done in the pipeline
release pipeline before, such us:

- Support for hashing the image path (for nicer path in GitHub)
- Support for skipping build and test tasks, to be enabled for nightly
  builds to save CPU time

Signed-off-by: Andrea Frittoli <andrea.frittoli@uk.ibm.com>
@afrittoli
Copy link
Member Author

Migrate catalog bundles to ghcr.io: #2220

This was referenced Oct 2, 2024
Open
Closed
Open
Open
Open
afrittoli added a commit to afrittoli/tektoncd-chains that referenced this issue Oct 2, 2024
@afrittoli
Prepare release pipeline for ghcr.io
a5e8847 
To reduce CI spending, we're in the process of moving all new
release to ghcr.io - tektoncd/plumbing#2157

This prepares the release pipeline for that switch (which will
require a PR in the plumbing repo).

This PR also aligns the pipeline with some other small improvements
already adopted by other projects:
- use remote resolution for tasks
- only run build and tests when requested - it will be disabled by
  default on nightly builds

Signed-off-by: Andrea Frittoli <andrea.frittoli@gmail.com>
@afrittoli afrittoli mentioned this issue Oct 2, 2024
Merged
6 tasks
tekton-robot pushed a commit to tektoncd/chains that referenced this issue Oct 2, 2024
@afrittoli
Prepare release pipeline for ghcr.io (#1220)
6bcfe64 
To reduce CI spending, we're in the process of moving all new
release to ghcr.io - tektoncd/plumbing#2157

This prepares the release pipeline for that switch (which will
require a PR in the plumbing repo).

This PR also aligns the pipeline with some other small improvements
already adopted by other projects:
- use remote resolution for tasks
- only run build and tests when requested - it will be disabled by
  default on nightly builds

Signed-off-by: Andrea Frittoli <andrea.frittoli@gmail.com>
afrittoli added a commit to afrittoli/plumbing that referenced this issue Nov 12, 2024
@afrittoli
Switch CI/CD jobs to ghcr.io images
976badd 
Switch all CI/CD jobs from the plumbing images gcr.io/tekton-releases/dogfooding
to the plumbing images ghcr.io/tektoncd/plumbing.

Images have been copied over with crane, until the publish
template are updated. This PR has been generated with regex
search and replace.

Part of tektoncd#2157

Signed-off-by: Andrea Frittoli <andrea.frittoli@uk.ibm.com>
@tekton-robot tekton-robot mentioned this issue Nov 12, 2024
Closed
2 tasks
@afrittoli afrittoli mentioned this issue Nov 12, 2024
Merged
2 tasks
tekton-robot pushed a commit that referenced this issue Nov 12, 2024
@afrittoli @tekton-robot
Switch CI/CD jobs to ghcr.io images
890270b 
Switch all CI/CD jobs from the plumbing images gcr.io/tekton-releases/dogfooding
to the plumbing images ghcr.io/tektoncd/plumbing.

Images have been copied over with crane, until the publish
template are updated. This PR has been generated with regex
search and replace.

Part of #2157

Signed-off-by: Andrea Frittoli <andrea.frittoli@uk.ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@afrittoli afrittoli

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@afrittoli