Skip to content

fix: resolve Docker Hub rate limit issues in GitHub Actions - #1511

Merged
Andriy Knysh (aknysh) merged 6 commits into
mainfrom
remove-dockerhub
Sep 25, 2025
Merged

Andriy Knysh (aknysh) merged 6 commits into
mainfrom
remove-dockerhub

Conversation

@osterman

@osterman Erik Osterman (Cloud Posse) (osterman) commented Sep 25, 2025 •

Copy link
Copy Markdown
Member

what

  • Replace Docker-based PR Size Labeler with custom GitHub Script action
  • Update LocalStack to use AWS ECR Public registry instead of Docker Hub
  • Add retry logic with exponential backoff for k3s image pulls
  • Configure Docker daemon to use mirror.gcr.io as a registry mirror for k3s tests

why

  • GitHub Actions jobs were failing due to Docker Hub rate limits when pulling images
  • The PR Size Labeler action was using a Docker-based action that required pulling from Docker Hub
  • LocalStack and k3s images were being pulled directly from Docker Hub without any fallback mechanism
  • These failures were blocking CI/CD pipelines and preventing PRs from being properly tested

Solution Details

Custom PR Size Labeler

  • Created .github/actions/pr-sizer/ - a custom GitHub Action using GitHub Script API
  • Eliminates Docker Hub dependency completely
  • Maintains all original functionality (size labels, file ignoring, configurable thresholds)

LocalStack Updates

  • Updated docker-compose.yml to use public.ecr.aws/localstack/localstack:1.4.0
  • Modified GitHub workflow to pull directly from ECR Public
  • AWS ECR Public has no rate limits for anonymous pulls

K3s Resilience

  • Added Docker daemon configuration to use mirror.gcr.io as a registry mirror
  • Implemented retry logic with exponential backoff (30s, 60s, 90s) for image pulls
  • Pre-pulls images before Docker Compose starts to ensure availability

Test Plan

  • PR Size Labeler action correctly labels this PR based on size
  • LocalStack test job successfully pulls image from ECR Public
  • K3s test job successfully pulls images (with retry if needed)
  • All CI checks pass without Docker Hub rate limit errors

references

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Added a configurable PR Size Labeler that auto-labels PRs by size (XS–XL), supports custom thresholds and ignore rules, and can optionally fail on XL.
  • Chores

    • CI workflows updated to use the local PR size labeling action and preserved ignore patterns.
    • Switched LocalStack images to the AWS Public ECR registry in CI and demo.
    • Minor YAML formatting cleanups.

@github-actions github-actions Bot added the size/m Medium size PR label Sep 25, 2025
@mergify

mergify Bot commented Sep 25, 2025

Copy link
Copy Markdown
Contributor

Important

Cloud Posse Engineering Team Review Required

This pull request modifies files that require Cloud Posse's review. Please be patient, and a core maintainer will review your changes.

To expedite this process, reach out to us on Slack in the #pr-reviews channel.

@mergify mergify Bot added the needs-cloudposse Needs Cloud Posse assistance label Sep 25, 2025
- Replace Docker-based PR Size Labeler with custom GitHub Script action
- Update LocalStack to use AWS ECR Public registry (public.ecr.aws/localstack/localstack)
- Add retry logic with exponential backoff for k3s image pulls
- Configure Docker daemon to use mirror.gcr.io for k3s tests

These changes eliminate Docker Hub dependencies where possible and add
resilience for remaining Docker Hub pulls to avoid rate limit failures.

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
Remove unnecessary complexity:
- No need for Docker mirror configuration for k3s
- No need for pre-pull with retry logic
- GitHub Actions has better rate limits than regular users
- Let docker-compose handle pulling directly

Keep only the essential changes:
- Custom PR Size Labeler (no Docker needed)
- LocalStack using AWS ECR Public registry

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Sep 25, 2025 •

Copy link
Copy Markdown
Contributor
📝 Walkthrough

Walkthrough

Adds a local composite GitHub Action (.github/actions/pr-sizer) that computes PR change size, applies size labels (xs/s/m/l/xl) and can fail on XL; updates workflows to use it instead of an external action; switches LocalStack images to public ECR in CI and examples.

Changes

Cohort / File(s) Summary
New local PR sizer action
.github/actions/pr-sizer/action.yml
Introduces a composite action using actions/github-script@v7 to list PR files, apply ignore rules, compute total changes, pick a size label (xs/s/m/l/xl), update PR labels, and optionally fail when XL is chosen.
Workflows: use local PR sizer
.github/workflows/codeql.yml
Replaces codelytv/pr-size-labeler@... with the local action ./.github/actions/pr-sizer in two steps; adjusts YAML scalar style for files_to_ignore and removes explicit fetch-depth/ref usage in those steps.
LocalStack image source switch
.github/workflows/test.yml, examples/demo-localstack/docker-compose.yml
Changes LocalStack image from localstack/localstack:1.4.0 to public.ecr.aws/localstack/localstack:1.4.0; no functional config changes otherwise.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  actor Dev as Developer
  participant GH as GitHub
  participant Runner as Actions Runner
  participant Sizer as PR Size Labeler (composite)
  participant API as GitHub API

  Dev->>GH: Open/Update Pull Request
  GH-->>Runner: Trigger workflow
  Runner->>Sizer: Run composite action
  Sizer->>API: List PR files (paginated)
  Sizer->>Sizer: Apply ignore rules & sum changes
  Sizer->>Sizer: Determine size bucket (XS/S/M/L/XL)
  Sizer->>API: Get current PR labels
  Sizer->>API: Remove old size labels
  Sizer->>API: Add new size label
  alt XL selected and fail_if_xl = true
    Sizer-->>Runner: Exit with failure
  else
    Sizer-->>Runner: Exit success
  end
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

Suggested reviewers

  • aknysh
  • bateller

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check ✅ Passed The title concisely captures the primary goal of the pull request—resolving Docker Hub rate limit issues in GitHub Actions—which aligns directly with the changes to workflows, registry sources, and custom actions. It is clear, specific, and follows conventional commit style without extraneous detail.
Docstring Coverage ✅ Passed No functions found in the changes. Docstring coverage check skipped.
✨ Finishing touches
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch remove-dockerhub

📜 Recent review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Disabled knowledge base sources:

  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between aa3d29a and 16573c1.

📒 Files selected for processing (1)
  • .github/actions/pr-sizer/action.yml (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • .github/actions/pr-sizer/action.yml
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (8)
  • GitHub Check: Build (macos-latest, macos)
  • GitHub Check: Build (ubuntu-latest, linux)
  • GitHub Check: Build (windows-latest, windows)
  • GitHub Check: autofix
  • GitHub Check: Analyze (go)
  • GitHub Check: Lint (golangci)
  • GitHub Check: Run pre-commit hooks
  • GitHub Check: Summary

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

The original action doesn't fail builds for XL PRs, it only labels them.
Remove the unnecessary fail_if_xl parameter and logic to match the
original behavior exactly.

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
@osterman Erik Osterman (Cloud Posse) (osterman) added the no-release Do not create a new release (wait for additional code changes) label Sep 25, 2025
@codecov

codecov Bot commented Sep 25, 2025 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 57.02%. Comparing base (e5a6b31) to head (16573c1).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1511      +/-   ##
==========================================
+ Coverage   56.98%   57.02%   +0.03%     
==========================================
  Files         284      284              
  Lines       30292    30292              
==========================================
+ Hits        17262    17274      +12     
+ Misses      11206    11193      -13     
- Partials     1824     1825       +1     
Flag Coverage Δ
unittests 57.02% <ø> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Disabled knowledge base sources:

  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between e5a6b31 and aa3d29a.

📒 Files selected for processing (4)
  • .github/actions/pr-sizer/action.yml (1 hunks)
  • .github/workflows/codeql.yml (2 hunks)
  • .github/workflows/test.yml (2 hunks)
  • examples/demo-localstack/docker-compose.yml (1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (6)
  • GitHub Check: Build (windows-latest, windows)
  • GitHub Check: Build (macos-latest, macos)
  • GitHub Check: Build (ubuntu-latest, linux)
  • GitHub Check: Lint (golangci)
  • GitHub Check: Analyze (go)
  • GitHub Check: Summary
🔇 Additional comments (3)
examples/demo-localstack/docker-compose.yml (1)

4-4: Looks good.

Same tag, new registry. No objections.

.github/workflows/test.yml (1)

254-254: LocalStack mirror change looks solid.

Pulling from ECR Public sidesteps the Docker Hub quotas and should unblock the job.

.github/workflows/codeql.yml (1)

154-166: All good on the PR sizer switch.

Nice swap to the in-repo action; inputs stay consistent and we avoid the Docker Hub pull entirely. Appreciated.

Comment thread .github/actions/pr-sizer/action.yml Outdated
- Replace single-page fetch with github.paginate() to handle PRs with >100 files
- Add logging to show total file count being processed
- Ensures accurate size calculations for large PRs

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@aknysh
Andriy Knysh (aknysh) merged commit 6b4d40f into main Sep 25, 2025
83 of 85 checks passed
@aknysh
Andriy Knysh (aknysh) deleted the remove-dockerhub branch September 25, 2025 13:45
@mergify mergify Bot removed the needs-cloudposse Needs Cloud Posse assistance label Sep 25, 2025
@github-actions

Copy link
Copy Markdown

These changes were released in v1.191.1-test.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no-release Do not create a new release (wait for additional code changes) size/m Medium size PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants