Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: implement bandwidth limit for proxy-cache #20812

Merged
merged 1 commit into from
Aug 8, 2024

Conversation

zyyw
Copy link
Contributor

@zyyw zyyw commented Aug 6, 2024

Thank you for contributing to Harbor!

Comprehensive Summary of your change

Issue being fixed

Fixes #20589

Please indicate you've done the following:

  • Well Written Title and Summary of the PR
  • Label the PR as needed. "release-note/ignore-for-release, release-note/new-feature, release-note/update, release-note/enhancement, release-note/community, release-note/breaking-change, release-note/docs, release-note/infra, release-note/deprecation"
  • Accepted the DCO. Commits without the DCO will delay acceptance.
  • Made sure tests are passing and test coverage is added if needed.
  • Considered the docs impact and opened a new docs issue or PR with docs changes if needed in website repository.

Testing

test case 1: proxy_speed_kb = "-1"

        "metadata": {
            "proxy_speed_kb": "-1",
            "public": "false",
            "retention_id": "1"
        },
        "name": "proxy1-unlimited1",
Screenshot 2024-08-06 at 3 13 15 PM

test case 2: proxy_speed_kb = "100"

        "metadata": {
            "proxy_speed_kb": "100",
            "public": "false",
            "retention_id": "2"
        },
        "name": "proxy2-limited-100kb",

100 is in kilo bit per second, 100/8 = 12.5 kilo bytes per second
Screenshot 2024-08-06 at 3 16 50 PM

test case 3: proxy_speed_kb = "300"

        "metadata": {
            "proxy_speed_kb": "300",
            "public": "false",
            "retention_id": "3"
        },
        "name": "proxy3-limited-300kb",

300 is in kilo bit per second, 300/8 = 37.5 kilo bytes per second
Screenshot 2024-08-06 at 3 18 08 PM

test case 4: proxy_speed_kb = "1024"

        "metadata": {
            "proxy_speed_kb": "1024",
            "public": "false",
            "retention_id": "4"
        },
        "name": "proxy4-limited-1024kb",

1024 is in kilo bit per second, 1024/8 = 128 kilo bytes per second
Screenshot 2024-08-06 at 3 23 31 PM

test case 5: proxy_speed_kb = "102400"

        "metadata": {
            "proxy_speed_kb": "102400",
            "public": "false",
            "retention_id": "5"
        },
        "name": "proxy5-limited-102400kb",

1024 is in kilo bit per second, 1024/8 = 12800 kilo bytes per second
Screenshot 2024-08-06 at 3 23 57 PM

Copy link

codecov bot commented Aug 6, 2024

Codecov Report

Attention: Patch coverage is 17.24138% with 48 lines in your changes missing coverage. Please review.

Project coverage is 66.28%. Comparing base (c8c11b4) to head (67df2c9).
Report is 252 commits behind head on main.

Files Patch % Lines
src/controller/proxy/remote.go 0.00% 12 Missing ⚠️
src/pkg/project/models/project.go 0.00% 10 Missing ⚠️
src/server/v2.0/handler/project.go 0.00% 10 Missing ⚠️
src/server/v2.0/handler/project_metadata.go 0.00% 6 Missing ⚠️
src/server/middleware/repoproxy/proxy.go 0.00% 5 Missing ⚠️
.../controller/replication/transfer/image/transfer.go 0.00% 2 Missing ⚠️
src/server/middleware/repoproxy/tag.go 0.00% 2 Missing ⚠️
src/controller/proxy/controller.go 0.00% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff             @@
##             main   #20812       +/-   ##
===========================================
+ Coverage   45.36%   66.28%   +20.91%     
===========================================
  Files         244     1046      +802     
  Lines       13333   114043   +100710     
  Branches     2719     2845      +126     
===========================================
+ Hits         6049    75591    +69542     
- Misses       6983    34334    +27351     
- Partials      301     4118     +3817     
Flag Coverage Δ
unittests 66.28% <17.24%> (+20.91%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
src/controller/proxy/options.go 100.00% <100.00%> (ø)
src/lib/iothrottler.go 0.00% <ø> (ø)
src/controller/proxy/controller.go 14.91% <0.00%> (ø)
.../controller/replication/transfer/image/transfer.go 33.76% <0.00%> (ø)
src/server/middleware/repoproxy/tag.go 0.00% <0.00%> (ø)
src/server/middleware/repoproxy/proxy.go 4.42% <0.00%> (ø)
src/server/v2.0/handler/project_metadata.go 0.00% <0.00%> (ø)
src/pkg/project/models/project.go 40.14% <0.00%> (ø)
src/server/v2.0/handler/project.go 5.15% <0.00%> (ø)
src/controller/proxy/remote.go 0.00% <0.00%> (ø)

... and 1278 files with indirect coverage changes

@zyyw zyyw force-pushed the proxycache-limit-bandwidth branch 4 times, most recently from 8178660 to 74e97ad Compare August 6, 2024 09:00
api/v2.0/swagger.yaml Outdated Show resolved Hide resolved
Copy link
Contributor

@wy65701436 wy65701436 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@zyyw zyyw force-pushed the proxycache-limit-bandwidth branch from 98fe2f3 to 6257806 Compare August 8, 2024 08:03
Signed-off-by: Shengwen Yu <yshengwen@vmware.com>
@zyyw zyyw force-pushed the proxycache-limit-bandwidth branch from bb20fc1 to 67df2c9 Compare August 8, 2024 09:55
Copy link
Member

@chlins chlins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@zyyw zyyw merged commit 1f75b7a into goharbor:main Aug 8, 2024
11 of 12 checks passed
kunal-511 pushed a commit to kunal-511/harbor_local that referenced this pull request Aug 22, 2024
Signed-off-by: Shengwen Yu <yshengwen@vmware.com>
Signed-off-by: kunal-511 <yoyokvunal@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Enhance proxy-cache feature to limit the bandwidth to pull image from upstream
6 participants