Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(gc): align the blobs in the artifact trash and useless blobs #20735

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

chlins
Copy link
Member

@chlins chlins commented Jul 12, 2024

During the GC mark stage, blobs corresponding to newly deleted artifacts during the query process are removed to ensure the consistency of data in trash and blobs and prevent dirty data from being left behind.

Thank you for contributing to Harbor!

Comprehensive Summary of your change

Issue being fixed

Fixes #20711

Please indicate you've done the following:

  • Well Written Title and Summary of the PR
  • Label the PR as needed. "release-note/ignore-for-release, release-note/new-feature, release-note/update, release-note/enhancement, release-note/community, release-note/breaking-change, release-note/docs, release-note/infra, release-note/deprecation"
  • Accepted the DCO. Commits without the DCO will delay acceptance.
  • Made sure tests are passing and test coverage is added if needed.
  • Considered the docs impact and opened a new docs issue or PR with docs changes if needed in website repository.

Copy link

codecov bot commented Jul 12, 2024

Codecov Report

Attention: Patch coverage is 50.00000% with 25 lines in your changes missing coverage. Please review.

Project coverage is 66.25%. Comparing base (c8c11b4) to head (c947027).
Report is 276 commits behind head on main.

Files with missing lines Patch % Lines
src/jobservice/job/impl/gc/garbage_collection.go 50.00% 14 Missing and 5 partials ⚠️
src/pkg/artifactrash/dao/dao.go 40.00% 4 Missing and 2 partials ⚠️
Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff             @@
##             main   #20735       +/-   ##
===========================================
+ Coverage   45.36%   66.25%   +20.88%     
===========================================
  Files         244     1045      +801     
  Lines       13333   113551   +100218     
  Branches     2719     2845      +126     
===========================================
+ Hits         6049    75228    +69179     
- Misses       6983    34207    +27224     
- Partials      301     4116     +3815     
Flag Coverage Δ
unittests 66.25% <50.00%> (+20.88%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/pkg/artifactrash/manager.go 100.00% <100.00%> (ø)
src/pkg/artifactrash/dao/dao.go 57.81% <40.00%> (ø)
src/jobservice/job/impl/gc/garbage_collection.go 45.75% <50.00%> (ø)

... and 1284 files with indirect coverage changes

@NehemiahMi
Copy link

如果 在删除 artifact 的时候 touch 一下 artifact 对应的 blob 行,将 blob 行 update 更新,会不会更好一些呢?这样在 GC 时,删除的 artifact 整个 image 内容都会保留,而不会像现在一样,只是保留了 manifest 的数据层。

@chlins chlins force-pushed the fix/gc-trash-blob-align branch 2 times, most recently from aebf178 to 8febfdf Compare July 17, 2024 07:44
During the GC mark stage, blobs corresponding to newly deleted artifacts
during the query process are removed to ensure the consistency of data
in trash and blobs and prevent dirty data from being left behind.

Signed-off-by: chlins <chlins.zhang@gmail.com>
Copy link

This PR is being marked stale due to a period of inactivty. If this PR is still relevant, please comment or remove the stale label. Otherwise, this PR will close in 30 days.

@github-actions github-actions bot added the Stale label Sep 16, 2024
@chlins chlins added never-stale Do not stale and removed Stale labels Sep 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: No status
Development

Successfully merging this pull request may close these issues.

GC data residuals when with large amount of artifact trash
6 participants