Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add api to delete blob and manifest #12006

Merged

Conversation

wy65701436
Copy link
Contributor

Enable the capability of registry controller to delete blob and manifest

Signed-off-by: wang yan wangyan@vmware.com

@codecov
Copy link

codecov bot commented May 22, 2020

Codecov Report

Merging #12006 into master will decrease coverage by 0.02%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #12006      +/-   ##
==========================================
- Coverage   55.79%   55.77%   -0.03%     
==========================================
  Files         915      917       +2     
  Lines       53108    53234     +126     
  Branches     1797     1797              
==========================================
+ Hits        29633    29690      +57     
- Misses      20174    20230      +56     
- Partials     3301     3314      +13     
Flag Coverage Δ
#unittests 55.77% <ø> (-0.03%) ⬇️
Impacted Files Coverage Δ
...m/goharbor/harbor/src/registryctl/config/config.go 61.53% <0.00%> (-11.19%) ⬇️
....com/goharbor/harbor/src/common/utils/passports.go 86.36% <0.00%> (-4.55%) ⬇️
...ig/vulnerability/vulnerability-config.component.ts 47.77% <0.00%> (-4.46%) ⬇️
...thub.com/goharbor/harbor/src/server/error/error.go 82.35% <0.00%> (ø)
...goharbor/harbor/src/registryctl/handlers/router.go 100.00% <0.00%> (ø)
...oharbor/harbor/src/registryctl/handlers/handler.go 100.00% <0.00%> (ø)
...om/goharbor/harbor/src/registryctl/api/registry.go
...r/harbor/src/registryctl/api/registry/blob/blob.go 52.63% <0.00%> (ø)
.../src/registryctl/api/registry/manifest/manifest.go 46.42% <0.00%> (ø)
...arbor/harbor/src/registryctl/api/registry/gc/gc.go 12.00% <0.00%> (ø)
... and 5 more

@wy65701436 wy65701436 force-pushed the non-blocking-gc-api-blob-delete branch from 50e4333 to f02e21a Compare May 22, 2020 08:11
src/registryctl/main.go Outdated Show resolved Hide resolved
@wy65701436 wy65701436 force-pushed the non-blocking-gc-api-blob-delete branch from f02e21a to 3703768 Compare May 22, 2020 08:36
@wy65701436
Copy link
Contributor Author

#12024

@wy65701436 wy65701436 force-pushed the non-blocking-gc-api-blob-delete branch 4 times, most recently from 27140ff to 83b2195 Compare June 2, 2020 09:41
Enable the capability of registry controller to delete blob and manifest

Signed-off-by: wang yan <wangyan@vmware.com>
@wy65701436 wy65701436 force-pushed the non-blocking-gc-api-blob-delete branch from 83b2195 to c9fa405 Compare June 2, 2020 09:46
src/go.mod Outdated
@@ -83,3 +87,5 @@ require (
k8s.io/client-go v0.17.3
k8s.io/helm v2.16.3+incompatible
)

replace github.com/Azure/go-autorest => github.com/Azure/go-autorest v13.3.3+incompatible
Copy link
Contributor

@reasonerjt reasonerjt Jun 2, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why we need the replace here?
why can't we put the replace statements together?

Copy link
Contributor Author

@wy65701436 wy65701436 Jun 4, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@reasonerjt
Copy link
Contributor

I find there are more files that need to be vendored than I expected.
Have you run go mod tidy?

}
var tags []string
v := r.URL.Query()
queryTags := v.Get("tags")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the tags query is not mentioned in
goharbor/community#133

Would it be better if we write some doc for the rest api? It may or may not be swagger as we don't plan to allow user to write client at least in 2.1.0

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need tags here? Since 2.0, tags may exist in harbor db but not exist in registry.

Copy link
Contributor Author

@wy65701436 wy65701436 Jun 4, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fundamental method needs this pararmeter, and it can be a null. Leave it at here will not impact GC deletes manifest via digest only.
And it's a query string.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel this may reflect a misunderstanding here, I don't think the parm "tags" makes sense for this API...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, I will remove it.

"github.com/goharbor/harbor/src/registryctl/api/registry/gc"
)

// const definition
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the client core used to call registryctl's API right?
We should attach request ID and extract request ID on the server side.
If you don't want to handle it in this PR could you open an issue?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will raise an issue for it, and handle it in another PR.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

@@ -100,4 +101,9 @@ func (c *Configuration) loadEnvs() {
c.LogLevel = loggerLevel
}

registryConf := os.Getenv("REGISTRY_CONFIG")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this necessary?
Is this for making testing easier?

Copy link
Contributor Author

@wy65701436 wy65701436 Jun 4, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's easier for testing &debugging.

)

// StorageDriver ...
var StorageDriver storagedriver.StorageDriver
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not make storagedriver part of the config of registryctl?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The configuraitons of the driver are arleady in the registry configuration file, and the driver is inited on registryctl container launch. I think it's ok to use a varaible here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I strongly think putting in in config is a better design.
I'll let @heww chime in ,if he's OK with this I'm fine

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If putting StorageDriver in the config of the registryctl, it will be clear in the main of the registryctl.

In the Load method of the registryctl conf, we can resolve configuration of the distrubtion and create StorageDriver from the configurtion. The result is that is subpackage config/registry can be removed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added.

@wy65701436 wy65701436 force-pushed the non-blocking-gc-api-blob-delete branch from 09b833f to b237db7 Compare June 4, 2020 09:19
Signed-off-by: wang yan <wangyan@vmware.com>
@wy65701436 wy65701436 force-pushed the non-blocking-gc-api-blob-delete branch from b237db7 to fb3f06c Compare June 4, 2020 10:04
@wy65701436
Copy link
Contributor Author

@reasonerjt @heww code is updated and comments added, can you help to review?

@wy65701436 wy65701436 force-pushed the non-blocking-gc-api-blob-delete branch 2 times, most recently from 5c8d61f to 30b9367 Compare June 5, 2020 09:46
Signed-off-by: wang yan <wangyan@vmware.com>
@wy65701436 wy65701436 force-pushed the non-blocking-gc-api-blob-delete branch from 30b9367 to d9e1b48 Compare June 5, 2020 10:05
_ "github.com/docker/distribution/registry/storage/driver/middleware/redirect"
_ "github.com/docker/distribution/registry/storage/driver/oss"
_ "github.com/docker/distribution/registry/storage/driver/s3-aws"
_ "github.com/docker/distribution/registry/storage/driver/swift"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These drivers can be move to config pkg as the storage driver is created there.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a blank import should be only in a main or test package

Signed-off-by: wang yan <wangyan@vmware.com>
Signed-off-by: wang yan <wangyan@vmware.com>
@wy65701436 wy65701436 force-pushed the non-blocking-gc-api-blob-delete branch from f9380cd to ebecc8b Compare June 5, 2020 16:35
@wy65701436 wy65701436 merged commit dec8397 into goharbor:master Jun 5, 2020
cafeliker pushed a commit to cafeliker/harbor that referenced this pull request Jul 14, 2020
* Add api to delete blob and manifest

Enable the capability of registry controller to delete blob and manifest

Signed-off-by: wang yan <wangyan@vmware.com>
Signed-off-by: Ye Liu <ye.liu@hp.com>
bmfp pushed a commit to bmfp/harbor that referenced this pull request Jul 22, 2020
* Add api to delete blob and manifest

Enable the capability of registry controller to delete blob and manifest

Signed-off-by: wang yan <wangyan@vmware.com>
Signed-off-by: molinber <bertrand.molin@credit-agricole.net>
tedgxt pushed a commit to tedgxt/harbor that referenced this pull request Aug 11, 2020
* Add api to delete blob and manifest

Enable the capability of registry controller to delete blob and manifest

Signed-off-by: wang yan <wangyan@vmware.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants