From f5517b234175c53d953bbc0dc0d268255d8411a8 Mon Sep 17 00:00:00 2001
From: Yiyang Huang
+
+ Fig. 4 Workflow of Pushing an Artifact using the General Processor
-
Fig. 4 Design of HTTPProcessor
+Fig. 5 Design of HTTPProcessor
For a remote processor, the functions defined in ```Processor``` interface can be abstract to HTTP service API. By using these APIs, Harbor Core can communicate with remote HTTP processor extender. @@ -252,11 +404,11 @@ func (h *HTTPProcessor) AbstractAddition(ctx context.Context, artifact *artifact } ``` -The workflow of pushing a custom artifact to Harbor with the help of `HTTPProcessor` is shown in Fig. 5. The Harbor works as a proxy to the registry when the user uploads the content layers and config layer. Harbor ensures that the repository exists. Then Harbor puts the manifest to the registry. After that, Harbor will check if the artifact by digest exists. In this step, Harbor will use `Processor.AbstractMetatda` in `Abstractor` to abstract the metadata ant keep in the artifact.Artifact model. +The workflow of pushing a custom artifact to Harbor with the help of `HTTPProcessor` is shown in Fig. 6. The Harbor works as a proxy to the registry when the user uploads the content layers and config layer. Harbor ensures that the repository exists. Then Harbor puts the manifest to the registry. After that, Harbor will check if the artifact by digest exists. In this step, Harbor will use `Processor.AbstractMetatda` in `Abstractor` to abstract the metadata ant keep in the artifact.Artifact model.-
Fig. 5 Workflow of Pushing an Artifact using the HTTPProcessor
+Fig. 6 Workflow of Pushing an Artifact using the HTTPProcessor
When `HTTPProcessor.AbstractMetadata(ctx context.Context, manifest []byte, artifact *artifact.Artifact) error)` is invoked, it will send a HTTP POST request to the processor extender: @@ -277,11 +429,11 @@ POST {remote-processor-endpoint}/abstractmetadata The `manifest` and `artifact` will be sent to the extender to abstract the metadata. If the registry needs Auth, `registry` will be sent, just like Scanner. -When users deal with these built-in artifact types like OCI Image, CNAB or Helm Chart, the workflow **is not affected**, like Fig. 6. For example, the user uploads a Helm Chart to Harbor, we will use Helm Chart processor to abstract the metadata. Thus the design of `HTTPProcessor` is non-invasive to the current design. +When users deal with these built-in artifact types like OCI Image, CNAB or Helm Chart, the workflow **is not affected**, like Fig. 7. For example, the user uploads a Helm Chart to Harbor, we will use Helm Chart processor to abstract the metadata. Thus the design of `HTTPProcessor` is non-invasive to the current design.-
Fig. 6 Workflow of Pushing an Artifact using the Build-in Processor
+Fig. 7 Workflow of Pushing an Artifact using the Build-in Processor
#### Processor Extender @@ -415,7 +567,7 @@ It is reasonable to use the same way for processor extender using bearer tokens Refer to scan job using credentials generated by robot account mechanism, we can use the same way to use the robot account mechanism to generate credentials that work with these common OCI/Docker tooling libraries to provide credentialed access to the image data. The lifecycle of the robot account credentials can be bound to the HTTP request. For every HTTP request call remote processor API, a robot account expired at certain time will be created.Additionally, a modification is needed to ensure that the generated credentials have access to bypass the configured policy checks on the image that normal users are subject to if those checks are configured. -## Development Process +### Development Process There are totally three things we need to do to complete the proposal: @@ -425,16 +577,16 @@ There are totally three things we need to do to complete the proposal: Thus we propose to have three iterations. Each of them is self-contained and supposed to be merged into Harbor Core. -### First Iteration: HTTPProcessor and Extender without Auth +#### First Iteration: HTTPProcessor and Extender without Auth At this stage, user defined processor will not register to harbor. So if users want to use processor extender, they still need to hard-code some logic to harbor code to register the type with the corresponding processor manually. Also, ```HTTPProcessor``` will make HTTP request to the extender without providing authentication and Harbor external endpoint. So users need to do some work to generate authentication using other user account. Harbor external endpoint should be configured any way. And policy check interceptor can not be bypassed. -### Second Iteration: Registration +#### Second Iteration: Registration At the second stage, registration logic will be added. Users don't need to modify harbor code any more. A remote processor configuration file is required to register specific processor to harbor. When harbor core starts, it will read the configuration file and register the processor to harbor. -### Third Iteration: Auth in the Processor Extender +#### Third Iteration: Auth in the Processor Extender At the final stage, using robot account mechanism to generate credentials will be finished. Harbor external endpoint and authentication will be passed directly in HTTP POST request body. Users don't need to worry about the authentication problem. But still need to find a way to use authentication properly. diff --git a/proposals/images/artifact-processor-extender/in-tree-workflow.png b/proposals/images/artifact-processor-extender/in-tree-workflow.png new file mode 100644 index 0000000000000000000000000000000000000000..34a6c05ee824331ca3a1e4a7d77a57d684c65104 GIT binary patch literal 78686 zcmeFZWmJ^i9yd%XAT=NY(ybyb-7SJ3h=g=^NewOCAW}*rA|Oah4&60$cQ*_n9na=I z_c`Z2Yd!B;?}zu}yVj^{jn~ZV+1G#n;};Y9Mo|_A>oFD*5)zKwODPp3B$O%eCkx{q zc!sVeR{#kKS=vHU@{OFNB=wsQcBU5ACP+w6VvO|l3FMwX=`=9V*YE6Oev0+MRV5@O zLPg)Fxv{+wPTfr1n34QNTYG^7e*sms87WJ-snQBNg8HJAwX?u%dWQ7_?BUPxs>Aiz zMed@!FI&n;>aQ#vI6vK>wpw(dOEX|&M?4j)_~0go95Rb+9R3iQ0^PVDxn!7T2Z!2) zAixW)fRVnFw6h-B_yN5cJxZUxC$jN2Jxyp1T}kGNA5wI5tzWSGCyA=$;0MU-pK)KQ zFuxKjck~uFQ8sm#3H766k5eHMI{N&_>0tzR1U3nYm;tWx_4z%%=cH|X_=f0)*w~M_ zKKkK73=hOatrLy(7xpDX@{q5Y7up)_?e)YBm1$>?Bjz>2h9e@Wt@n1Zu?vQ8Zg0uq zjg5s}jfm5W+uMf9+uK`LQdHCjLQ0WnB#b6vvB0M2m;m$8ovez
zy+xoc( kP}*}||Y`QTaOpl294My)&4
zm&?Sa_5zcVF|RQ;)*Cu^-B%YrFLY~6i_f@F6n?1dOs{=^tP>X&Wy=lOLl@WZ*K+oY
z_q%#VwYYo^|4Cj@(<;n@Z?#8t+7@FL;bf(LxqKd3w)N)6ws{%WD{9-fO`kn%BQ5u=
z3LGe) d8;U=)}3Z5K7{sd+YqcN4@FU^rblbky!sd7ou!%^hy#7r8KRAsEL&Ssbqm
zjla*)j2BY{#)#VcZ222AKm}(TNG`=r7{ZmR8koC$qdxMIxH+=2Z4XGgY$LtZk``Z=
zEOzx`sBC>=rcK^<*_kNWt3#4b6&4VGzKXej$S$Nk)ggRCsERD5>~lK>*N
zHi_?xkTsipZ6Mh@3=0pGc{rq4F&MfKd#L%<_X^Vk3ObFMo>Ig~Nvb`G5MsGK(vkKU
z93zKC%7Fe)LEMX=qp;bYt}LoMXeZlyU99jp>#oaBcLu?n(G6pG#6j_Bp*#{sY9jW1
zzkFD+V)IAxqwknLG%5EWTcl~Ob8bP1HEDzL^dq+}hQulelGJk(nv~5?_&lwjzd^lP
zdSD!!2P?5kHN>ug+eECqSE&LKl)_WqdS|tNHYQu*#m(rttKe|{b?ZA_vvxB6)<_~8
zO&h97LPO*9p3hF(EQMJ@bHkXUyP3a*q%MQ5sizM=%xPlCDBL30ho8icGnV~FuKRim
zj{xnar#bNzW)l{hK{DR%JF+FrhkJV+R5`-`8X)su-c8-UAvUXXFT*>C?VivI?g+0g
zMogHHq2NZ-PvfUD;IJS@O{+Ot8Bqp3#39
_N`Swn&
zT|>B_)bzhEia{f|#5$2mv*CV^_9~|F4*I2&jVfnKgErq>Y_n-)4L){hY(#xkf@Z?-
z_uZ*}4c1coE?=zzK!W+;U)yx?QC+m|uZ_T1wtHtmO17=(0%u=6Up4RZZvgOn-!8in
zi!{&wD;RuU@lw{)dVimk$@GdykIxm#0gjMx*6~3jHqc
zF79j8Y}D}r3ncV2Ct(3tAo=mxqaR++otfIzkkvp&_o$}D*Cg?d;RYZQy^RugK8 3v!U
0QN!VDs(l}d#+372e
z@8d+mPMz!BDxaS#zfsS8?l-O