Use cloud build to build images instead #1923
Conversation
| IMAGES_BUILDING=true | ||
| CLOUD_BUILD_COMMON_ARGS=(. --async --format='value(id)' --substitutions=_GCR_BASE=${GCR_IMAGE_BASE_DIR}) | ||
| # Use faster machine because this is CPU intensive | ||
| BUILD_ID_API_SERVER=$(gcloud builds submit ${CLOUD_BUILD_COMMON_ARGS[@]} \ |
There was a problem hiding this comment.
is it possible to combine these steps to a single file?
cloud build seems support parallel execution
waitFor: ['-']
There was a problem hiding this comment.
There was a problem hiding this comment.
Yes, I also tried that.
There are some tradeoffs using waitFor: ['-']:
- [major] slower because cloud build will run all the image builds in one machine, they compete for resources. Time taken is ~9min -> ~12min. Consider that we may need to build more images later. I think current approach is better. (or I can batch 3 image builds and only let api-server use a single job)
- [minor] log will interleave for 4 builds, a little harder to read
There was a problem hiding this comment.
I prefer
- batch the 3 fast image builds
- let api-server image build still be standalone
There was a problem hiding this comment.
Made corresponding changes.
There was a problem hiding this comment.
SG. BTW with this change #1904 I'm not sure if building api-server images still consumes lots of resources. the change removed most of the dependencies such as tensorflow.
There was a problem hiding this comment.
Thanks for the heads up. I will take a look and change if building api-server is faster.
|
/lgtm |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: IronPan The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Switched to use cloud build for building images.
I tested this could reduce image building time from 18m to ~10m.
Also, image building can be happening at the same time as cluster set up. This will save a lot of time.
/assign @IronPan
This change is