Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Remote Asset API #112

Merged
merged 1 commit into from
Jan 27, 2020
Merged

Add Remote Asset API #112

merged 1 commit into from
Jan 27, 2020

Conversation

sstriker
Copy link
Collaborator

Based on the design document
and updates in bazelbuild/proposals#160.

@googlebot googlebot added the cla: yes Pull requests whose authors are covered by a CLA with Google. label Jan 19, 2020
@sstriker
Copy link
Collaborator Author

@jmillikin-stripe @EricBurnett please have a look. Alternatively we fold it all in the remote_execution.proto.

The final proto is based on the [Fetch API design document](https://docs.google.com/document/d/10ari9WtTTSv9bqB_UU-oe2gBtaAA7HyQgkpP-RFP80c)
and updates in bazelbuild/proposals#160.

* build/bazel/remote/asset/v1/BUILD
  Generate the various language outputs.
* build/bazel/remote/asset/v1/qualifiers.md
  Registry for well known qualifiers.
* build/bazel/remote/asset/v1/remote_asset.pb.go
  Generated proto output for Go.
* build/bazel/remote/asset/v1/remote_asset.proto
  The actual Remote Asset API
* hooks/pre-commit
  Hook in the build for the new API.

Co-Authored-By: Eric Burnett <ericburnett@gmail.com>
Co-authored-by: John Millikin <jmillikin@stripe.com>
@googlebot
Copy link

All (the pull request submitter and all commit authors) CLAs are signed, but one or more commits were authored or co-authored by someone other than the pull request submitter.

We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that by leaving a comment that contains only @googlebot I consent. in this pull request.

Note to project maintainer: There may be cases where the author cannot leave a comment, or the comment is not properly detected as consent. In those cases, you can manually confirm consent of the commit author(s), and set the cla label to yes (if enabled on your project).

ℹ️ Googlers: Go here for more info.

@googlebot googlebot added cla: no Pull requests whose authors are not covered by a CLA with Google. and removed cla: yes Pull requests whose authors are covered by a CLA with Google. labels Jan 23, 2020
@sstriker sstriker changed the title [WIP] Add Remote Fetch API. Add Remote Asset API Jan 24, 2020
@sstriker sstriker marked this pull request as ready for review January 24, 2020 19:48
@EricBurnett
Copy link
Collaborator

@googlebot I consent

@jmillikin-stripe
Copy link
Contributor

@googlebot I consent

@buchgr
Copy link
Contributor

buchgr commented Jan 27, 2020

Huge congrats @jmillikin-stripe @sstriker @EricBurnett for shipping the API!

@buchgr buchgr merged commit 2846a67 into bazelbuild:master Jan 27, 2020
jmillikin-stripe added a commit to jmillikin-stripe/bazel that referenced this pull request Jan 28, 2020
@jmillikin-stripe
Copy link
Contributor

FYI: While implementing this in Bazel I discovered that some of the quotes in the comment blocks have been replaced with Unicode versions. This makes ASCII-only consumers, including the Bazel CI runners, very unhappy.

I'll fix this for Bazel in the initial vendor. Can someone with a commit bit fix it in this repo, so other clients don't hit the same problem?

Example error (from https://storage.googleapis.com/bazel-untrusted-buildkite-artifacts/beb93ad4-5b4f-43c0-925c-2a6a2b6d73d7/src/test/shell/bazel/bazel_bootstrap_distfile_test/attempt_1.log):

derived/src/java/build/bazel/remote/asset/v1/PushGrpc.java:168: error: unmappable character for encoding ASCII
     * and only return resources that can be Fetch???d from origin.

jmillikin-stripe added a commit to jmillikin-stripe/bazel that referenced this pull request Jan 31, 2020
jmillikin-stripe added a commit to jmillikin-stripe/bazel that referenced this pull request Feb 19, 2020
bazel-io pushed a commit to bazelbuild/bazel that referenced this pull request Feb 26, 2020
Closes #10818.

Signed-off-by: Philipp Wollermann <philwo@google.com>
@philwo
Copy link
Member

philwo commented Mar 9, 2020

@jmillikin-stripe Sure, I can merge a PR that fixes that.

bazel-io pushed a commit to bazelbuild/bazel that referenced this pull request Mar 10, 2020
This is the Bazel client implementation of bazelbuild/proposals#160. It allows downloading of external dependencies to be delegated to a remote service.

TODOs:
- [x] Once bazelbuild/remote-apis#112 is merged, the vendored copy of `bazelbuild/remote-apis` should be updated. I've used a [WIP] placeholder for now.
- [x] If the general approach looks reasonable then I'll add tests. Currently I've been testing with an in-house implementation of the downloader server.

R: @buchgr @dslomov
CC: @EricBurnett @sstriker @ulfjack

Closes #10622.

PiperOrigin-RevId: 300116716
katre pushed a commit to bazelbuild/bazel that referenced this pull request Mar 10, 2020
This is the Bazel client implementation of bazelbuild/proposals#160. It allows downloading of external dependencies to be delegated to a remote service.

TODOs:
- [x] Once bazelbuild/remote-apis#112 is merged, the vendored copy of `bazelbuild/remote-apis` should be updated. I've used a [WIP] placeholder for now.
- [x] If the general approach looks reasonable then I'll add tests. Currently I've been testing with an in-house implementation of the downloader server.

R: @buchgr @dslomov
CC: @EricBurnett @sstriker @ulfjack

Closes #10622.

PiperOrigin-RevId: 300116716
@sstriker sstriker deleted the remote-fetch-api branch March 13, 2020 08:47
santigl pushed a commit to santigl/remote-apis that referenced this pull request Aug 26, 2020
Copy link

@Sec32fun32 Sec32fun32 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#'nin kopyası

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla: no Pull requests whose authors are not covered by a CLA with Google.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants