new: Support Bazel Remote APIs.

[milesj]

Will start with caching.

[harlequin]

Hey,

I have seen that this feature was not introduced into the 1.29 release.
What are the missing pieces, despite the merge conflicts.

Maybe I can support here.

[milesj]

@harlequin The requests from moon to bazel-remote fail with cryptic http2 errors that I've been unable to figure out.

hyperium/tonic#1949

[harlequin]

As far as I remember HTTP2 implies that you have SSL enabled. Will try to setup an environment for testing.

[milesj]

Yeah I think the problem has to do with the bazel/tonic libraries being http2, while bazel-remote is http1, and the communication between the 2 just isn't working. This area isn't an expertise of mine.

[harlequin]

I have played around with the remote service ...

What I have found so far

1. TLS has to be activated and the endpoint has to include .assume_http2(true)

https://docs.rs/tonic/latest/tonic/transport/channel/struct.ClientTlsConfig.html#method.assume_http2

endpoint = endpoint
            .tls_config(
                ClientTlsConfig::new()
                    .with_enabled_roots()
                    .assume_http2(true)                    
            )
            .unwrap();

With this setting I can connect to the remote service

2. GZIP, ZSTD compression

So with the enabled connection moon tries to upload the cache into the remote service

bazel-remote-service shows

2024/10/15 07:37:48 POST 400 192.168.44.1 /build.bazel.remote.execution.v2.ContentAddressableStorage/BatchUpdateBlobs

But on moon side I am receiving

[crates\remote\src\cache_api.rs:96:13] &error = Status {
    code: Internal,
    message: "protocol error: received message with invalid compression flag: 114 (valid flags are 0 and 1) while receiving response with status: 400 Bad Request",
    metadata: MetadataMap {
        headers: {
            "content-type": "text/plain; charset=utf-8",
            "x-content-type-options": "nosniff",
            "date": "Tue, 15 Oct 2024 07:37:48 GMT",
            "content-length": "126",
            "set-cookie": "4afd39e673bbfe62f6f7f4a9622d53cd=9c659af26f02f4f78a396f6570abab67; path=/; HttpOnly; Secure; SameSite=None",
        },
    },
    source: None,
}
[crates\remote\src\cache_api.rs:97:13] error.code() = Internal
[crates\remote\src\cache_api.rs:98:13] error.metadata() = MetadataMap {
    headers: {
        "content-type": "text/plain; charset=utf-8",
        "x-content-type-options": "nosniff",
        "date": "Tue, 15 Oct 2024 07:37:48 GMT",
        "content-length": "126",
        "set-cookie": "4afd39e673bbfe62f6f7f4a9622d53cd=9c659af26f02f4f78a396f6570abab67; path=/; HttpOnly; Secure; SameSite=None",
    },
}
[crates\remote\src\cache_api.rs:99:13] error.message() = "protocol error: received message with invalid compression flag: 114 (valid flags are 0 and 1) while receiving response with status: 400 Bad Request"
Error:   × Main thread panicked.
  ├─▶ at C:\Users\rustmec\code\contributions\moon\crates\remote\src\cache_api.rs:101:13
  ╰─▶ Status { code: Internal, message: "protocol error: received message with invalid compression flag: 114 (valid flags are 0 and 1) while receiving response with status: 400 Bad Request", metadata: MetadataMap { headers: {"content-type": "text/plain; charset=utf-8", "x-content-type-    
      options": "nosniff", "date": "Tue, 15 Oct 2024 07:37:48 GMT", "content-length": "126", "set-cookie": "4afd39e673bbfe62f6f7f4a9622d53cd=9c659af26f02f4f78a396f6570abab67; path=/; HttpOnly; Secure; SameSite=None"} }, source: None }
  help: set the `RUST_BACKTRACE=1` environment variable to display a backtrace.

So what I have seen bazel-remote supports uncompressed and zstd for communication.
In moon is only gzip available as config.

My best guess by today, that something has to change additional to the compression ...

Hope this helps