Add support for crendential helpers to replace default headers

[mentalnote]

Description of the feature request:

Credential helpers can be used to not only provide authentication information but also as a workaround for requests which need to define specific headers to succeed. The headers from the credential helper output are added to an HttpRequest which is already initialised with default values (constructed here).

If a credential helper emits a non-authorisation header with an identifier which matches one already in the default HttpRequest it adds it alongside the existing header matching that identifier, rather than replacing it (here). This means that if any of these default headers elicit an undesirable response, there's no way to replace them.

Would it be possible to have some method for credential helpers to replace headers instead of add them?

Which category does this issue belong to?

Core

What underlying problem are you trying to solve with this feature?

To use a release asset in a private Github repository as a source for a Bazel registry module a request needs to be made to the Github API. As the repository is private this requires setting the Authorization header. In addition to this the Accept header must be set to application/octet-stream to retrieve the actual asset as opposed to JSON with information about the asset.

The default HttpRequest sets Accept: */*, rendering a credential helper setting Accept: application/octet-stream pointless. It seems that the Github API given the choice of being able to return any MIME type it will pick a default of providing the JSON with information about the asset instead of the asset itself.

This means there's currently no means (to my knowledge), of using a private Github asset as a source for a Bazel registry module or any Bazel http repository rules.

Which operating system are you running Bazel on?

Windows/Linux/Mac

What is the output of bazel info release?

release 7.1.1

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse HEAD ?

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

No response

[fmeum]

@Yannic

[Yannic]

I can see cases where someone passes headers to repo_ctx.download() and then passes it again in the credential helper and expects both values to be present, so I don't think headers from the credential helper should always override and erase what could be user-provided headers.

For default headers, I think it would be reasonable to only set them if such a header is not already present (i.e., move setting the default headers later in the pipeline). @tjgq WDYT?

[tjgq]

For default headers, I think it would be reasonable to only set them if such a header is not already present (i.e., move setting the default headers later in the pipeline). @tjgq WDYT?

I think this is a reasonable compromise; I'd like to avoid making add/replace explicit unless it's absolutely necessary.

[mentalnote]

Anything I can do to help move this one forward? Would really help our workflow if this could be fixed.

[Yannic]

I think sending a PR would be great for this if you're up for it :)

[mentalnote]

I think sending a PR would be great for this if you're up for it :)

I need to seek internal approval on this but hopefully I can do it in the new year.