Support Bearer Authorisation for `http` based object stores

[CommanderStorm]

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

It would be great to support Bearer Authorization for e.g. PMTiles URLs.

Databricks Files API requires setting Bearer headers.
Adding the ability to specify the header would allow to load a PMTiles file from Unity Catalog Volumes.

Describe the solution you'd like

I think the simplest solution would be to allow configuring the following for http stores:

arrow-rs-object-store/src/client/mod.rs

Line 486 in 95b0a94

pub fn with_default_headers(mut self, headers: HeaderMap) -> Self {

The difficult part here is mainly the "how do I configure this in the config file" part.
I am not sure if the ideas I have there are good and would like to hear others oppinons first ^^

Describe alternatives you've considered

Alternatively, this functionality could be made public api:

arrow-rs-object-store/src/client/builder.rs

Lines 154 to 155 in 95b0a94

	#[cfg(feature = "gcp")]
	pub(crate) fn bearer_auth(mut self, token: &str) -> Self {

Not sure what is the best approach.

Additional context

This issue was originally raised by @danielsparing in maplibre/martin#2353

[tustvold]

https://docs.rs/object_store/latest/object_store/http/struct.HttpBuilder.html#method.with_client_options

https://docs.rs/object_store/latest/object_store/client/struct.ClientOptions.html#method.with_default_headers

Perhaps

[CommanderStorm]

🤔 True, that likely works indeed.

Should this be configurable via ClientOptions::with_config?
If yes, what would you prefer as a scheme for this?