Add Azure Blob Storage source

[spencergilbert]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Use Cases

Similar use-case to our s3 and gcs sources, and the mirror of the azure_blob_storage sink we have today.

Attempted Solutions

N/A

Proposal

Implement azure_blob_storage source akin to gcs and s3 sources.

References

No response

Version

master@ 461130

[kwapik]

I'd like to work on this issue together with @ldrozdz93.

[Xuanwo]

cc @jszwedko, Similar to what I mentioned in #7501 (comment), I'm willing to start an implementation for azblob based on opendal.

After these implementations, I'm eager to add documentation on how to add more sources based on opendal and work on improving existing sources to better align with others.

Maybe we could establish a tracking issue for those tasks. Do you think it's a good direction?

[ldrozdz93]

FYI: me and @kwapik implemented the azblob source on our client's fork of v0.38 and it's been working on prod for the last few months. Soon I'll start porting that to vector upstream.

[Xuanwo]

FYI: me and @kwapik implemented the azblob source on our client's fork of v0.38 and it's been working on prod for the last few months. Soon I'll start porting that to vector upstream.

That will be great. Looking forward to future collaboration, we can build on your work.