-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Request: Add google cloud storage source #7501
Comments
Is there anything on the roadmap for this source? |
Not yet, but we have been experimenting with OpenDAL, which was recently used to add a WebHDFS sink, and does have support for GCS. It could be an avenue to experiment with if anyone wants to take a shot at this. |
@jszwedko Would you accept a PR that implements this in roughly the same way that the |
Hey! Yes, I think that would make sense as the initial implementation to match the behavior of the |
Hi, @jszwedko. I'm willing to help implement the GCS source, but I might not have time to complete the full documentation. Do you think it's a good idea to start the implementation first? For example, all content under |
Hey! That'd be great! I think starting with the implementation makes sense. We can help with the docs if you get stuck. |
That's really appreciated. I will find some time next week to get started. |
As a replacement for logstash, we will need vector to support google cloud storage as a source similar to AWS S3 (https://vector.dev/docs/reference/configuration/sources/aws_s3/)
We do something similar in logstash
The text was updated successfully, but these errors were encountered: