Skip to content

Google Cloud Storage (GCS) output plugin #9022

Description

@epsteina16

Is your feature request related to a problem? Please describe.
We would like to write logs from Fluent Bit to GCS from applications running in Google's GKE. The log volume we are working is substantial and we don't require advanced searching/indexing on these logs. GCS is the ideal storage for us in this situation.

This plugin would be very similar to the S3 output plugin and the Azure Blob output plugin.
A GCS output plugin has come up many times in previous issues:
#6636
#8525
https://stackoverflow.com/questions/72467592/fluentbit-to-gcs

Describe the solution you'd like
A new output plugin intended for uploading objects to GCS, that has similar feature parity to the Azure Blob and S3 output plugins.

Describe alternatives you've considered
Although GCS is S3 compatible, the default credentials chain is different. The S3 plugin fetches credentials via: https://github.com/fluent/fluent-bit-docs/blob/43c4fe134611da471e706b0edb2f9acd7cdfdbc3/administration/aws-credentials.md#aws-credentials while GCS has https://cloud.google.com/docs/authentication/application-default-credentials

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions