Skip to content

docs: fix Helm chart V2 GCS storage configuration #68638

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
+122 −24
Open

docs: fix Helm chart V2 GCS storage configuration #68638

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
09ebba0
docs: fix Helm chart V2 GCS storage configuration
devin-ai-integration[bot] Oct 24, 2025
0cb42a1
docs: use 3-tab interface for storage configuration and fix V1 GCS
devin-ai-integration[bot] Oct 24, 2025
3e63021
docs: add secretName back for S3 and Azure storage types
devin-ai-integration[bot] Oct 24, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 57 additions & 10 deletions docs/platform/deploying-airbyte/chart-v2-community.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -224,40 +224,87 @@ Add external log storage (if applicable)

If you're using external log storage, implement it. If you aren't, skip this step.

<Tabs groupId="storage-type">
<TabItem value='s3' label='AWS S3' default>

```yaml
global:
storage:
secretName: ""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this not used for anything anymore in s3/azure? I'm thinking at this point these docs are too confusing. It might be better to create a 3-tab interface with AWS, GCS, and Azure configurations defined separately. It seems like they're different enough that this is warranted. You should apply the same treatment to the enterprise migration guide for chart V2 too.

type: minio # default storage is minio. Set to s3, gcs, or azure, according to what you use.

type: s3
secretName: airbyte-config-secrets # Name of your Kubernetes secret
bucket:
log: airbyte-bucket
state: airbyte-bucket
workloadOutput: airbyte-bucket
activityPayload: airbyte-bucket

# Set ONE OF the following storage types, according to your specification above

# S3
s3:
region: "" ## e.g. us-east-1
authenticationType: credentials ## Use "credentials" or "instanceProfile"
accessKeyId: ""
secretAccessKey: ""
```

</TabItem>
<TabItem value='gcs' label='Google Cloud Storage'>

**Option 1: Provide credentials directly in values.yaml**

# GCS
```yaml
global:
storage:
type: gcs
bucket:
log: airbyte-bucket
state: airbyte-bucket
workloadOutput: airbyte-bucket
activityPayload: airbyte-bucket
gcs:
projectId: <project-id>
credentialsJson: <base64-encoded>
credentialsJson: <raw-json-content> # Provide the raw JSON service account key (not base64-encoded)
credentialsJsonPath: /secrets/gcs-log-creds/gcp.json
```

**Option 2: Use an external Kubernetes secret**

# Azure
If you've already created a Kubernetes secret with your GCS credentials:

```yaml
global:
storage:
type: gcs
storageSecretName: <your-existing-secret-name> # Name of your existing Kubernetes secret
bucket:
log: airbyte-bucket
state: airbyte-bucket
workloadOutput: airbyte-bucket
activityPayload: airbyte-bucket
gcs:
projectId: <project-id>
credentialsJsonPath: /secrets/gcs-log-creds/gcp.json
```

</TabItem>
<TabItem value='azure' label='Azure Blob Storage'>

```yaml
global:
storage:
type: azure
secretName: airbyte-config-secrets # Name of your Kubernetes secret
bucket:
log: airbyte-bucket
state: airbyte-bucket
workloadOutput: airbyte-bucket
activityPayload: airbyte-bucket
azure:
# one of the following: connectionString, connectionStringSecretKey
connectionString: <azure storage connection string>
connectionStringSecretKey: <secret coordinate containing an existing connection-string secret>
```

</TabItem>
</Tabs>

</details>

<details>
Expand Down
4 changes: 2 additions & 2 deletions docs/platform/deploying-airbyte/integrations/storage.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,7 @@ Ensure you've already created a Kubernetes secret containing the credentials blo
global:
storage:
type: "GCS"
secretName: airbyte-config-secrets
storageSecretName: airbyte-config-secrets # Use storageSecretName (not secretName) for GCS external secrets
bucket: # GCS bucket names that you've created. We recommend storing the following all in one bucket.
log: airbyte-bucket
state: airbyte-bucket
Expand All @@ -153,7 +153,7 @@ global:
global:
storage:
type: "GCS"
secretName: airbyte-config-secrets
storageSecretName: airbyte-config-secrets # Use storageSecretName (not secretName) for GCS external secrets
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume this should also be applied to the helm chart V1 sample to resolve #48842

bucket: # GCS bucket names that you've created. We recommend storing the following all in one bucket.
log: airbyte-bucket
state: airbyte-bucket
Expand Down
67 changes: 57 additions & 10 deletions docs/platform/enterprise-setup/chart-v2-enterprise.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -236,40 +236,87 @@ postgresql:
Add external log storage
</summary>

<Tabs groupId="storage-type">
<TabItem value='s3' label='AWS S3' default>

```yaml title="values.yaml"
global:
storage:
secretName: ""
type: minio # default storage is minio. Set to s3, gcs, or azure, according to what you use.

type: s3
secretName: airbyte-config-secrets # Name of your Kubernetes secret
bucket:
log: airbyte-bucket
state: airbyte-bucket
workloadOutput: airbyte-bucket
activityPayload: airbyte-bucket

# Set ONE OF the following storage types, according to your specification above

# S3
s3:
region: "" ## e.g. us-east-1
authenticationType: credentials ## Use "credentials" or "instanceProfile"
accessKeyId: ""
secretAccessKey: ""
```

</TabItem>
<TabItem value='gcs' label='Google Cloud Storage'>

**Option 1: Provide credentials directly in values.yaml**

# GCS
```yaml title="values.yaml"
global:
storage:
type: gcs
bucket:
log: airbyte-bucket
state: airbyte-bucket
workloadOutput: airbyte-bucket
activityPayload: airbyte-bucket
gcs:
projectId: <project-id>
credentialsJson: <base64-encoded>
credentialsJson: <raw-json-content> # Provide the raw JSON service account key (not base64-encoded)
credentialsJsonPath: /secrets/gcs-log-creds/gcp.json
```

**Option 2: Use an external Kubernetes secret**

# Azure
If you've already created a Kubernetes secret with your GCS credentials:

```yaml title="values.yaml"
global:
storage:
type: gcs
storageSecretName: <your-existing-secret-name> # Name of your existing Kubernetes secret
bucket:
log: airbyte-bucket
state: airbyte-bucket
workloadOutput: airbyte-bucket
activityPayload: airbyte-bucket
gcs:
projectId: <project-id>
credentialsJsonPath: /secrets/gcs-log-creds/gcp.json
```

</TabItem>
<TabItem value='azure' label='Azure Blob Storage'>

```yaml title="values.yaml"
global:
storage:
type: azure
secretName: airbyte-config-secrets # Name of your Kubernetes secret
bucket:
log: airbyte-bucket
state: airbyte-bucket
workloadOutput: airbyte-bucket
activityPayload: airbyte-bucket
azure:
# one of the following: connectionString, connectionStringSecretKey
connectionString: <azure storage connection string>
connectionStringSecretKey: <secret coordinate containing an existing connection-string secret>
```

</TabItem>
</Tabs>

</details>

<details>
Expand Down
8 changes: 6 additions & 2 deletions docs/platform/enterprise-setup/implementation-guide.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -558,17 +558,21 @@ global:
```yaml title="values.yaml"
global:
storage:
secretName: ""
type: minio # default storage is minio. Set to s3, gcs, or azure, according to what you use.
bucket:
log: airbyte-bucket
state: airbyte-bucket
workloadOutput: airbyte-bucket
activityPayload: airbyte-bucket

# Option 1: Provide credentials directly in values.yaml (credentials will be stored in Kubernetes secret)
gcs:
projectId: <project-id>
credentialsJson: <base64-encoded>
credentialsJson: <raw-json-content> # Provide the raw JSON service account key (not base64-encoded)
credentialsJsonPath: /secrets/gcs-log-creds/gcp.json

# Option 2: Use an external Kubernetes secret you've already created
# storageSecretName: <your-existing-secret-name> # Uncomment and set this to use your own secret
```

</TabItem>
Expand Down
Loading