-
Notifications
You must be signed in to change notification settings - Fork 4
DOC-1186 Redpanda Connect for BYOVPC on AWS #274
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the 📝 WalkthroughWalkthroughThe documentation was updated to reflect expanded support for Redpanda Connect in BYOVPC clusters, now indicating beta availability on both AWS and GCP. Updates were made across feature availability tables, beta feature announcements, and cluster configuration guides. The AWS BYOVPC cluster creation documentation was revised to include new required AWS resources and updated validation checks. Additionally, a new beta feature announcement for Redpanda Connect was added, and references to platform support were clarified throughout the documentation. Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant Docs
User->>Docs: Reads BYOVPC AWS/GCP cluster creation guide
Docs-->>User: Shows updated resource requirements and validation
User->>Docs: Checks feature availability table
Docs-->>User: Displays Redpanda Connect beta status for AWS and GCP
User->>Docs: Reads What's New section
Docs-->>User: Announces Redpanda Connect beta for BYOVPC AWS/GCP
Suggested reviewers
Poem
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
✅ Deploy Preview for rp-cloud ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
modules/get-started/pages/cluster-types/byoc/aws/vpc-byo-aws.adoc
Outdated
Show resolved
Hide resolved
modules/get-started/pages/cluster-types/byoc/aws/vpc-byo-aws.adoc
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 0
🧹 Nitpick comments (5)
modules/shared/partials/feature-flag-rpcn.adoc (1)
5-5: Clarify unsupported platform
The bullet correctly indicates beta support on AWS and GCP. To avoid ambiguity, consider appending “(not available on Azure)” so readers know Azure isn’t supported here either.modules/develop/partials/availability-message.adoc (1)
5-5: Grammar refinement for parallelism
The phrase “but not on Azure” works, but for consistency with other bullets you might use “and not on Azure” to maintain parallel structure.modules/get-started/pages/cloud-overview.adoc (2)
268-268: Clarify limited availability bullet
The entry “* Redpanda Connect for Dedicated and BYOC (not BYOVPC on Azure)” could read more clearly as “* Redpanda Connect on Dedicated and BYOC clusters (excluding BYOVPC on Azure)”.
280-280: Remove unnecessary comma
In “* Redpanda Connect on Serverless, and for BYOVPC on GCP and AWS”, the comma is redundant. Consider:
“* Redpanda Connect on Serverless and for BYOVPC on GCP and AWS”.modules/get-started/pages/cluster-types/byoc/aws/vpc-byo-aws.adoc (1)
351-353: Remove duplicate profile in validation list
The validation checks listconnectors_node_group_instance_profiletwice. Remove the duplicate and ensure it reads:
connectors_node_group_instance_profile,redpanda_connect_node_group_instance_profile,utility_node_group_instance_profile, andk8s_cluster_role.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (5)
modules/develop/partials/availability-message.adoc(1 hunks)modules/get-started/pages/cloud-overview.adoc(2 hunks)modules/get-started/pages/cluster-types/byoc/aws/vpc-byo-aws.adoc(3 hunks)modules/get-started/pages/whats-new-cloud.adoc(1 hunks)modules/shared/partials/feature-flag-rpcn.adoc(1 hunks)
🔇 Additional comments (2)
modules/get-started/pages/whats-new-cloud.adoc (1)
38-43: Verify existing AWS BYOVPC cluster support
The update notes adding Connect to existing BYOVPC GCP clusters. Please confirm whether existing AWS BYOVPC clusters can also have Redpanda Connect added; if so, update this section for parity.modules/get-started/pages/cluster-types/byoc/aws/vpc-byo-aws.adoc (1)
225-227: Add Redpanda Connect resource entries
The newredpanda_connect_node_group_instance_profileandredpanda_connect_security_groupkeys are correctly inserted. Please verify these names match the Terraform output variables in the cloud-examples repo and that sample Terraform code is updated to provide these ARNs.Also applies to: 237-239
|
@asimms41 Could you change the networks json to the following so quotes are removed? Still going through validating the workflow. |
|
We also need to wrap the contents of the body of then network request with "network" so it should be something like this cat > redpanda-network.json <<EOF
{
"network": {
"name":"sample-redpanda-network",
"resource_group_id": "${REDPANDA_RG_ID}",
"cloud_provider":"CLOUD_PROVIDER_AWS",
"region": "${AWS_REGION}",
"cluster_type":"TYPE_BYOC",
"customer_managed_resources": {
"aws": {
"management_bucket": {
"arn": "${AWS_MANAGEMENT_BUCKET}"
},
"dynamodb_table": {
"arn": "${AWS_DYNAMODB_TABLE}"
},
"private_subnets": {
"arns": ${AWS_PRIVATE_SUBNETS}
},
"vpc": {
"arn": "${AWS_VPC}"
}
}
}
}
}
EOF |
@david-yu - I've added in your updates. |
|
Thanks could you also please make sure we do the following to create the payload for the create cluster API call? The json is not correct with v1 and we should populate the field via env variables from Terraform. |
|
Please also update the byoc.auto.tfvars.json file with the following contents otherwise the cluster will fail. At least one public subnet is required to provision the NAT Gateway for the cluster to connect back to the control plane. |
|
Currently stuck on provisioning a NAT gateway after public subnet is specified. |
|
For API calls please ensure we are using v1 endpoints instead of |
|
Ignore, I didn't realize the docs were also recently updated to use v1. |
|
Found an issue should be |
|
For |
|
Related to
The issue was that we needed to provision an Internet Gateway and specify at least 6 public CIDR ranges to automatically create public subnets. |
|
Hi @david-yu, I think I have updated the PR with your comments except for this last one: Where does this fit in the workflow. Also, is the use of Private Link a known limitation. It's difficult to tell from the Slack thread. |
modules/get-started/pages/cluster-types/byoc/aws/vpc-byo-aws.adoc
Outdated
Show resolved
Hide resolved
modules/get-started/pages/cluster-types/byoc/aws/vpc-byo-aws.adoc
Outdated
Show resolved
Hide resolved
modules/get-started/pages/cluster-types/byoc/aws/vpc-byo-aws.adoc
Outdated
Show resolved
Hide resolved
modules/get-started/pages/cluster-types/byoc/aws/vpc-byo-aws.adoc
Outdated
Show resolved
Hide resolved
modules/get-started/pages/cluster-types/byoc/aws/vpc-byo-aws.adoc
Outdated
Show resolved
Hide resolved
modules/get-started/pages/cluster-types/byoc/aws/vpc-byo-aws.adoc
Outdated
Show resolved
Hide resolved
modules/get-started/pages/cluster-types/byoc/aws/vpc-byo-aws.adoc
Outdated
Show resolved
Hide resolved
|
One more comment: https://github.com/redpanda-data/cloud-docs/pull/274/files#r2080131798 |
modules/get-started/pages/cluster-types/byoc/aws/vpc-byo-aws.adoc
Outdated
Show resolved
Hide resolved
modules/get-started/pages/cluster-types/byoc/aws/vpc-byo-aws.adoc
Outdated
Show resolved
Hide resolved
micheleRP
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added a couple clarifying suggestions, but lgtm
Description
Resolves DOC-1186
Review deadline: 9th May
This pull request updates documentation for Redpanda Connect and BYOVPC clusters, introduces new configuration options for AWS BYOVPC setups, and improves clarity and usability of example configurations. Below is a summary of the most important changes:
Updates to Redpanda Connect Availability
availability-message.adoc,cloud-overview.adoc, andfeature-flag-rpcn.adoc[1] [2] [3] [4].Enhancements to AWS BYOVPC Configuration
Improvements to Example Files and Validation
redpanda-network.jsonandredpanda-cluster.json) to use Terraform output variables for dynamic values, ensuring consistency and reducing manual errors. [1] [2] [3]rpk cloud byoc aws applycommand by including additional IAM instance profile permissions, such asredpanda_connect_node_group_instance_profile.New Documentation Section
Page previews
What's new
Redpanda Cloud Overview
Create a BYOVPC cluster on AWS
Checks
Summary by CodeRabbit