DOC-1608 auto topic creation in cloud #418
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
✅ Deploy Preview for rp-cloud ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Contributor
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the 📝 WalkthroughWalkthroughDocumentation updates across three pages:
Sequence Diagram(s)sequenceDiagram
autonumber
actor Producer as Client Producer
participant Kafka as Redpanda Cloud Kafka API
participant Controller as Cluster Controller
participant Storage as Log Storage
Producer->>Kafka: Produce to Topic T
Kafka->>Kafka: Check metadata for T
alt Topic T exists
Kafka->>Storage: Append message to T
Storage-->>Kafka: Ack append
Kafka-->>Producer: Produce ack
else Topic T missing
Kafka->>Controller: Request auto-create Topic T
Controller-->>Kafka: Topic T created
Kafka->>Storage: Append message to T
Storage-->>Kafka: Ack append
Kafka-->>Producer: Produce ack
end
note over Kafka,Controller: Automatic topic creation enabled in Redpanda Cloud
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Possibly related PRs
Suggested reviewers
Pre-merge checks and finishing touches❌ Failed checks (2 warnings)
✅ Passed checks (3 passed)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (4)
modules/get-started/pages/whats-new-cloud.adoc (2)
19-22: Clarify scope, rollout, and limits for auto topic creation.State whether this applies to Serverless, Dedicated, and BYOC, and note the rollout behavior for existing clusters. Also reference the topic count clamp and defaults so users understand constraints and how to tune them.
=== Automatic topic creation - -Automatic topic creation is now enabled in Redpanda Cloud, so the Kafka service can automatically create topics when a message is produced to a topic that doesn't exist. +Automatic topic creation is now enabled in Redpanda Cloud. When a client produces to a non‑existent topic, the topic is created automatically. + +This applies to Serverless, Dedicated, and BYOC clusters. New clusters have this behavior immediately; existing clusters gain it as they are upgraded to the latest version. + +Auto‑created topics use cluster defaults (for example, partitions and replication factor). See xref:reference:properties/cluster-properties.adoc[Cluster properties] for defaults and how to adjust them. Cloud‑level safeguards such as a topic count clamp may limit creation; contact Support if you need higher limits.
21-21: Use “Kafka API” for consistency.Elsewhere we use “Kafka API” rather than “Kafka service.”
-Automatic topic creation is now enabled in Redpanda Cloud, so the Kafka service can automatically create topics when a message is produced to a topic that doesn't exist. +Automatic topic creation is now enabled in Redpanda Cloud, so the Kafka API can automatically create topics when a message is produced to a topic that does not exist.modules/get-started/pages/cloud-overview.adoc (2)
338-338: Mention rollout nuance to set expectations.Small addition helps align with what’s-new and avoids implying instantaneous enablement everywhere.
-Because Redpanda Cloud is a fully-managed service that provides maintenance, data and partition balancing, upgrades, and recovery, much of the cluster maintenance required for Self-Managed users is not necessary for Redpanda Cloud users. New clusters in Redpanda Cloud generally include functionality added in Self-Managed versions immediately. Existing clusters include new functionality when they get upgraded to the latest version. +Because Redpanda Cloud is a fully-managed service that provides maintenance, data and partition balancing, upgrades, and recovery, much of the cluster maintenance required for Self-Managed users is not necessary for Redpanda Cloud users. New clusters in Redpanda Cloud generally include functionality added in Self-Managed versions immediately. Existing clusters include new functionality when they get upgraded to the latest version as part of normal maintenance windows.
338-340: Cross-link to the announcement for discoverability.Light internal link improves navigation from overview to details.
Existing clusters include new functionality when they get upgraded to the latest version as part of normal maintenance windows. +For recent changes, see xref:get-started:whats-new-cloud.adoc[What’s New in Redpanda Cloud] (for example, automatic topic creation).
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
Disabled knowledge base sources:
- Jira integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (3)
modules/get-started/pages/cloud-overview.adoc(1 hunks)modules/get-started/pages/cluster-types/serverless.adoc(0 hunks)modules/get-started/pages/whats-new-cloud.adoc(1 hunks)
💤 Files with no reviewable changes (1)
- modules/get-started/pages/cluster-types/serverless.adoc
🧰 Additional context used
🧠 Learnings (3)
📓 Common learnings
Learnt from: micheleRP
PR: redpanda-data/cloud-docs#267
File: modules/manage/pages/maintenance.adoc:91-92
Timestamp: 2025-04-25T01:41:57.162Z
Learning: The notification timeline for Redpanda Cloud deprecations has been deliberately removed from the documentation, even though the PR summary mentioned a 180-day advance notice period.
Learnt from: micheleRP
PR: redpanda-data/cloud-docs#267
File: modules/manage/pages/maintenance.adoc:63-64
Timestamp: 2025-04-25T01:42:09.318Z
Learning: The timeline for major upgrade notifications (180 days in advance) was intentionally removed from the Redpanda Cloud maintenance documentation, even though it was mentioned in the PR objectives.
📚 Learning: 2025-04-25T01:41:57.162Z
Learnt from: micheleRP
PR: redpanda-data/cloud-docs#267
File: modules/manage/pages/maintenance.adoc:91-92
Timestamp: 2025-04-25T01:41:57.162Z
Learning: The notification timeline for Redpanda Cloud deprecations has been deliberately removed from the documentation, even though the PR summary mentioned a 180-day advance notice period.
Applied to files:
modules/get-started/pages/cloud-overview.adoc
📚 Learning: 2025-04-25T01:42:09.318Z
Learnt from: micheleRP
PR: redpanda-data/cloud-docs#267
File: modules/manage/pages/maintenance.adoc:63-64
Timestamp: 2025-04-25T01:42:09.318Z
Learning: The timeline for major upgrade notifications (180 days in advance) was intentionally removed from the Redpanda Cloud maintenance documentation, even though it was mentioned in the PR objectives.
Applied to files:
modules/get-started/pages/cloud-overview.adoc
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: Redirect rules - rp-cloud
- GitHub Check: Header rules - rp-cloud
- GitHub Check: Pages changed - rp-cloud
🔇 Additional comments (1)
modules/get-started/pages/whats-new-cloud.adoc (1)
19-22: Repo-wide sanity check for stale "auto topic creation is disabled" warningsAutomated search failed — run this locally and paste any matches:
git grep -n -i -E 'auto(-|[[:space:]])?create(ion)?([[:space:]])?topic(s)?.*(disable|disabled|not (enabled|supported|available|allowed))|automatic topic creation.*(disable|disabled|not (enabled|supported|available|allowed))' || true
4 tasks
mattschumpert
approved these changes
Sep 19, 2025
Contributor
Author
|
@travisdowns would you please confirm that this topic limit is now enforced for all clusters? |
Member
|
@micheleRP it is enforced for all 25.2 and later version clusters, but not 25.1 or earlier. There should be very few 25.1 clusters around, but that would be a quesiton for the cloud team, asked here: https://redpandadata.slack.com/archives/C0191NM4WU9/p1758640209193899 |
micheleRP
deleted the
DOC-1608-Document-Cloud-Feature-Finish-Rollout-topic-count-clamp-and-auto-topic-creation-in-cloud
branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This pull request updates documentation to reflect that automatic topic creation is now enabled in Redpanda Cloud.
modules/get-started/pages/whats-new-cloud.adocannouncing that you can enable automatic topic creation for BYOC and Dedicated clusters. All clusters now have a 40,000 topic limitmodules/get-started/pages/cloud-overview.adocabout automatic topic creationauto_create_topics_enabledcluster property to referenceSingle sourcing done in redpanda-data/docs#1361
Resolves https://redpandadata.atlassian.net/browse/DOC-1608
Review deadline:
Page previews
What's New
Cluster Configuration Properties
Cloud Overview
Create a Topic
Manage Topics
Checks