Restructure the config properties to support automation #1485
Conversation
✅ Deploy Preview for redpanda-docs-preview ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the 📝 WalkthroughWalkthroughThis PR implements an automated property documentation generation system for Redpanda. It introduces a new documentation guide and GitHub Actions workflow that generates AsciiDoc partials for property documentation using a doc-tools CLI instead of maintaining inline definitions. The workflow triggers on manual dispatch or repository dispatch events, validates semantic versioning, and creates pull requests with updated partials. Existing property documentation pages (broker, topic, object-storage) are refactored to include auto-generated partials filtered by category rather than containing inline property definitions. Configuration files are updated to reference new documentation sources, and new partial files are created to contain auto-generated property definitions and mappings. Sequence DiagramsequenceDiagram
actor Manual/Dispatch as Manual Dispatch or<br/>Repository Dispatch
participant Workflow as update-property-docs.yaml
participant AWS as AWS Secrets Manager
participant CLI as doc-tools CLI
participant Repo as Repository
participant PR as Pull Request
Manual/Dispatch->>Workflow: Trigger (tag input)
Workflow->>AWS: Retrieve GITHUB_TOKEN secret
Workflow->>Repo: Checkout main branch
Workflow->>Workflow: Normalize & compare tag<br/>vs latest-redpanda-tag
alt Tag is newer
Workflow->>CLI: Run: generate property-docs<br/>(tag, partials, cloud-support,<br/>overrides)
CLI->>Repo: Generate/update partials<br/>(docs-data/property-overrides.json)
Workflow->>PR: Create PR<br/>(branch: update-property-docs-${tag})
PR-->>Repo: Commit with generated content
else Tag is not newer
Workflow->>Workflow: Skip doc generation<br/>(print notice)
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60–90 minutes
Possibly related PRs
Suggested reviewers
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🧹 Nitpick comments (2)
.github/copilot-instructions.md (1)
57-74: Add language identifiers to fenced code blocks for proper syntax highlighting.Several fenced code blocks are missing language specifiers. For example:
- Lines 57-74: ASCII diagram should be
```textor```ascii- Lines 84-88: Plaintext should be
```textor left with```for plain- Lines 475-507: Multiple JSON and shell blocks need appropriate language tags
Examples of corrections:
-``` +```text Source Code Metadata | v doc-tools CLI-``` +```json { "properties": { "property_name": {This improves readability and enables syntax highlighting in documentation renderers.
Also applies to: 84-88, 475-507
.github/workflows/update-property-docs.yaml (1)
62-83: Version comparison logic is sound for stable releases but may not handle pre-releases correctly.The semantic version comparison using
sort -Vis appropriate for typical version tags. However, be aware that pre-release versions (e.g.,v1.0.0-rc1,v1.0.0-alpha) may not be compared correctly with this approach. If your versioning scheme includes pre-releases, consider using a dedicated semver comparison tool.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
Disabled knowledge base sources:
- Jira integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (10)
.github/copilot-instructions.md(1 hunks).github/workflows/update-property-docs.yaml(1 hunks)antora.yml(1 hunks)local-antora-playbook.yml(1 hunks)modules/reference/pages/properties/broker-properties.adoc(4 hunks)modules/reference/pages/properties/object-storage-properties.adoc(1 hunks)modules/reference/pages/properties/topic-properties.adoc(1 hunks)modules/reference/partials/deprecated/deprecated-properties.adoc(1 hunks)modules/reference/partials/properties/topic-properties.adoc(1 hunks)modules/reference/partials/properties/topic-property-mappings.adoc(1 hunks)
🧰 Additional context used
🧠 Learnings (2)
📚 Learning: 2025-07-16T19:33:20.420Z
Learnt from: Feediver1
Repo: redpanda-data/docs PR: 1153
File: modules/reference/pages/properties/topic-properties.adoc:45-50
Timestamp: 2025-07-16T19:33:20.420Z
Learning: In the Redpanda documentation, topic property cross-references like <<max.compaction.lag.ms>> and <<min.compaction.lag.ms>> require corresponding property definition sections with anchors like [[maxcompactionlagms]] and [[mincompactionlagms]] to prevent broken links.
Applied to files:
modules/reference/partials/properties/topic-property-mappings.adocmodules/reference/pages/properties/object-storage-properties.adoc.github/copilot-instructions.mdmodules/reference/partials/properties/topic-properties.adocmodules/reference/pages/properties/topic-properties.adocmodules/reference/partials/deprecated/deprecated-properties.adocmodules/reference/pages/properties/broker-properties.adoc
📚 Learning: 2025-07-14T19:28:43.296Z
Learnt from: Feediver1
Repo: redpanda-data/docs PR: 1153
File: antora.yml:3-5
Timestamp: 2025-07-14T19:28:43.296Z
Learning: In Redpanda docs, during beta releases, the version metadata may intentionally show inconsistencies where the header displays the beta version (e.g., 25.2 Beta) while internal attributes like full-version, latest-redpanda-tag, operator-beta-tag still reference the stable version (e.g., 25.1). This is resolved during the GA merge process when all version references are synchronized.
Applied to files:
antora.yml
🪛 markdownlint-cli2 (0.18.1)
.github/copilot-instructions.md
57-57: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
84-84: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
475-475: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
482-482: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
497-497: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
504-504: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
690-690: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: Redirect rules - redpanda-docs-preview
- GitHub Check: Header rules - redpanda-docs-preview
- GitHub Check: Pages changed - redpanda-docs-preview
🔇 Additional comments (16)
local-antora-playbook.yml (1)
20-20: Verify feature branch will be merged to main before release.Line 20 references
config-automationbranch, which appears to be a feature branch. Ensure this branch is merged tomainbefore the docs site is published, or this configuration will not find the updated content.Similarly, the new
docs-sitesource (lines 26-28) points tomainand should be verified to exist and contain the expected structure.Also applies to: 26-28
antora.yml (1)
22-22: Verify console version downgrade: 3.30.1 → v3.3.1.The console tag was changed from
3.30.1tov3.3.1. While the format change (addingvprefix) aligns with the PR's goal of consistency, the version number itself appears to have decreased numerically (3.30.1 → 3.3.1).Confirm this version change is intentional and represents the correct latest console version that should be documented.
modules/reference/partials/deprecated/deprecated-properties.adoc (1)
1-114: Auto-generated deprecated properties partial looks well-structured.The file has appropriate auto-generated headers, clear categorization by property scope, and straightforward list formatting. The metadata at the top correctly documents that this is auto-generated content.
modules/reference/pages/properties/object-storage-properties.adoc (1)
15-15: Verify object-storage-properties partial exists and has expected structure.Line 15 includes
reference:partial$properties/object-storage-properties.adocwith filters for deprecated and excluded properties. Ensure:
- The partial file exists at
modules/reference/partials/properties/object-storage-properties.adoc- The partial uses the same tagging structure as the topic-properties partial (tag::category-*, tag::deprecated[], tag::exclude-from-docs[])
- The partial is auto-generated and documented in copilot-instructions.md
.github/copilot-instructions.md (1)
1-100: Guide clearly establishes critical rules for LLM property documentation updates.The document provides comprehensive guidance on the override system, with strong emphasis on:
- Never editing generated files in
/modules/reference/partials/properties/- Using only
/docs-data/property-overrides.jsonfor manual edits- Clear requirements for doc-tools CLI invocation with all required flags
- Mandatory inclusion of
--cloud-supportflagThe examples are practical and cover common scenarios. The validation section and troubleshooting guide add value for users implementing the system.
modules/reference/pages/properties/broker-properties.adoc (2)
89-89: Verify Schema Registry API port change: 8082 → 8081.The example on line 89 shows Schema Registry API port changed from
8082to8081. This is a significant change that affects:
- User-facing documentation examples
- Documentation accuracy for deployments
- Consistency with actual Redpanda defaults
Confirm this port change:
- Reflects the actual current default in Redpanda
- Is not a breaking change from previous versions
- Is intentional and not a documentation error
47-47: Include directives for property partials follow consistent pattern.Lines 47, 97, and 131 use consistent include directives with appropriate category tags and filtering:
include::reference:partial$properties/broker-properties.adoc[tags=category-redpanda,!deprecated,!exclude-from-docs]This pattern enables:
- Modular content organization by category
- Automatic exclusion of deprecated properties
- Hiding internal/exclude-from-docs properties
- Future updates without touching main property pages
Also applies to: 97-97, 131-131
.github/workflows/update-property-docs.yaml (6)
23-27: Verify GitHub Organization variables and secrets are properly configured.The AWS role ARN is constructed from GitHub variables and secrets. Ensure that
RP_AWS_CRED_REGION,RP_AWS_CRED_ACCOUNT_ID, andRP_AWS_CRED_BASE_ROLE_NAMEare properly configured in your GitHub Organization settings to prevent workflow failures.
36-40: Checkout configuration is appropriate.Using a separate bot token for authentication is a security best practice. The main branch checkout is correct for this automation workflow.
42-48: Node.js and dependency setup is appropriate.Using Node.js v20 and
npm cifollows CI/CD best practices for reproducible builds.
50-60: Tag determination logic is correct and defensive.The step properly prioritizes workflow inputs and falls back to repository dispatch payload, with explicit error handling for missing tags.
85-96: Doc generation step is properly configured with error handling.The
set -euo pipefailensures the step fails on any error, and the doc-tools invocation passes all necessary arguments including tag, generation flags, and overrides file path.
98-108: PR creation step is well-configured with appropriate metadata.The pull request is created with clear commit messages, titles, and labels for easy identification and automation. The branch naming includes the tag for traceability.
modules/reference/pages/properties/topic-properties.adoc (3)
13-15: Topic property mappings include is appropriately placed.The new section with the
topic-property-mappings.adocpartial is well-organized and placed before property categories for logical flow. Ensure the referenced partial file exists and contains valid xref anchors matching any cross-references in the documentation.
18-51: Category-based filtering approach is well-structured and maintainable.The consistent use of category-based tags with negative filters for deprecated and excluded content is an effective pattern for managing auto-generated documentation. Each section clearly indicates which properties it covers, making the structure easy to understand and maintain.
15-15: Verify that auto-generated partials exist with correct structure and anchors.The topic-properties page now depends on two auto-generated partial files:
modules/reference/partials/properties/topic-property-mappings.adoc— for topic-to-cluster property mappingsmodules/reference/partials/properties/topic-properties.adoc— for categorized topic properties with tagsPer the learning note, ensure that property xref cross-references (e.g.,
<<max.compaction.lag.ms>>) have corresponding anchors in the partial (e.g.,[[maxcompactionlagms]]) to prevent broken links. Also verify that all required category tags are applied in the properties partial.Also applies to: 22-22, 28-28, 34-34, 40-40, 46-46
Description
This pull request introduces a new automated workflow for generating Redpanda property documentation and updates several configuration files and documentation partials to support improved property doc generation and mapping.
Page previews
Checks