Skip to content

Conversation

@weliang1
Copy link

@weliang1 weliang1 commented Dec 5, 2025

Summary

OpenShift networking teams frequently need to troubleshoot Multus CNI issues across different cluster configurations, but analyzing must-gather data manually is time-consuming and error-prone. This plugin addresses this by providing automated analysis of Multus infrastructure, NetworkAttachmentDefinitions (NADs), and multi-networked pod configurations.

This PR introduces the must-gather:multus command, a comprehensive tool for analyzing Multus CNI configuration and multi-networked pods from must-gather data without manual inspection.

Use Cases

This plugin can be used for troubleshooting and post-mortem analysis of Multus networking:

  • Post-incident analysis: Quickly analyze must-gather data from failed CI jobs or production incidents
  • Configuration auditing: Review NetworkAttachmentDefinitions and multi-networked pod configurations across namespaces
  • Health checks: Verify Multus infrastructure (DaemonSets, Deployments, Pods) is running correctly
  • Documentation: Generate comprehensive HTML reports for sharing with team members or support cases
  • Prow CI integration: Analyze must-gather data directly from Prow CI job artifacts

Key Features

  • Fast Multus Analysis: Analyze complete Multus configuration in 10-60 seconds
  • Comprehensive Coverage: Cluster info, health, Multus infrastructure, NADs, and multi-networked pods
  • Rich HTML Reports: Interactive reports with tabs, visual indicators, and professional styling
  • Failure Detection: Automatic identification of issues with severity levels and recommendations
  • Multiple Formats: HTML summary report + text-based failure analysis
  • Framework Support: Works with OVN and SDN networking, handles both tarball and extracted must-gather

Commands Added

/must-gather:multus <must-gather-file-path> <MULTUS_ANALYSIS_REPORT-path>

Plugin Structure

  • Commands: Main command for Multus analysis (commands/multus.md)
  • Skills: Python analyzer in skills/must-gather-analyzer/scripts/
  • Analysis Script: analyze_multus.py - comprehensive Multus data parser
  • Report Generator: HTML template with embedded CSS and JavaScript
  • Failure Analyzer: Intelligent issue detection and prioritization

What's Included

  • Command definition following man page format
  • Python analyzer for Multus infrastructure and NADs
  • HTML report generator with professional styling
  • Failure analysis with recommendations
  • NetworkAttachmentDefinition parser
  • Multi-networked pod detector
  • Whereabouts IPPool analyzer
  • Comprehensive README with examples

Analysis Capabilities

The analyzer examines:

  • Cluster Information: Version, platform, network type, node status
  • Cluster Health: Operator status and overall health assessment
  • Multus Infrastructure: DaemonSets, Deployments, pod status
  • CNI Configuration: Config files, binaries, Multus daemon settings
  • NetworkAttachmentDefinitions: Cluster-scoped and namespace-scoped NADs
  • Multi-Networked Pods: Pods with k8s.v1.cni.cncf.io/networks annotation
  • Network Topology: Visual representation of network architecture

Files Changed

  • 2 files added/modified
  • ~1,500 lines of new code
  • Full Multus analysis framework

Generated Reports Examples

https://file.corp.redhat.com/~weliang/MULTUS_ANALYSIS_REPORT1.html
https://file.corp.redhat.com/~weliang/MULTUS_ANALYSIS_REPORT2.html
https://file.corp.redhat.com/~weliang/MULTUS_ANALYSIS_REPORT3.html
https://file.corp.redhat.com/~weliang/MULTUS_ANALYSIS_REPORT4.html
https://file.corp.redhat.com/~weliang/MULTUS_ANALYSIS_REPORT5.html

Summary by CodeRabbit

Release Notes

  • New Features

    • Added /must-gather:multus command to analyze Multus CNI configuration and multi-networked pods from must-gather data.
  • Documentation

    • Reorganized Must-Gather Plugin documentation with command-based usage guide and examples.
    • Added comprehensive documentation for the new Multus command including usage patterns and output details.

✏️ Tip: You can customize this high-level summary in your review settings.

@openshift-ci openshift-ci bot requested review from bentito and dgoodwin December 5, 2025 20:26
@openshift-ci
Copy link

openshift-ci bot commented Dec 5, 2025

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: weliang1
Once this PR has been reviewed and has the lgtm label, please assign enxebre for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Dec 5, 2025

Walkthrough

Restructures the Must-Gather plugin docs from script-focused to command-centric usage, adding a new /must-gather:multus command, updating README content and examples, and adding command metadata entries for the multus analysis command.

Changes

Cohort / File(s) Change Summary
Plugin README restructuring
plugins/must-gather/README.md
Renamed header to "Must-Gather Plugin", replaced script-oriented docs with a command-based usage model, introduced commands /must-gather:analyze, /must-gather:ovn-dbs, /must-gather:multus, refocused outputs on consolidated HTML reports and failure analysis, reorganized prerequisites/skills/use-cases, and updated installation/contribution/support references to the central ai-helpers repo.
New command documentation
plugins/must-gather/commands/multus.md
Added detailed documentation for the multus command: synopsis, arguments (must-gather-path, output-html-path), analysis scope (cluster info, cluster health, Multus infra, NADs, multi-networked pods, topology), outputs (HTML report, failure analysis text), prerequisites, implementation steps, examples, return values, and error-handling notes.
Plugin registry / docs metadata
PLUGINS.md, docs/data.json
Registered the new /must-gather:multus [must-gather-path] [output-html-path] command in plugin index and UI metadata (docs/data.json) with description and argument hint; preserved existing analyze/ovn-dbs entries.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

  • Focus areas for review:
    • Accuracy and consistency of command signatures between README.md, commands/multus.md, PLUGINS.md, and docs/data.json.
    • Clarity and correctness of prerequisites, example invocations, and described outputs in commands/multus.md.
    • Links and references updated to point to the ai-helpers repository.

Pre-merge checks and finishing touches

❌ Failed checks (1 inconclusive)
Check name Status Explanation Resolution
No Assumed Git Remote Names ❓ Inconclusive Shell scripts could not be executed due to repository cloning failures, preventing direct code inspection. PR summary does not indicate hardcoded git remote operations. Repository access needed for direct file inspection to search for hardcoded git remote names like 'git push origin' or 'git fetch upstream'.
✅ Passed checks (6 passed)
Check name Status Explanation
Title check ✅ Passed The title 'must-gather: Add multus CNI analyzer' directly and concisely summarizes the main change: adding a Multus CNI analyzer to the must-gather plugin.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
No Real People Names In Style References ✅ Passed PR documentation for Multus CNI analyzer contains no references to real people's names in style references or example prompts.
Git Push Safety Rules ✅ Passed Pull request contains only documentation and data structure updates with no git push operations or git commands executed.
No Untrusted Mcp Servers ✅ Passed No MCP server installations, npm packages, or untrusted dependencies detected in PR changes.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (3)
plugins/must-gather/commands/multus.md (2)

10-10: Specify language identifiers for all code blocks.

Multiple fenced code blocks are missing language specifiers, which impacts syntax highlighting and readability. All the following blocks should specify their language:

  • Line 10: bash (Synopsis example)
  • Line 14: bash (Usage example)
  • Line 42: bash (Script path)
  • Line 56: bash (Script discovery)
  • Line 122: text (Output structure)
  • Line 131, 137, 143, 149: bash (Examples)
  • Line 157, 164, 170, 177: text (Error messages)

Apply this diff to fix the code blocks:

  ## Synopsis
  • bash
    /must-gather:multus [must-gather-path] [output-html-path]

For all other code blocks, add the appropriate language identifier (`bash`, `text`, `yaml`, etc.) on the first line after the opening triple backticks.



Also applies to: 14-14, 42-42, 56-56, 122-122, 131-131, 137-137, 143-143, 149-149, 157-157, 164-164, 170-170, 177-177

---

`50-106`: **Reframe Implementation section to clarify whether steps are user-facing or internal.**

The "Implementation" section mixes two distinct concerns: (1) how the command itself locates and invokes the script (lines 55–62), and (2) what the Python analyzer script does internally (lines 69–106). This blurs the distinction between command orchestration and script internals.

For user-facing documentation, consider restructuring to emphasize what the user observes, or split into "Command Flow" (external behavior) and "Analysis Steps" (internal script logic) subsections. Alternatively, if these are instructions for command authors, move them to a separate implementation guide or inline comments.

</blockquote></details>
<details>
<summary>plugins/must-gather/README.md (1)</summary><blockquote>

`14-14`: **Specify language identifiers for all code blocks.**

Multiple fenced code blocks are missing language specifiers. Add language identifiers to lines 14, 22, 36, 77, 82, 87, and 101:
- Line 14, 22, 36: `bash` (command examples)
- Line 77, 82, 87: `bash` (usage examples)
- Line 101: `text` (failure analysis output)



Apply this diff to fix the code blocks:

```diff
  **Usage:**
  • bash
    /must-gather:ovn-dbs [must-gather-path] [--node ] [--query ]

Apply the same pattern to all other code blocks throughout the file.



Also applies to: 22-22, 36-36, 77-77, 82-82, 87-87, 101-101

</blockquote></details>

</blockquote></details>

<details>
<summary>📜 Review details</summary>

**Configuration used**: Path: .coderabbit.yaml

**Review profile**: CHILL

**Plan**: Pro

**Cache: Disabled due to data retention organization setting**

**Knowledge base: Disabled due to data retention organization setting**

<details>
<summary>📥 Commits</summary>

Reviewing files that changed from the base of the PR and between edabeae251c1c255112f7a60c5554c32d7dd3cb3 and 1fc94f17a4c085b04f0bc407346ec1b9050ccaf6.

</details>

<details>
<summary>📒 Files selected for processing (2)</summary>

* `plugins/must-gather/README.md` (1 hunks)
* `plugins/must-gather/commands/multus.md` (1 hunks)

</details>

<details>
<summary>🧰 Additional context used</summary>

<details>
<summary>🪛 markdownlint-cli2 (0.18.1)</summary>

<details>
<summary>plugins/must-gather/commands/multus.md</summary>

10-10: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

---

42-42: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

---

122-122: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

---

131-131: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

---

137-137: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

---

143-143: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

---

149-149: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

---

157-157: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

---

164-164: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

---

170-170: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

---

177-177: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

</details>
<details>
<summary>plugins/must-gather/README.md</summary>

14-14: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

---

22-22: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

---

36-36: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

---

77-77: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

---

82-82: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

---

87-87: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

---

101-101: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

</details>

</details>

</details>

<details>
<summary>🔇 Additional comments (1)</summary><blockquote>

<details>
<summary>plugins/must-gather/README.md (1)</summary><blockquote>

`18-38`: **Verify consistency between README and multus.md command documentation.**

The command documentation in multus.md (the dedicated command file) provides extensive detail on behavior, implementation steps, error handling, and use cases. Ensure that any cross-references or examples in the README align with the detailed command documentation, especially around:
- Output file naming and structure (HTML report + `.failure-analysis.txt` suffix)
- Expected prerequisites and error messages
- Usage patterns and examples

</blockquote></details>

</blockquote></details>

</details>

<!-- This is an auto-generated comment by CodeRabbit for review status -->

@weliang1
Copy link
Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant