USHIFT-7435 - Repurpose create-bugs to find-regressions; clarify doctor Step 3

[copejon]

Summary

• Renames microshift-ci:create-bugs to microshift-ci:find-regressions to reflect the skill's actual scope (search-only, no ticket creation)
• Removes the --create mode and Jira-write logic from search-bugs.py, reducing the script by ~140 lines
• Rewrites the doctor's Step 3 prose for determinism: specifies the exact file (prs-status.json), field (title), and pattern for rebase source extraction; separates critical outputs from downstream artifacts

Test plan

• Run /microshift-ci:doctor 4.19,4.20,4.21,4.22 end-to-end and verify bug mapping files are produced
• Run /microshift-ci:find-regressions 4.22 standalone and verify outputs match the old create-bugs behavior (minus ticket creation)
• Verify /microshift-ci:fix-test-bugs and /microshift-ci:close-stale-bugs still reference the correct skill name
• Verify doctor-refresh and continue-session cross-references are correct

Made with Cursor

Summary by CodeRabbit

• New Features

  • Added regression analysis that searches existing issue records for likely duplicates and previously resolved regressions.
  • Reports now categorize findings as suggested matches, linked issues, or items to skip.
  • Added regression-focused reports with clearer summaries and HTML report integration.
  • Added command-line help and improved validation for search results.
  • Issues can be reviewed and filed manually from generated reports.

• Documentation

  • Updated CI workflows and guidance to use regression searching instead of automatic issue creation.
  • Added guidance for analyzing performance metrics and correlating failures with source changes.

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: copejon

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [copejon]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Pro Plus

Run ID: fa102dff-ba28-418c-8cb1-c3e8ad62cd08

Walkthrough

This PR replaces the create-bugs workflow with find-regressions. It adds Jira search, candidate categorization, mapping and report generation. Related MicroShift CI workflows now consume search results and support manual bug filing.

Changes

Regression search workflow

Layer / File(s)	Summary
Search and report contract plugins/microshift-ci/scripts/search-bugs.py	The script validates search results and emits suggest, linked, and skip outcomes with updated reports, URLs, filenames, and help text.
Regression candidate and Jira search workflow plugins/microshift-ci/skills/find-regressions/SKILL.md	Adds candidate preparation, Jira searches, mapping files, merged candidates, failure categorization, and report generation.
Doctor workflow integration plugins/microshift-ci/skills/doctor/SKILL.md, plugins/microshift-ci/skills/doctor-refresh/SKILL.md	Updates doctor workflows to invoke find-regressions for release and rebase sources and consume its outputs.
Workflow and investigation guidance plugins/microshift-ci/README.md, plugins/microshift-ci/skills/close-stale-bugs/SKILL.md, plugins/microshift-ci/skills/continue-session/SKILL.md, plugins/microshift-ci/skills/fix-test-bugs/SKILL.md, plugins/shared/scripts/create-report.py, plugins/microshift-ci/agents/prow-job-analyzer.md, plugins/microshift-ci/agents/references/microshift-ci-primer.md	Updates related workflows and investigation guidance. The changes add PCP and source-correlation guidance and remove the create-bugs skill references.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Possibly related PRs

• openshift-eng/edge-tooling#148: Replaces the create-bugs workflow and updates related consumers.
• openshift-eng/edge-tooling#226: Modifies the same regression-analysis and MicroShift CI integration areas.
• openshift-eng/edge-tooling#239: Updates the same investigation guidance and evidence-correlation files.

Suggested labels: ready-for-human-review

🚥 Pre-merge checks | ✅ 10 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Ai-Attribution	⚠️ Warning	AI use is stated in the PR and commits include Co-Authored-By for Claude and CodeRabbit; no Assisted-by or Generated-by trailer appears in the PR commits.	Add the required Assisted-by or Generated-by Red Hat trailer to AI-assisted commits and remove Co-Authored-By trailers for AI tools.

✅ Passed checks (10 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
No-Weak-Crypto	✅ Passed	The available pull-request diff adds no MD5, SHA1, DES, RC4, 3DES, Blowfish, ECB, custom crypto, or secret-comparison usage.
Container-Privileges	✅ Passed	The actual diff changes only Python validation and Markdown documentation; no container/Kubernetes manifest changes or privileged settings were added.
No-Sensitive-Data-In-Logs	✅ Passed	The feature diff adds only help output; raw_error remains pre-existing JSON data handling, and the new report does not print it or any credentials, PII, hostnames, or customer data.
No-Hardcoded-Secrets	✅ Passed	The full PR diff adds no API keys, passwords, credential URLs, private-key blocks, or base64 strings over 32 characters; security terms are documentation/search keywords only.
No-Injection-Vectors	✅ Passed	The PR adds only jira_key validation and a documentation reference; the diff introduces no SQL concatenation, shell=True, eval/exec, unsafe YAML/pickle, os.system, or user-data HTML sink.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly identifies the main command change and the related doctor Step 3 documentation update.

✨ Finishing Touches 💡 1

🛠️ Fix failing CI checks 💡

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 9

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

plugins/microshift-ci/scripts/search-bugs.py (1)

463-476: 📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win

Add positive and negative tests for the changed parser and validator.

CONTRIBUTING.md requires positive/negative tests for changed parsing or validation logic. Add tests for mode handling, suggest / skip / linked actions, jira_key and skip_category combinations, duplicate/missing error signatures, unknown -h / --help handling, and missing --source; extend shared coverage only rather than duplicating it.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@plugins/microshift-ci/scripts/search-bugs.py` around lines 463 - 476, Add
positive and negative tests covering the parser and validator changes near
_validate_results: mode handling, suggest/skip/linked actions, valid and invalid
jira_key/skip_category combinations, duplicate and missing error signatures,
unknown -h/--help options, and missing --source. Extend existing shared test
coverage where available instead of duplicating tests.

Source: Path instructions

🧹 Nitpick comments (1)

plugins/microshift-ci/skills/find-regressions/SKILL.md (1)

1-6: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick win

Reduce the allowed tool set.

Glob, Grep, and mcp__jira__jira_get_issue are not used by the workflow steps in this file. Remove them, or document the step that requires each tool. The skill guidelines require the minimum allowed tools to limit the skill's blast radius.

As per path instructions: use minimum required allowed-tools, especially for Bash-running skills.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@plugins/microshift-ci/skills/find-regressions/SKILL.md` around lines 1 - 6,
Update the frontmatter allowed-tools declaration for the
microshift-ci:find-regressions skill to remove Glob, Grep, and
mcp__jira__jira_get_issue, since no workflow step uses them; retain only the
tools required by the documented workflow.

Source: Path instructions

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@plugins/microshift-ci/README.md`:
- Around line 40-48: Complete the search-only terminology migration: in
plugins/microshift-ci/README.md lines 40-48, describe search results, drafted
suggestions, and manual filing; in
plugins/microshift-ci/skills/close-stale-bugs/SKILL.md line 31, identify the
actual producer or previously generated bug source; in
plugins/microshift-ci/skills/continue-session/SKILL.md lines 66-69, replace
automatic bug-creation wording with search, draft suggestions, and manual
filing; and in plugins/microshift-ci/skills/fix-test-bugs/SKILL.md line 207,
describe the analysis and candidate-workflow relationship instead of feeding bug
creation. Keep component names and cross-references aligned with the new
search-only behavior.

In `@plugins/microshift-ci/scripts/search-bugs.py`:
- Around line 508-509: Extend the validation around the existing linked-action
check so suggest and skip actions reject any non-empty jira_key. Preserve the
current requirement that linked actions have a non-empty jira_key, and add the
inverse validation for non-linked actions in the same result-validation flow.

In `@plugins/microshift-ci/skills/find-regressions/SKILL.md`:
- Around line 73-79: Make the work directory and all merged-candidate/result
output names invocation-specific rather than date- or first-source-only. Update
the `SOURCE_TAG` generation and related output paths in the workflow to include
a unique run identifier or complete normalized source list, and add protection
for shared cache writes so concurrent runs cannot overwrite or interleave
artifacts.
- Around line 87-95: Update the source parsing and preparation flow in the
find-regressions instructions to validate each `<ARGUMENTS>` source against the
documented release, PR, and rebase naming patterns, normalize it, and reject
invalid values before execution. Quote or pass each validated source as a
distinct argument to `search-bugs.py`, and derive `SOURCE_TAG` and any output
paths only from the validated normalized first source rather than raw input.
- Around line 110-123: The find-regressions workflow must make Jira cache
validation source-specific and failure-aware. Update the cache lookup and
validation in the SOURCES workflow to key entries by source and error_signature,
validate the mapping schema and complete candidate coverage, and discard the
entire cache when any check fails. Define inline guard checks around each
mandatory Jira search and foreground-agent invocation so failures stop the
workflow and prevent writing or reusing cache data; write the completion marker
only after every required search succeeds, and document these failure rules
between the relevant phases.
- Around line 116-120: Update the cached Jira results instructions in the
find-regressions skill to require explicit user confirmation before running the
wildcard rm command, and provide a non-destructive refresh option scoped to the
selected sources when available. Preserve the existing cache reuse guidance
while clearly distinguishing the destructive reset from the safer refresh path.
- Around line 142-145: Define deterministic selection for the deduplicated
Search A/B results and multiple Search C regressions. Sort results by a stable
key before selecting the first duplicate, and specify a consistent date-ordering
rule for regressions, including how missing or invalid dates are handled. Use
the selected Jira key and stale-versus-suggest outcome consistently across runs.
- Around line 161-174: Update Search C in the regression-check instructions to
run 1–2 focused text-search queries derived from candidate.keywords rather than
one array-wide query. Merge and deduplicate the closed/verified results before
recording every unique issue as a regression entry, preserving the existing JQL
filters and reporting behavior.
- Around line 131-139: Sanitize candidate keywords and test IDs before
interpolating them into JQL in the search flow. Update extract_keywords() to
allow only safe token characters or escape JQL metacharacters, and update
extract_test_ids() to accept only the documented OCP-NNNNN or numeric formats;
apply this validation consistently to the JQL clauses at all three search
construction points.

---

Outside diff comments:
In `@plugins/microshift-ci/scripts/search-bugs.py`:
- Around line 463-476: Add positive and negative tests covering the parser and
validator changes near _validate_results: mode handling, suggest/skip/linked
actions, valid and invalid jira_key/skip_category combinations, duplicate and
missing error signatures, unknown -h/--help options, and missing --source.
Extend existing shared test coverage where available instead of duplicating
tests.

---

Nitpick comments:
In `@plugins/microshift-ci/skills/find-regressions/SKILL.md`:
- Around line 1-6: Update the frontmatter allowed-tools declaration for the
microshift-ci:find-regressions skill to remove Glob, Grep, and
mcp__jira__jira_get_issue, since no workflow step uses them; retain only the
tools required by the documented workflow.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Pro Plus

Run ID: 215edb06-4a98-44d8-a463-b61a01a54033

📥 Commits

Reviewing files that changed from the base of the PR and between 71fbada and 4e620e4.

📒 Files selected for processing (10)

• plugins/microshift-ci/README.md
• plugins/microshift-ci/scripts/search-bugs.py
• plugins/microshift-ci/skills/close-stale-bugs/SKILL.md
• plugins/microshift-ci/skills/continue-session/SKILL.md
• plugins/microshift-ci/skills/create-bugs/SKILL.md
• plugins/microshift-ci/skills/doctor-refresh/SKILL.md
• plugins/microshift-ci/skills/doctor/SKILL.md
• plugins/microshift-ci/skills/find-regressions/SKILL.md
• plugins/microshift-ci/skills/fix-test-bugs/SKILL.md
• plugins/shared/scripts/create-report.py

💤 Files with no reviewable changes (1)

• plugins/microshift-ci/skills/create-bugs/SKILL.md

[copejon]

Addressed CodeRabbit review-body and summary findings:

• Nitpick (SKILL.md:1-6) "Reduce the allowed tool set": Won't fix — Glob, Grep, and jira_get_issue provide useful fallback capabilities for an AI agent; removing them makes error recovery harder.
• Outside diff (search-bugs.py:463-476) "Add positive and negative tests": Won't fix — generic "add tests" advice without identifying a specific untested edge case.
• No-Sensitive-Data-In-Logs pre-merge check: Won't fix — raw_error comes from publicly accessible CI logs on prow.ci.openshift.org; no credentials, tokens, or PII.
• Ai-Attribution pre-merge check: Won't fix — org-specific commit trailer policy, not a code correctness issue.

[copejon]

/verified by ci