Skip to content

Auto-create QA tickets for Dependabot PRs #48007

Closed
#48008
Closed
Auto-create QA tickets for Dependabot PRs#48007
#48008
Assignees
AndreyKizimenkosharon-fdm
Labels
#g-orchestrationOrchestration product groupstoryA user story defining an entire feature~engineering-initiatedEngineering-initiated story, such as a bug, refactor, or contributor experience improvement.~postmortem-action-itemIssue is an action item resulting from an incident postmortem.
Milestone
4.91.0

Description

@sharon-fdm
sharon-fdm
opened on Jun 22, 2026

Related postmortem: #45763

Goal

User story
As a QA engineer,
I want Dependabot code-dependency PRs to auto-create a QA tracking issue on the board
so that I can verify dependency bumps don't introduce behavioral regressions before release.

Changes

Product

  • UI changes: No changes
  • CLI (fleetctl) usage changes: No changes
  • YAML changes: No changes
  • REST API changes: No changes
  • Fleet's agent (fleetd) changes: No changes
  • Fleet server configuration changes: No changes
  • Exposed, public API endpoint changes: No changes
  • fleetdm.com changes: No changes
  • GitOps mode UI changes: No changes
  • GitOps generation changes: No changes
  • Activity changes: No changes
  • Permissions changes: No changes
  • Changes to paid features or tiers: No changes
  • My device and fleetdm.com/better changes: No changes
  • Usage statistics: No changes
  • Other reference documentation changes: No changes
  • First draft of test plan added
  • Once shipped, requester has been notified
  • Once shipped, dogfooding issue has been filed

Engineering

  • Test plan is finalized
  • Contributor API changes: No changes
  • Feature guide changes: No changes
  • Database schema migrations: No changes
  • Load testing: No changes
  • Pre-QA load test: No changes
  • Load testing/osquery-perf improvements: No changes
  • This is a premium only feature: No

Risk assessment

  • Risk level: Low
  • No product code changes. Only adds a new GitHub Actions workflow and a small exemption to an existing one.

Test plan

Core flow

  • Trigger the workflow by opening a test PR authored by dependabot[bot] (or simulate via workflow_dispatch) on a dependabot/gomod/... branch
  • Verify a QA issue is created with the dependabot label, assigned to @AndreyKizimenko
  • Verify a comment is posted on the PR linking to the issue
  • Verify the issue lands on the :help-qa board in "Awaiting QA" (requires FLEET_GITHUB_TOKEN_PROJECTS secret)

Edge cases

  • Verify non-code PRs (docker, github-actions, website npm) are skipped
  • Verify the workflow warns gracefully if the project token secret is missing
  • Verify dependabot[bot] PRs no longer fail the check-pr-template CI check

Confirmation

  1. Engineer: Added comment to user story confirming successful completion of test plan.
  2. QA: Added comment to user story confirming successful completion of test plan.
  3. QA: Determined whether this story needs Playwright automation.
    • Needs automation: No

Activity

  1. sharon-fdm
    self-assigned this
    on Jun 22, 2026
  2. sharon-fdm
    added
    ~postmortem-action-itemIssue is an action item resulting from an incident postmortem.
    on Jun 22, 2026
  3. sharon-fdm
    added this to 🪄 #g-orchestrationon Jun 22, 2026
  4. github-project-automation
    moved this to 🥚 Ready in 🪄 #g-orchestrationon Jun 22, 2026
  5. sharon-fdm
    added
    #g-orchestrationOrchestration product group
    storyA user story defining an entire feature
    ~engineering-initiatedEngineering-initiated story, such as a bug, refactor, or contributor experience improvement.
    on Jun 22, 2026
  6. github-project-automation
    added this to 🚀 :help-engineeringon Jun 22, 2026
  7. sharon-fdm
    moved this from 🥚 Ready to 🐣 In progress in 🪄 #g-orchestrationon Jun 22, 2026
  8. sharon-fdm
    moved this from 🐣 In progress to 🦤 ‎In review in 🪄 #g-orchestrationon Jul 28, 2026
  9. sharon-fdm
    added a commit that references this issue on Jul 28, 2026

    Auto-create QA tickets for Dependabot PRs (#48008)

    0fd5739
  10. sharon-fdm
    added this to the 4.91.0 milestone on Jul 28, 2026
  11. sharon-fdm
    moved this from 🦤 ‎In review to ✔️Awaiting QA in 🪄 #g-orchestrationon Jul 28, 2026

  12. sharon-fdm commented on Jul 28, 2026

    @sharon-fdm
    sharon-fdm
    on Jul 28, 2026
    CollaboratorAuthor

    @xpkoala , @AndreyKizimenko, Per a postmortem we had, this automation will open a QA ticket every time dependabot creates a bump ticket.
    If you don't think there is anything to QA in this new automation, pass it to Ready for release.

  13. AndreyKizimenko
    self-assigned this
    on Aug 6, 2026

  14. AndreyKizimenko commented on Aug 6, 2026

    @AndreyKizimenko
    AndreyKizimenko
    on Aug 6, 2026
    Member

    @sharon-fdm could you please link the postmortem doc for this?

  15. sharon-fdm commented on Aug 7, 2026

    @sharon-fdm
    sharon-fdm
    on Aug 7, 2026
    CollaboratorAuthor

    @AndreyKizimenko , this

  16. xpkoala commented on Aug 24, 2026

    @xpkoala
    xpkoala
    on Aug 24, 2026
    Contributor

    @sharon-fdm it looks like this feature was reverted. Can you let me know what the state of this ticket should be because of this revert.
    #51429

  17. xpkoala
    moved this from ✔️Awaiting QA to 🥚 Ready in 🪄 #g-orchestrationon Aug 24, 2026

  18. sharon-fdm commented on Aug 24, 2026

    @sharon-fdm
    sharon-fdm
    on Aug 24, 2026
    CollaboratorAuthor

    Ahhh, I wasn't aware of this.
    @lucasmrod, you approved the revert. Do you know why it was reverted? (Noise? Bug?)
    @xpkoala, if we decide not to have this feature, this should be closed with a msg "won't do".

  19. lucasmrod commented on Aug 24, 2026

    @lucasmrod
    lucasmrod
    on Aug 24, 2026
    Member

    @lucasmrod, you approved the revert. Do you know why it was reverted? (Noise? Bug?)

    I assumed noise, worth checking with Luke. (I thought there was some discussion beforehand and I just approved the deletion)

  20. AndreyKizimenko commented on Aug 24, 2026

    @AndreyKizimenko
    AndreyKizimenko
    on Aug 24, 2026
    Member

    @sharon-fdm yes, it was noise. We can't test every single dependency bump. We only need to do that for impactful uprgades, like we already do with Go.
    I believe we were talking with you about this in a different github PR where we agreed that these things need to be tested within unit tests and QA will cover the manual portion during smokes.

  21. sharon-fdm commented on Aug 24, 2026

    @sharon-fdm
    sharon-fdm
    on Aug 24, 2026
    CollaboratorAuthor

    Thanks y'all!
    Closing with "Won't do" due to the above discussion.

  22. sharon-fdm
    closed this as completedon Aug 24, 2026
  23. github-project-automation
    moved this to Done in 🚀 :help-engineeringon Aug 24, 2026
  24. github-project-automation
    moved this from 🥚 Ready to Done in 🪄 #g-orchestrationon Aug 24, 2026

  25. fleet-release commented on Aug 24, 2026

    @fleet-release
    fleet-release
    on Aug 24, 2026
    Contributor

    Dependencies bump in code
    QA tracking flows like streams
    safe updates arrive

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

#g-orchestrationOrchestration product groupstoryA user story defining an entire feature~engineering-initiatedEngineering-initiated story, such as a bug, refactor, or contributor experience improvement.~postmortem-action-itemIssue is an action item resulting from an incident postmortem.

Type

No type

Projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions