Skip to content

label test for more granular testing - #1855

Merged
aljazkonec1 merged 2 commits into
developfrom
label_tests
Jun 19, 2026
Merged

label test for more granular testing#1855
aljazkonec1 merged 2 commits into
developfrom
label_tests

Conversation

@danilo-pejovic

@danilo-pejovic danilo-pejovic commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

label tests to tone down hil usage

Summary by CodeRabbit

  • Chores
    • Updated the CI test workflow to make test execution more selective by computing separate enablement flags per platform and feature (e.g., Linux, Windows, macOS, replay, sanitizers).
    • Adjusted downstream job conditions so each test suite now runs only when its corresponding criteria are met, using pull request labels for pull requests and event-based defaults otherwise.

@coderabbitai

coderabbitai Bot commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: c2f704f9-9896-422b-ac32-4c08f300fd18

📥 Commits

Reviewing files that changed from the base of the PR and between 0791550 and 7e7a30d.

📒 Files selected for processing (1)
  • .github/workflows/test.workflow.yml
📜 Recent review details
🧰 Additional context used
🪛 zizmor (1.25.2)
.github/workflows/test.workflow.yml

[warning] 40-40: code injection via template expansion (template-injection): may expand into attacker-controllable code

(template-injection)

🔇 Additional comments (4)
.github/workflows/test.workflow.yml (4)

48-50: The concern about has_label silently masking jq parse errors was already raised in a previous review.


33-33: The unused dcl output was already flagged in a previous review.


34-42: RAW_LABELS template injection fix correctly applied.

The RAW_LABELS is now properly passed via the env: section rather than direct shell interpolation, addressing the template injection concern from the previous review.

The static analysis tool also flags line 40 (EVENT_NAME="${{ github.event_name }}"), but github.event_name is a GitHub-controlled value constrained to a fixed set of strings (e.g., push, pull_request), not user-supplied input, so the risk is negligible. For full consistency, you could move it to env: as well, but this is optional.

Source: Linters/SAST tools


52-90: LGTM!


📝 Walkthrough

Walkthrough

The precheck job in the GitHub Actions workflow is reworked to emit six per-target boolean outputs (linux, windows, mac, replay, sanitizers, dcl) evaluated from github.event_name and PR labels. Each downstream test job's if: condition is updated to reference its specific output instead of the old single should_run output.

Changes

Per-platform CI gating

Layer / File(s) Summary
precheck job: per-target output logic
.github/workflows/test.workflow.yml
Replaces the single should_run output with six boolean outputs (linux, windows, mac, replay, sanitizers, dcl). A bash script sets these outputs: for non-pull_request events most flags default to true; for pull_request events flags are derived from PR labels (testable_all, testable_os, testable_replay, testable_sanitizers, testable_with_dcl, testable).
Downstream job if: conditions
.github/workflows/test.workflow.yml
Each test job (run_tsan_tests, run_asan-ubsan_tests, run_windows_tests, run_replay_tests, run_vanilla_mac_tests) has its if: condition updated from should_run == 'true' to the corresponding new output (sanitizers, windows, replay, mac).

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

🐇 Hippity-hop, one flag won't do,
Six little outputs, each platform anew!
Linux and Windows, Mac, replay, and more,
Labels now knock on each test job's door.
DCL stays quiet unless told to run —
Granular gating for everyone! 🎉

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title directly relates to the main change: implementing label-based test selection to enable more granular testing by adding per-target boolean outputs driven by pull request labels.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch label_tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/test.workflow.yml:
- Around line 47-49: The has_label function silently masks JSON parsing errors
from jq when $LABELS contains malformed JSON, causing the function to return
false instead of failing. Add error handling at the beginning of the has_label
function (such as set -e or adding error checks after the jq command) to ensure
that if jq encounters invalid JSON, the function exits with an error rather than
continuing silently and returning a false result.
- Line 33: The `dcl` output that is set based on the `testable_with_dcl` label
is never consumed by any downstream job in the workflow, unlike the other
outputs (`linux`, `windows`, `mac`, `replay`, `sanitizers`) which are referenced
via `needs.precheck.outputs.*` conditions. Either remove the `dcl` output
assignment entirely, or if it is intended for future use, add a clear comment
above the output definition explaining its intended purpose and when it will be
used.
- Around line 38-45: The RAW_LABELS variable is being assigned directly via
shell string interpolation of the GitHub template expression, which poses a
template injection risk. Instead, declare the labels data as an environment
variable at the step level using an env section, and then reference that
environment variable within the shell script. This ensures the JSON data is
passed safely through the environment rather than being directly interpolated
into the shell command.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 13ecd202-4367-49ca-bb24-89dc01cbe6f7

📥 Commits

Reviewing files that changed from the base of the PR and between 77d995d and 0791550.

📒 Files selected for processing (1)
  • .github/workflows/test.workflow.yml
📜 Review details
🧰 Additional context used
🪛 zizmor (1.25.2)
.github/workflows/test.workflow.yml

[warning] 38-38: code injection via template expansion (template-injection): may expand into attacker-controllable code

(template-injection)


[error] 39-39: code injection via template expansion (template-injection): may expand into attacker-controllable code

(template-injection)

🔇 Additional comments (1)
.github/workflows/test.workflow.yml (1)

92-94: LGTM!

Also applies to: 109-111, 122-124, 135-137, 148-150, 164-166

mac: ${{ steps.check.outputs.mac }}
replay: ${{ steps.check.outputs.replay }}
sanitizers: ${{ steps.check.outputs.sanitizers }}
dcl: ${{ steps.check.outputs.dcl }}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Description: Check if dcl output is referenced anywhere in workflow files

# Search for dcl output usage in workflow files
rg -n 'outputs\.dcl' .github/workflows/

Repository: luxonis/depthai-core

Length of output: 145


🏁 Script executed:

# Read the workflow file to understand the context around lines 33 and 82-89
cat -n .github/workflows/test.workflow.yml | sed -n '1,50p'

Repository: luxonis/depthai-core

Length of output: 1596


🏁 Script executed:

# Search for references to precheck outputs in downstream jobs
rg -n 'needs\.precheck\.outputs' .github/workflows/

Repository: luxonis/depthai-core

Length of output: 1234


🏁 Script executed:

# Search for any reference to dcl variable or precheck job usage
rg -n 'needs\.precheck|precheck:' .github/workflows/test.workflow.yml

Repository: luxonis/depthai-core

Length of output: 395


🏁 Script executed:

# Search for any reference to dcl in the workflow file
rg -n '\bdcl\b' .github/workflows/test.workflow.yml

Repository: luxonis/depthai-core

Length of output: 210


🏁 Script executed:

# View lines 70-100 to see the context around 82-89
cat -n .github/workflows/test.workflow.yml | sed -n '70,100p'

Repository: luxonis/depthai-core

Length of output: 1440


Remove the unused dcl output or document if intended for future use.

The dcl output is computed based on the testable_with_dcl label (lines 82–89) but is never referenced by any downstream job. All other outputs (linux, windows, mac, replay, sanitizers) are consumed via needs.precheck.outputs.* conditions, while needs.precheck.outputs.dcl has no references in the workflow. Either remove the output or add a comment explaining its intended purpose.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/test.workflow.yml at line 33, The `dcl` output that is set
based on the `testable_with_dcl` label is never consumed by any downstream job
in the workflow, unlike the other outputs (`linux`, `windows`, `mac`, `replay`,
`sanitizers`) which are referenced via `needs.precheck.outputs.*` conditions.
Either remove the `dcl` output assignment entirely, or if it is intended for
future use, add a clear comment above the output definition explaining its
intended purpose and when it will be used.

Comment thread .github/workflows/test.workflow.yml
Comment on lines +47 to +49
has_label() {
echo "$LABELS" | jq -r '.[].name' | grep -qx "$1"
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial | 💤 Low value

has_label silently fails if jq encounters invalid JSON.

If $LABELS contains malformed JSON, jq will error but has_label will return false (grep finds nothing), masking the parse failure. Consider adding set -e or validating JSON first.

💡 Optional: Add error handling
+          set -euo pipefail
+
           has_label() {
-            echo "$LABELS" | jq -r '.[].name' | grep -qx "$1"
+            echo "$LABELS" | jq -e -r '.[].name' 2>/dev/null | grep -qx "$1"
           }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
has_label() {
echo "$LABELS" | jq -r '.[].name' | grep -qx "$1"
}
set -euo pipefail
has_label() {
echo "$LABELS" | jq -e -r '.[].name' 2>/dev/null | grep -qx "$1"
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/test.workflow.yml around lines 47 - 49, The has_label
function silently masks JSON parsing errors from jq when $LABELS contains
malformed JSON, causing the function to return false instead of failing. Add
error handling at the beginning of the has_label function (such as set -e or
adding error checks after the jq command) to ensure that if jq encounters
invalid JSON, the function exits with an error rather than continuing silently
and returning a false result.

@danilo-pejovic danilo-pejovic added testable Trigger PR testing, latest python build, and Core CI/CD and removed testable_os labels Jun 19, 2026
@aljazkonec1
aljazkonec1 merged commit bcd268e into develop Jun 19, 2026
57 of 58 checks passed
@aljazkonec1
aljazkonec1 deleted the label_tests branch June 19, 2026 13:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

testable Trigger PR testing, latest python build, and Core CI/CD

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants