Make "test cases" have tags for easier selection of what subset is being considered. #24834
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
pullapprove
bot
requested review from
amitnj,
anush-apple,
arkq,
Byungjoo-Lee,
bzbarsky-apple,
carol-apple,
chrisdecenzo,
chshu,
chulspro,
CodeChronos928,
Damian-Nordic,
dhrishi,
electrocucaracha,
emargolis,
franck-apple,
gjc13,
harimau-qirex,
harsha-rajendran,
hawk248,
jelderton,
jepenven-silabs,
jmartinez-silabs,
jmeg-sfy,
jtung-apple,
ksperling-apple,
lazarkov,
lpbeliveau-silabs and
LuDuda
tehampson
approved these changes
Feb 2, 2023
|
PR #24834: Size comparison from 653a5c0 to 34a8882 Increases above 0.2%:
Increases (53 builds for bl602, bl702, cc13x2_26x2, cyw30739, efr32, esp32, k32w, linux, mbed, nrfconnect, psoc6, qpg, telink)
Decreases (40 builds for bl602, bl702, cc13x2_26x2, cyw30739, efr32, esp32, k32w, mbed, nrfconnect, psoc6, qpg, telink)
Full report (53 builds for bl602, bl702, cc13x2_26x2, cyw30739, efr32, esp32, k32w, linux, mbed, nrfconnect, psoc6, qpg, telink)
|
|
PR #24834: Size comparison from 87e7510 to b61e555 Decreases (1 build for k32w)
Full report (6 builds for k32w, mbed, qpg)
|
|
PR #24834: Size comparison from 87e7510 to 9f97289 Increases (3 builds for cc13x2_26x2, psoc6)
Decreases (11 builds for bl602, bl702, cc13x2_26x2, nrfconnect, psoc6, telink)
Full report (47 builds for bl602, bl702, cc13x2_26x2, cyw30739, k32w, linux, mbed, nrfconnect, psoc6, qpg, telink)
|
|
PR #24834: Size comparison from 87e7510 to 020b391 Increases (4 builds for cc13x2_26x2, esp32, psoc6)
Decreases (16 builds for bl602, bl702, cc13x2_26x2, esp32, k32w, psoc6, qpg, telink)
Full report (53 builds for bl602, bl702, cc13x2_26x2, cyw30739, efr32, esp32, k32w, linux, mbed, nrfconnect, psoc6, qpg, telink)
|
|
PR #24834: Size comparison from 87e7510 to 0f07fbb Increases (5 builds for cc13x2_26x2, esp32, psoc6, telink)
Decreases (7 builds for bl602, cc13x2_26x2, k32w, psoc6, telink)
Full report (53 builds for bl602, bl702, cc13x2_26x2, cyw30739, efr32, esp32, k32w, linux, mbed, nrfconnect, psoc6, qpg, telink)
|
|
PR #24834: Size comparison from 38f510c to cf7873e Increases (7 builds for cc13x2_26x2, nrfconnect, psoc6, telink)
Decreases (6 builds for bl602, bl702, esp32, nrfconnect, telink)
Full report (53 builds for bl602, bl702, cc13x2_26x2, cyw30739, efr32, esp32, k32w, linux, mbed, nrfconnect, psoc6, qpg, telink)
|
bzbarsky-apple
approved these changes
Feb 6, 2023
|
PR #24834: Size comparison from 3d09d2d to 5d17af1 Increases (6 builds for bl602, bl702, cc13x2_26x2, esp32, telink)
Decreases (8 builds for bl702, cc13x2_26x2, cyw30739, efr32, esp32, qpg, telink)
Full report (42 builds for bl602, bl702, cc13x2_26x2, cyw30739, efr32, esp32, k32w, linux, mbed, nrfconnect, psoc6, qpg, telink)
|
lecndav
pushed a commit
to lecndav/connectedhomeip
that referenced
this pull request
Mar 22, 2023
…ing considered. (project-chip#24834) * Add ability to tag test cases, and include/exclude tags * Restyle * Move examples to be completely ignored * Update for code review * Typo fix * Restyle * Fix logic for tag inclusion * Fix runtime selection logic * Skip slow tests during PR pull requests. Skip flaky tests by default * Use - instead of _ for arguments to fix proper naming * Code review updates * Add notes about why some yamls are removed from chiptest
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Specifically things like:
One can now set arguments like:
--include-tags IN_DEVELOPMENTto run only in development tests, or--exclude-tags SLOWto not run slow tests. Currently only YAML tests flag things, chip-tool tests retain the older "manual/non-manual" logic.PR now excludes slow tests from running during PR pull requests, however they still run in branch pushes.