Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The skipping of items without a relevant path was failing to pick ite… #26328

Conversation

hicklin
Copy link
Contributor

@hicklin hicklin commented May 2, 2023

Upon running ./scripts/tests/run_test_suite.py --chip-tool ./examples/chip-tool/out/debug/chip-tool --target 'TestModeSelectCluster' run --all-clusters-app examples/all-clusters-app/linux/out/debug/chip-all-clusters-app to run the tests against an application, the following error was returned.

2023-05-02 14:00:46.638 INFO    Ensuring /run is privately accessible
2023-05-02 14:00:46.834 INFO    Waiting for IPv6 DaD to complete (no tentative addresses)
2023-05-02 14:00:48.864 INFO    No more tentative addresses
2023-05-02 14:00:48.864 INFO    Each test will be executed 1 times
2023-05-02 14:00:48.866 INFO    Starting iteration 1
2023-05-02 14:00:48.866 INFO    TestModeSelectCluster - Starting test
2023-05-02 14:00:48.866 ERROR   !!!!!!!!!!!!!!!!!!!! ERROR !!!!!!!!!!!!!!!!!!!!!!
2023-05-02 14:00:48.866 ERROR   ================ CAPTURED LOG START ==================
2023-05-02 14:00:48.867 ERROR   ================ CAPTURED LOG END ====================
2023-05-02 14:00:48.867 ERROR   TestModeSelectCluster          - FAILED in 0.00 seconds
Traceback (most recent call last):
  File "./scripts/tests/run_test_suite.py", line 342, in cmd_run
    test.Run(
  File "/home/whicklin/Documents/openRepos/connectedhomeip/scripts/tests/chiptest/test_definition.py", line 293, in Run
    key = os.path.basename(path[-1])
  File "/usr/lib/python3.8/posixpath.py", line 142, in basename
    p = os.fspath(p)
TypeError: expected str, bytes or os.PathLike object, not NoneType

This is caused by the test_definition.py script not skipping invalid application paths because those paths happen to have a length != 1. This PR skips all paths where the last element is None, meaning that it is invalid.

…ms with the final element set to None. Now we skip whenever the final element in the path is None.
@CLAassistant
Copy link

CLAassistant commented May 2, 2023

CLA assistant check
All committers have signed the CLA.

@github-actions
Copy link

github-actions bot commented May 2, 2023

PR #26328: Size comparison from fa505f6 to f2b9ce8

Full report (4 builds for cc32xx, mbed, qpg)
platform target config section fa505f6 f2b9ce8 change % change
cc32xx lock CC3235SF_LAUNCHXL 0 0 0 0.0
(read only) 604042 604042 0 0.0
(read/write) 204156 204156 0 0.0
.ARM.attributes 44 44 0 0.0
.ARM.exidx 8 8 0 0.0
.bss 197568 197568 0 0.0
.comment 206 206 0 0.0
.data 1468 1468 0 0.0
.debug_abbrev 956764 956764 0 0.0
.debug_aranges 103240 103240 0 0.0
.debug_frame 349432 349432 0 0.0
.debug_info 19559510 19559510 0 0.0
.debug_line 2687404 2687404 0 0.0
.debug_line_str 513 513 0 0.0
.debug_loc 33340 33340 0 0.0
.debug_loclists 1515630 1515630 0 0.0
.debug_ranges 4984 4984 0 0.0
.debug_rnglists 95966 95966 0 0.0
.debug_str 3082308 3082308 0 0.0
.ramVecs 780 780 0 0.0
.resetVecs 64 64 0 0.0
.rodata 104282 104282 0 0.0
.shstrtab 265 265 0 0.0
.stack 2048 2048 0 0.0
.strtab 479192 479192 0 0.0
.symtab 286736 286736 0 0.0
.text 497636 497636 0 0.0
mbed lock-app CY8CPROTO_062_4343W+release (read only) 6224 6224 0 0.0
(read/write) 2496168 2496168 0 0.0
.bss 216296 216296 0 0.0
.data 5144 5144 0 0.0
.text 1458852 1458852 0 0.0
qpg lighting-app qpg6105+debug (read/write) 1167360 1167360 0 0.0
.bss 99632 99632 0 0.0
.data 856 856 0 0.0
.text 614460 614460 0 0.0
lock-app qpg6105+debug (read/write) 1136720 1136720 0 0.0
.bss 94784 94784 0 0.0
.data 852 852 0 0.0
.text 583816 583816 0 0.0

@github-actions
Copy link

github-actions bot commented May 2, 2023

PR #26328: Size comparison from f587c34 to 0c53e1f

Decreases (1 build for cc32xx)
platform target config section f587c34 0c53e1f change % change
cc32xx lock CC3235SF_LAUNCHXL .debug_info 19559511 19559509 -2 -0.0
Full report (1 build for cc32xx)
platform target config section f587c34 0c53e1f change % change
cc32xx lock CC3235SF_LAUNCHXL 0 0 0 0.0
(read only) 604042 604042 0 0.0
(read/write) 204156 204156 0 0.0
.ARM.attributes 44 44 0 0.0
.ARM.exidx 8 8 0 0.0
.bss 197568 197568 0 0.0
.comment 206 206 0 0.0
.data 1468 1468 0 0.0
.debug_abbrev 956764 956764 0 0.0
.debug_aranges 103240 103240 0 0.0
.debug_frame 349432 349432 0 0.0
.debug_info 19559511 19559509 -2 -0.0
.debug_line 2687404 2687404 0 0.0
.debug_line_str 513 513 0 0.0
.debug_loc 33340 33340 0 0.0
.debug_loclists 1515630 1515630 0 0.0
.debug_ranges 4984 4984 0 0.0
.debug_rnglists 95966 95966 0 0.0
.debug_str 3082308 3082308 0 0.0
.ramVecs 780 780 0 0.0
.resetVecs 64 64 0 0.0
.rodata 104282 104282 0 0.0
.shstrtab 265 265 0 0.0
.stack 2048 2048 0 0.0
.strtab 479192 479192 0 0.0
.symtab 286736 286736 0 0.0
.text 497636 497636 0 0.0

if len(path) == 1 and path[0] is None:
if path[-1] is None:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What was the actual tuple for the failing path here? How did we get something where the "leaf" is None but the length is not 1?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is a fair point. TBH I did not put in a lot of investigation time to figure out how the situation arose. I printed the list of paths before the for loop and got

ApplicationPaths(
  chip_tool=['ip', 'netns', 'exec', 'tool', './examples/chip-tool/out/debug/chip-tool'], 
  all_clusters_app=['ip', 'netns', 'exec', 'app', 'examples/all-clusters-app/linux/out/debug/chip-all-clusters-app'], 
  lock_app=['ip', 'netns', 'exec', 'app', None], 
  ota_provider_app=['ip', 'netns', 'exec', 'app', None], 
  ota_requestor_app=['ip', 'netns', 'exec', 'app', None], 
  tv_app=['ip', 'netns', 'exec', 'app', None], 
  bridge_app=['ip', 'netns', 'exec', 'app', None], 
  chip_repl_yaml_tester_cmd=['ip', 'netns', 'exec', 'tool', 'python3', '/home/whicklin/Documents/openRepos/connectedhomeip/scripts/tests/chiptest/yamltest_with_chip_repl_tester.py'], 
  chip_tool_with_python_cmd=['ip', 'netns', 'exec', 'tool', 'python3', '/home/whicklin/Documents/openRepos/connectedhomeip/scripts/tests/yaml/chiptool.py']
)

so I thought the path might be being built by appending to a list and that in some cases it was appending none.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, on Linux there are these netns prefixes on everything, I see. Thank you!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should not instead here make netns not create a list and have None preserved if the input path is None.

Even more, we should probably have None or List of valid strings rather than trying to accept [None] and detect it. We seem to introduce odd data along the pipeline and then try to deal with it. I would rather not have odd data to begin with.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doable, but requires finding all places that manipulate this path and fixing them....

@bzbarsky-apple
Copy link
Contributor

Fast-tracking: this has been up for review for a while, and is a tests-only fix.

@bzbarsky-apple bzbarsky-apple enabled auto-merge (squash) May 5, 2023 17:08
@bzbarsky-apple bzbarsky-apple merged commit fc0a2fb into project-chip:master May 5, 2023
@hicklin hicklin deleted the hotfix/fix_bug_in_test_definition_py branch July 28, 2023 09:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants