CI scripting doesn't retry modifications to tests on non-default platforms

[andyross]

It seems like CI gets it's multipass/retry behavior confused in the presence of changes to test code. When a test is modified, the "--save-tests" pre-pass uses --all to enable all platforms to make sure the test at least builds everywhere. But the final "--only-failed" pass seems not to use that list and instead just retries the default list, so any tests that fail the first time never actually get retried, and the test as a whole shows a failure.

PR #13916 showed a situation like this, note the single (spurious) failure in the first run and the fact that zero tests got run in the retry:

+ /home/buildslave/src/github.com/zephyrproject-rtos/zephyr/scripts/sanitycheck --inline-logs --enable-coverage -N --load-tests test_file.txt --subset 1/5
JOBS: 12
Running only a subset: 1/5
Cleaning output directory /home/buildslave/src/github.com/zephyrproject-rtos/zephyr/sanity-out
445 tests selected, 0 tests discarded due to filters
  1/443 adafruit_trinket_m0       samples/synchronization/sample.synchronization     PASSED
...
...
...
443/443 native_posix              tests/net/tx_timestamp/net.timestamp               PASSED
442 of 443 tests passed with 0 warnings in 1263 seconds
+ sleep 10
+ /home/buildslave/src/github.com/zephyrproject-rtos/zephyr/scripts/sanitycheck --inline-logs --enable-coverage -N --only-failed --outdir=out-2nd-pass
JOBS: 12
Selecting default platforms per test case
Building testcase defconfigs...
0 tests selected, 91523 tests discarded due to filters
0 of 0 tests passed with 0 warnings in 2 seconds
+ rm -f test_file.txt