Unit test runner shows PASSED in summary when there are failing tests

[ssilverman]

I'm seeing the summary showing as passing when there are failures. Many tests pass, then there's a few failures, and my end-of-test summary looks like this (most names are removed for brevity):

...
test/test_math/test_main.cpp:518:PASS
test/test_math/test_main.cpp:519:PASS
test/test_math/test_main.cpp:520:PASS
test/test_math/test_main.cpp:521:PASS
test/test_math/test_main.cpp:522:PASS
test/test_math/test_main.cpp:414:FAIL
test/test_math/test_main.cpp:414:FAIL
test/test_math/test_main.cpp:414:FAIL
test/test_math/test_main.cpp:414:FAIL
test/test_math/test_main.cpp:414:FAIL
test/test_math/test_main.cpp:414:FAIL
test/test_math/test_main.cpp:529:PASS
test/test_math/test_main.cpp:530:PASS
test/test_math/test_main.cpp:531:PASS
...
-----------------------
91 Tests 6 Failures 0 Ignored 
FAIL

The final summary shows PASSED.

I responded to the following issue, but I don't think this is Teensy-specific: platformio/platform-teensy#68

[ssilverman]

Note that the summary also shows "SKIPPED", even when a test ran with some duration and passed. For example:

teensy41 test_math SKIPPED 00:00:31.206

[ssilverman]

Note: I usually run with pio test -v.