[lp ticker] Board test failures

[sg-]

Trying to get more boards in a CI system there are some test failures that need to first be resolved.
@ccli8

mbed test -t iar -m NUMAKER_PFM_NUC472 -n tests-mbed_hal-lp_ticker,tests-mbed_drivers-lp_timeout
...

Build successes:
  * NUMAKER_PFM_NUC472::IAR::MBED-BUILD
  * NUMAKER_PFM_NUC472::IAR::TESTS-MBED_DRIVERS-LP_TIMEOUT
  * NUMAKER_PFM_NUC472::IAR::TESTS-MBED_HAL-LP_TICKER
mbedgt: greentea test automation tool ver. 1.2.0
mbedgt: test specification file 'C:\Users\samgro01\test\mbed-os\.build/tests\NUMAKER_PFM_NUC472\iar\test_spec.json' (specified with --test-s
pec option)
mbedgt: using 'C:\Users\samgro01\test\mbed-os\.build/tests\NUMAKER_PFM_NUC472\iar\test_spec.json' from current directory!
mbedgt: detecting connected mbed-enabled devices...
mbedgt: detected 1 device
mbedgt: processing target 'NUMAKER_PFM_NUC472' toolchain 'IAR' compatible platforms... (note: switch set to --parallel 1)
mbedgt: test case filter (specified with -n option)
        test filtered in 'tests-mbed_hal-lp_ticker'
        test filtered in 'tests-mbed_drivers-lp_timeout'
mbedgt: running 2 tests for platform 'NUMAKER_PFM_NUC472' and toolchain 'IAR'
mbedgt: mbed-host-test-runner: started
mbedgt: checking for GCOV data...
mbedgt: test on hardware with target id: 1302AAAAAAAAAAAAAAAAAAAA
mbedgt: test suite 'tests-mbed_hal-lp_ticker' ........................................................ TIMEOUT in 32.90 sec
        test case: '1ms lp_ticker' ................................................................... SKIPPED in 0.00 sec
        test case: '1s lp_ticker' .................................................................... SKIPPED in 0.00 sec
        test case: '1s lp_ticker deepsleep' .......................................................... SKIPPED in 0.00 sec
        test case: '1s lp_ticker sleep' .............................................................. SKIPPED in 0.00 sec
        test case: '500us lp_ticker' ................................................................. ERROR in 0.00 sec
        test case: '5s lp_ticker' .................................................................... SKIPPED in 0.00 sec
mbedgt: mbed-host-test-runner: started
mbedgt: checking for GCOV data...
mbedgt: test on hardware with target id: 1302AAAAAAAAAAAAAAAAAAAA
mbedgt: test suite 'tests-mbed_drivers-lp_timeout' ................................................... TIMEOUT in 32.77 sec
        test case: '1ms LowPowerTimeout' ............................................................. SKIPPED in 0.00 sec
        test case: '1sec LowPowerTimeout' ............................................................ SKIPPED in 0.00 sec
        test case: '1sec LowPowerTimeout from deepsleep' ............................................. SKIPPED in 0.00 sec
        test case: '1sec LowPowerTimeout from sleep' ................................................. SKIPPED in 0.00 sec
        test case: '500us LowPowerTimeout' ........................................................... ERROR in 0.00 sec
        test case: '5sec LowPowerTimeout' ............................................................ SKIPPED in 0.00 sec
mbedgt: all tests finished!
mbedgt: shuffle seed: 0.1015920920
mbedgt: test suite report:
+------------------------+--------------------+-------------------------------+---------+--------------------+-------------+
| target                 | platform_name      | test suite                    | result  | elapsed_time (sec) | copy_method |
+------------------------+--------------------+-------------------------------+---------+--------------------+-------------+
| NUMAKER_PFM_NUC472-IAR | NUMAKER_PFM_NUC472 | tests-mbed_drivers-lp_timeout | TIMEOUT | 32.77              | shell       |
| NUMAKER_PFM_NUC472-IAR | NUMAKER_PFM_NUC472 | tests-mbed_hal-lp_ticker      | TIMEOUT | 32.9               | shell       |
+------------------------+--------------------+-------------------------------+---------+--------------------+-------------+
mbedgt: test suite results: 2 TIMEOUT
mbedgt: test case report:
+------------------------+--------------------+-------------------------------+-------------------------------------+--------+--------+-----
----+--------------------+
| target                 | platform_name      | test suite                    | test case                           | passed | failed | resu
lt  | elapsed_time (sec) |
+------------------------+--------------------+-------------------------------+-------------------------------------+--------+--------+-----
----+--------------------+
| NUMAKER_PFM_NUC472-IAR | NUMAKER_PFM_NUC472 | tests-mbed_drivers-lp_timeout | 1ms LowPowerTimeout                 | 0      | 0      | SKIP
PED | 0.0                |
| NUMAKER_PFM_NUC472-IAR | NUMAKER_PFM_NUC472 | tests-mbed_drivers-lp_timeout | 1sec LowPowerTimeout                | 0      | 0      | SKIP
PED | 0.0                |
| NUMAKER_PFM_NUC472-IAR | NUMAKER_PFM_NUC472 | tests-mbed_drivers-lp_timeout | 1sec LowPowerTimeout from deepsleep | 0      | 0      | SKIP
PED | 0.0                |
| NUMAKER_PFM_NUC472-IAR | NUMAKER_PFM_NUC472 | tests-mbed_drivers-lp_timeout | 1sec LowPowerTimeout from sleep     | 0      | 0      | SKIP
PED | 0.0                |
| NUMAKER_PFM_NUC472-IAR | NUMAKER_PFM_NUC472 | tests-mbed_drivers-lp_timeout | 500us LowPowerTimeout               | 0      | 0      | ERRO
R   | 0.0                |
| NUMAKER_PFM_NUC472-IAR | NUMAKER_PFM_NUC472 | tests-mbed_drivers-lp_timeout | 5sec LowPowerTimeout                | 0      | 0      | SKIP
PED | 0.0                |
| NUMAKER_PFM_NUC472-IAR | NUMAKER_PFM_NUC472 | tests-mbed_hal-lp_ticker      | 1ms lp_ticker                       | 0      | 0      | SKIP
PED | 0.0                |
| NUMAKER_PFM_NUC472-IAR | NUMAKER_PFM_NUC472 | tests-mbed_hal-lp_ticker      | 1s lp_ticker                        | 0      | 0      | SKIP
PED | 0.0                |
| NUMAKER_PFM_NUC472-IAR | NUMAKER_PFM_NUC472 | tests-mbed_hal-lp_ticker      | 1s lp_ticker deepsleep              | 0      | 0      | SKIP
PED | 0.0                |
| NUMAKER_PFM_NUC472-IAR | NUMAKER_PFM_NUC472 | tests-mbed_hal-lp_ticker      | 1s lp_ticker sleep                  | 0      | 0      | SKIP
PED | 0.0                |
| NUMAKER_PFM_NUC472-IAR | NUMAKER_PFM_NUC472 | tests-mbed_hal-lp_ticker      | 500us lp_ticker                     | 0      | 0      | ERRO
R   | 0.0                |
| NUMAKER_PFM_NUC472-IAR | NUMAKER_PFM_NUC472 | tests-mbed_hal-lp_ticker      | 5s lp_ticker                        | 0      | 0      | SKIP
PED | 0.0                |
+------------------------+--------------------+-------------------------------+-------------------------------------+--------+--------+-----
----+--------------------+
mbedgt: test case results: 10 SKIPPED / 2 ERROR
mbedgt: completed in 65.91 sec
mbedgt: exited with code 2
[mbed] ERROR: "mbedgt" returned error code 2.
[mbed] ERROR: Command "mbedgt --test-spec C:\Users\samgro01\test\mbed-os\.build/tests\NUMAKER_PFM_NUC472\iar\test_spec.json -n tests-mbed_ha
l-lp_ticker,tests-mbed_drivers-lp_timeout" in "C:\Users\samgro01\test\mbed-os"

---

[sg-]

It also seems that all tests from master fail using -t arm - resolved when #2589 is merged

[sg-]

With #2589 applied

mbed test -t arm -m NUMAKER_PFM_NUC472 -n tests-mbed_hal-lp_ticker,tests-mbed_drivers-lp_timeout
...
Build successes:
  * NUMAKER_PFM_NUC472::ARM::MBED-BUILD
  * NUMAKER_PFM_NUC472::ARM::TESTS-MBED_DRIVERS-LP_TIMEOUT
  * NUMAKER_PFM_NUC472::ARM::TESTS-MBED_HAL-LP_TICKER
mbedgt: greentea test automation tool ver. 1.2.0
mbedgt: test specification file 'C:\Users\samgro01\test\mbed-os\.build/tests\NUMAKER_PFM_NUC472\arm\test_spec.json' (specified with --test-s
pec option)
mbedgt: using 'C:\Users\samgro01\test\mbed-os\.build/tests\NUMAKER_PFM_NUC472\arm\test_spec.json' from current directory!
mbedgt: detecting connected mbed-enabled devices...
mbedgt: detected 1 device
mbedgt: processing target 'NUMAKER_PFM_NUC472' toolchain 'ARM' compatible platforms... (note: switch set to --parallel 1)
mbedgt: test case filter (specified with -n option)
        test filtered in 'tests-mbed_hal-lp_ticker'
        test filtered in 'tests-mbed_drivers-lp_timeout'
mbedgt: running 2 tests for platform 'NUMAKER_PFM_NUC472' and toolchain 'ARM'
mbedgt: mbed-host-test-runner: started
mbedgt: checking for GCOV data...
mbedgt: test on hardware with target id: 1302AAAAAAAAAAAAAAAAAAAA
mbedgt: test suite 'tests-mbed_hal-lp_ticker' ........................................................ TIMEOUT in 32.88 sec
        test case: '1ms lp_ticker' ................................................................... SKIPPED in 0.00 sec
        test case: '1s lp_ticker' .................................................................... SKIPPED in 0.00 sec
        test case: '1s lp_ticker deepsleep' .......................................................... SKIPPED in 0.00 sec
        test case: '1s lp_ticker sleep' .............................................................. SKIPPED in 0.00 sec
        test case: '500us lp_ticker' ................................................................. ERROR in 0.00 sec
        test case: '5s lp_ticker' .................................................................... SKIPPED in 0.00 sec
mbedgt: mbed-host-test-runner: started
mbedgt: checking for GCOV data...
mbedgt: test on hardware with target id: 1302AAAAAAAAAAAAAAAAAAAA
mbedgt: test suite 'tests-mbed_drivers-lp_timeout' ................................................... TIMEOUT in 32.83 sec
        test case: '1ms LowPowerTimeout' ............................................................. SKIPPED in 0.00 sec
        test case: '1sec LowPowerTimeout' ............................................................ SKIPPED in 0.00 sec
        test case: '1sec LowPowerTimeout from deepsleep' ............................................. SKIPPED in 0.00 sec
        test case: '1sec LowPowerTimeout from sleep' ................................................. SKIPPED in 0.00 sec
        test case: '500us LowPowerTimeout' ........................................................... ERROR in 0.00 sec
        test case: '5sec LowPowerTimeout' ............................................................ SKIPPED in 0.00 sec
mbedgt: all tests finished!
mbedgt: shuffle seed: 0.6150381767
mbedgt: test suite report:
+------------------------+--------------------+-------------------------------+---------+--------------------+-------------+
| target                 | platform_name      | test suite                    | result  | elapsed_time (sec) | copy_method |
+------------------------+--------------------+-------------------------------+---------+--------------------+-------------+
| NUMAKER_PFM_NUC472-ARM | NUMAKER_PFM_NUC472 | tests-mbed_drivers-lp_timeout | TIMEOUT | 32.83              | shell       |
| NUMAKER_PFM_NUC472-ARM | NUMAKER_PFM_NUC472 | tests-mbed_hal-lp_ticker      | TIMEOUT | 32.88              | shell       |
+------------------------+--------------------+-------------------------------+---------+--------------------+-------------+
mbedgt: test suite results: 2 TIMEOUT
mbedgt: test case report:
+------------------------+--------------------+-------------------------------+-------------------------------------+--------+--------+-----
----+--------------------+
| target                 | platform_name      | test suite                    | test case                           | passed | failed | resu
lt  | elapsed_time (sec) |
+------------------------+--------------------+-------------------------------+-------------------------------------+--------+--------+-----
----+--------------------+
| NUMAKER_PFM_NUC472-ARM | NUMAKER_PFM_NUC472 | tests-mbed_drivers-lp_timeout | 1ms LowPowerTimeout                 | 0      | 0      | SKIP
PED | 0.0                |
| NUMAKER_PFM_NUC472-ARM | NUMAKER_PFM_NUC472 | tests-mbed_drivers-lp_timeout | 1sec LowPowerTimeout                | 0      | 0      | SKIP
PED | 0.0                |
| NUMAKER_PFM_NUC472-ARM | NUMAKER_PFM_NUC472 | tests-mbed_drivers-lp_timeout | 1sec LowPowerTimeout from deepsleep | 0      | 0      | SKIP
PED | 0.0                |
| NUMAKER_PFM_NUC472-ARM | NUMAKER_PFM_NUC472 | tests-mbed_drivers-lp_timeout | 1sec LowPowerTimeout from sleep     | 0      | 0      | SKIP
PED | 0.0                |
| NUMAKER_PFM_NUC472-ARM | NUMAKER_PFM_NUC472 | tests-mbed_drivers-lp_timeout | 500us LowPowerTimeout               | 0      | 0      | ERRO
R   | 0.0                |
| NUMAKER_PFM_NUC472-ARM | NUMAKER_PFM_NUC472 | tests-mbed_drivers-lp_timeout | 5sec LowPowerTimeout                | 0      | 0      | SKIP
PED | 0.0                |
| NUMAKER_PFM_NUC472-ARM | NUMAKER_PFM_NUC472 | tests-mbed_hal-lp_ticker      | 1ms lp_ticker                       | 0      | 0      | SKIP
PED | 0.0                |
| NUMAKER_PFM_NUC472-ARM | NUMAKER_PFM_NUC472 | tests-mbed_hal-lp_ticker      | 1s lp_ticker                        | 0      | 0      | SKIP
PED | 0.0                |
| NUMAKER_PFM_NUC472-ARM | NUMAKER_PFM_NUC472 | tests-mbed_hal-lp_ticker      | 1s lp_ticker deepsleep              | 0      | 0      | SKIP
PED | 0.0                |
| NUMAKER_PFM_NUC472-ARM | NUMAKER_PFM_NUC472 | tests-mbed_hal-lp_ticker      | 1s lp_ticker sleep                  | 0      | 0      | SKIP
PED | 0.0                |
| NUMAKER_PFM_NUC472-ARM | NUMAKER_PFM_NUC472 | tests-mbed_hal-lp_ticker      | 500us lp_ticker                     | 0      | 0      | ERRO
R   | 0.0                |
| NUMAKER_PFM_NUC472-ARM | NUMAKER_PFM_NUC472 | tests-mbed_hal-lp_ticker      | 5s lp_ticker                        | 0      | 0      | SKIP
PED | 0.0                |
+------------------------+--------------------+-------------------------------+-------------------------------------+--------+--------+-----
----+--------------------+
mbedgt: test case results: 10 SKIPPED / 2 ERROR
mbedgt: completed in 65.93 sec
mbedgt: exited with code 2
[mbed] ERROR: "mbedgt" returned error code 2.
[mbed] ERROR: Command "mbedgt --test-spec C:\Users\samgro01\test\mbed-os\.build/tests\NUMAKER_PFM_NUC472\arm\test_spec.json -n tests-mbed_ha
l-lp_ticker,tests-mbed_drivers-lp_timeout" in "C:\Users\samgro01\test\mbed-os"
---

[ccli8]

@sg NUMAKER_PFM_NUC472 has a bug in lp_ticker.c and cannot pass these newly added tests tests-mbed_hal-lp_ticker and tests-mbed_drivers-lp_timeout. For the issue, there will be PR later.
#2589 is to fix heap configuration error on armcc toolchain (-t ARM). This error will cause most tests on armcc toolchain to fail without fix. So it needs to be fixed immediately.

[ccli8]

Fixed with 516efa3. Expect it to be closed. @sg.

[bridadan]

Thanks @ccli8!