[indigo] run_tests.py always reports failure?

[garaemon]

I'm in the middle of digging this problem...
According to run_test.py,
it expects ensure_junit_result_exist to return True.

However, in https://github.com/ros/catkin/blob/indigo-devel/python/catkin/test_results.py#L52, ensure_junit_result_exists does not return True.

From our testcase log:

... logging to /workspace/.ros/log/rostest-a9a70867430b-16376.log
[ROSUNIT] Outputting test results to /workspace/ros/ws_jsk_common/build/test_results/jsk_topic_tools/rostest-test_test_topic_buffer.xml
testhztest_chatter ... ok
testhztest_chatter_buffer ... ok
testhztest_chatter_update ... ok

[ROSTEST]-----------------------------------------------------------------------

[jsk_topic_tools.rosunit-hztest_chatter/test_hz][passed]
[jsk_topic_tools.rosunit-hztest_chatter_buffer/test_hz][passed]
[jsk_topic_tools.rosunit-hztest_chatter_update/test_hz][passed]

SUMMARY
�[32m * RESULT: SUCCESS�[0m
 * TESTS: 3
 * ERRORS: 0
 * FAILURES: 0

rostest log file is in /workspace/.ros/log/rostest-a9a70867430b-16376.log
-- run_tests.py: execute commands
  /opt/ros/indigo/share/rostest/cmake/../../../bin/rostest --pkgdir=/workspace/ros/ws_jsk_common/src/jsk_common/jsk_topic_tools --package=jsk_topic_tools --results-filename test_test_topic_buffer.xml /workspace/ros/ws_jsk_common/src/jsk_common/jsk_topic_tools/test/test_topic_buffer.test 
-- run_tests.py: verify result "/workspace/ros/ws_jsk_common/build/test_results/jsk_topic_tools/rostest-test_test_topic_buffer.xml"
<end of output>
Test time =  61.93 sec
----------------------------------------------------------
Test Failed.
"_ctest_jsk_topic_tools_rostest_test_test_topic_buffer.test" end time: Dec 29 05:35 UTC
"_ctest_jsk_topic_tools_rostest_test_test_topic_buffer.test" time elapsed: 00:01:01
----------------------------------------------------------

[garaemon]

I think this line https://github.com/ros/catkin/blob/indigo-devel/python/catkin/test_results.py#L91 should be return True

[tfoote]

That looks reasonable. Is there a simple test that can demonstrate this problem so we can add it as a regression test and verify this change fixes it?

[dirk-thomas]

This affects CTest since only for those the --return-code option is passed. E.g. catkin_make test on catkin marks all tests as failing.

[dirk-thomas]

Thank you for the report. This was a regression of 11217b3.

Since it hasn't been released yet it was only failing CTests when using catkin from source. Since the upcoming release 0.6.10 would have included this bug I just released 0.6.11.

[garaemon]

thanks 🙇