Relax flaky test validations #2802
Conversation
One `throttle triggers trailing call when invoked repeatedly` test showed up often on CITGM (https://github.com/nodejs/citgm). To reduce the noise this test causes and to prevent removing `underscore` from CITGM, I relaxed the asserts inside of the function. It would be great if the calls would have a stack trace but there is no such output if the calls fail, so I relaxed both statements. I can not fully say if this will resolve the issue completely but this seemed like the most straight forward fix besides removing the test completely.
|
@BridgeAR Is this still still an issue? |
|
@BridgeAR Is there a way I can see the output of those test runs without having to grant the Node.js Jenkins instance access to my private data? |
|
@BridgeAR A friendly reminder.
If my counting is correct, the links will expire tomorrow.
|
|
Thanks for the reminder. I will copy some results over tomorrow when I am back on my notebook. |
These tests fail relatively often, randomly, though more in some engines than in others. The random failures are a combination of a race condition and the imprecision of the global setTimeout function. This measure shouldn't be taken lightly. It can only be applied if it does not affect the timing of the throttled (or debounced) function invocations. In the current patch, only the time at which the result is read out is slightly delayed so that the readout is less likely to be too early. Part of #2879, also related to #2802.
|
@BridgeAR The commit referenced above (as well as another commit that came shortly before it) should have made the issue to occur much less frequently. Please observe for a while and then decide whether to keep this ticket open. |
|
@BridgeAR Please reopen if this is still an issue. |
One
throttle triggers trailing call when invoked repeatedlytestshowed up often on CITGM (https://github.com/nodejs/citgm). To reduce
the noise this test causes and to prevent removing
underscorefromCITGM, I relaxed the asserts inside of the function. It would be
great if the calls would have a stack trace but there is no such
output if the calls fail, so I relaxed both statements. I can not
fully tell if this will resolve the issue completely but this seemed
like the most straight forward fix besides removing the test
completely.
Refs: #2698