Repository navigation
Flaky Linux tests no longer flaky? #4446
Description
Currently there are 3 tests marked as flaky on Linux:
- test-http-client-timeout-event
- test-child-process-buffering
- test-child-process-exit-code
They were all added in the initial flaky test list back in August with #2424. I've been looking back through the test history for the Linux CI nodes and I cannot find those tests ever failing in recent months. Perhaps they can be removed from the list now?
Activity
jbergstroem commentedon Dec 28, 2015
Could you stress test them?
mscdex commentedon Dec 28, 2015
I couldn't find out which Linux nodes these tests were supposedly flaky on, so I'm just picking a few.
First up:
test-http-client-timeout-event:centos5-32: https://ci.nodejs.org/job/node-stress-single-test/nodes=centos5-32/239/ubuntu1504-64: https://ci.nodejs.org/job/node-stress-single-test/nodes=ubuntu1504-64/237/armv7-wheezy: https://ci.nodejs.org/job/node-stress-single-test/nodes=armv7-wheezy/240pi1-raspbian-wheezy(I'll never do that again until Cross-compile for (ARM) stress tests build#277 is resolved): https://ci.nodejs.org/job/node-stress-single-test/nodes=pi1-raspbian-wheezy/238Trott commentedon Dec 28, 2015
test-http-client-timeout-eventwas marked flaky because of failures on pi1-raspbian-wheezy. #2555test-child-process-bufferingwas marked flaky because of failures on armv7-wheezy.#2627 (If it's not flaky anymore, that issue can be closed, of course.)
Not sure what OS
test-http-client-timeout-eventtest-child-process-exit-codewas failing on so the scatter-shot method might need to be used there too. Although I don't think I've seen a yellow build in a long, long time on Linux, so it's probably not flaky now....mscdex commentedon Dec 28, 2015
@Trott With regard to your last remark, did you mean
test-child-process-exit-codeinstead oftest-http-client-timeout-event?Trott commentedon Dec 28, 2015
@mscdex Yes, that's exactly right. It was a cut and paste error. Sorry about that. I'll edit it...
mscdex commentedon Dec 28, 2015
Ok, here's a stress test for
armv7-wheezyandtest-child-process-buffering: https://ci.nodejs.org/job/node-stress-single-test/247/nodes=armv7-wheezy/Here's a stress test for
pi1-raspbian-wheezyandtest-http-client-timeout-event(I accidentally aborted the one I had going for 3 hours... argh): https://ci.nodejs.org/job/node-stress-single-test/248/nodes=pi1-raspbian-wheezy/jbergstroem commentedon Dec 28, 2015
Looking good so far!
Trott commentedon Jan 3, 2016
I'd be good with removing these three tests at this point. The only one that hasn't been run through the stress test is
test-child-process-exit-code. Like you, I can't find any examples of it failing recently. If we remove it from the flaky list and it turns out that it is in fact still flaky (which seems unlikely to happen), we can put it back in as flaky.mscdex commentedon Jan 3, 2016
@Trott Sounds good to me.
test: remove flaky designations for tests
Trott commentedon Jan 3, 2016
Created PR #4519 for this. @mscdex If you already have a PR for it that I'm just not seeing or wanted to put it in yourself for whatever reason (better commit message, your own personal tracking of xyz, whatever), just go ahead and close mine.
test: remove flaky designations for tests
test: remove flaky designations for tests
test: remove flaky designations for tests
test: remove flaky designations for tests
test: remove flaky designations for tests