Repository navigation
Fixathon / bug hunt #18660
Description
Currently we have a lot of tests failing on our CI and those failures probably also indicate that we have some real underlying bugs and not only flakes even though most of those are probably flaky tests.
To be concrete: almost 10% of all open issues are reports for "flaky" tests.
Nevertheless it is hard work to come by and it also increases the work for people who open PRs and us collaborators who land things because we always have to check the CI closely.
A lot of failures are also infrastructure related. But we simple do not have enough people to get a upper hand right now.
So I suggest to create a event like a hackathon where people are invited to fix those bugs. It is likely that it would be interesting for a couple persons and the combined knowledge in a room would probably also lead to even more bugs fixed than just having us sitting where ever we are right now. Especially for hard to come by bugs that do not have any stack trace.
Other programs do similar things like LibreOffice with their bug hunting sessions, even though that is nothing in person.
What do others think about that? @nodejs/collaborators @nodejs/tsc
I guess the foundation might be able to provide for all necessities for something like that?
Activity
apapirovski commentedon Feb 8, 2018
I'm in favour of this becoming a thing at the summits, at the very least. Having stable infrastructure is very important for a variety of reasons and it might be easier to figure out some of these flaky tests with the combined knowledge of a few collaborators.
maclover7 commentedon Feb 8, 2018
cc @nodejs/build, the flakiness of the test suite / infrastructure issues was brought up during out last WG meeting. I think having this kind of event would be very helpful :)
mhdawson commentedon Feb 8, 2018
I think it would be a good idea, even with best intentions it is sometimes hard to get to work like this. If we had a scheduled time we were all going to get together and make a push on it, it might help to make progress.
BridgeAR commentedon Feb 8, 2018
I can open a poll to find a good time but first we might want to figure out if we want to consider it during the week or over a weekend.
🎉 During the week.
❤️ During the weekend.
👍 It does not matter.
j0t3x commentedon Feb 9, 2018
this will only happen in summits? what about letting remote guys join this initiative?
joyeecheung commentedon Feb 9, 2018
BTW I am working on a tool that walks the CI for a given build number and guess the reason of the failed builds. It needs a bit more tests to land and get released but I have found it quite helpful since I started to use it. nodejs/node-core-utils#161
BridgeAR commentedon Feb 9, 2018
@joyeecheung that is really great! I am looking forward to have that :-)
jvelezpo commentedon Feb 9, 2018
I am in, My last 2 PR where failing due to something in the CI but was unrelated to the code in my PR.
I will be more than happy to help with this 😎
BridgeAR commentedon Jul 23, 2018
We actually fixed a couple of issues and currently our CI is becoming a little bit more stable. I guess this is not really a thing we are still going to follow, so I am going to close it. If someone believes it should stay open, please just go ahead.