build: fix itest parallelism

[bhandras]

The latest ARM build seems to have only ran half of the itests: https://app.travis-ci.com/github/lightningnetwork/lnd/jobs/537407681
This PR attempts to fix itest parallelism by fixing the number of tranches to the parallelism param.

[orijbot]

Visit https://dashboard.github.orijtech.com?back=0&pr=5734&remote=true&repo=bhandras%2Flnd to see benchmark details.

[guggero]

I think this needs to be documented better as I tend to forget it myself as well all the time.
But the idea of having two separate variables (NUM_ITEST_TRANCHES and ITEST_PARALLELISM) is to be able to run the same test in parallel multiple times as part of the flake hunting.
That way I could do make itest-parallel tranches=1 parallel=8 icase=whatever_test to run the same test 8 times in parallel.

So I think we should just fix the Windows and ARM builds by specifying tranches=2 parallel=2 and perhaps document the differences?

[guggero]

Thanks for noticing this by the way! Now I know why the Windows build was always successful, it also only ran half the tests (apparently the "easy" half).

[bhandras]

So I think we should just fix the Windows and ARM builds by specifying tranches=2 parallel=2 and perhaps document the differences?

Ah, ok! Sorry I think I remembered this from before now that you said it because I reviewed the original PR adding parallelism just forgot! :)

[bhandras]

Thanks for noticing this by the way! Now I know why the Windows build was always successful, it also only ran half the tests (apparently the "easy" half).

Funny because I didn't notice that the windows build was wrong 😅

[bhandras]

So I think we should just fix the Windows and ARM builds by specifying tranches=2 parallel=2 and perhaps document the differences?

Tbh I think our docs are good, I just didn't pay attention to the details: https://github.com/lightningnetwork/lnd/blob/master/docs/MAKEFILE.md#itest-parallel

[yyforyongyu]

I think this needs to be documented better as I tend to forget it myself as well all the time.
But the idea of having two separate variables (NUM_ITEST_TRANCHES and ITEST_PARALLELISM) is to be able to run the same test in parallel multiple times as part of the flake hunting.
That way I could do make itest-parallel tranches=1 parallel=8 icase=whatever_test to run the same test 8 times in parallel.

So I think we should just fix the Windows and ARM builds by specifying tranches=2 parallel=2 and perhaps document the differences?

Yeah I thought we were splitting the tests into several parts and run them in parallel, like a MapReduce.

[guggero]

This PR fixes the problem of only running half the itests. The ARM tests still aren't reliable but seem to fail much later, so maybe we can address those flakes individually.

So LGTM 🎉

[yyforyongyu]

LGTM⚡️