Investigate flaky tests on AIX

[sam-github]

Investigate tests skipped by:

• test: skip tests related to CI failures on AIX #28469
• test: skip external-exceed-max-by-1-hex on AIX #28516

The first batch might no longer be flaky after nodejs/build#1820 is fixed. Its unclear where the second batch is related or not.

• Version:
• Platform:
• Subsystem:

[bnoordhuis]

I re-enabled the addons/stringbytes-external-exceed-max tests and ran them repeatedly on test-osuosl-aix61-ppc64_be-2 but I can't get them to fail, not even with python tools/test.py -J.

I hacked test/common/index.js so it always says there is enough memory:

diff --git a/test/addons/addon.status b/test/addons/addon.status
index ee094f4..ec2d213 100644
--- a/test/addons/addon.status
+++ b/test/addons/addon.status
@@ -1,19 +1,3 @@
 prefix addons
 
 [true] # This section applies to all platforms
-
-[$system==aix]
-# https://github.com/nodejs/build/issues/1820#issuecomment-505998851
-# https://github.com/nodejs/node/pull/28469
-# https://github.com/nodejs/node/pull/28516
-stringbytes-external-exceed-max/test-stringbytes-external-exceed-max.js: SKIP
-
-# https://github.com/nodejs/node/pull/28516
-stringbytes-external-exceed-max/test-stringbytes-external-at-max: SKIP
-stringbytes-external-exceed-max/test-stringbytes-external-exceed-max-by-1-ascii: SKIP
-stringbytes-external-exceed-max/test-stringbytes-external-exceed-max-by-1-base64: SKIP
-stringbytes-external-exceed-max/test-stringbytes-external-exceed-max-by-1-binary: SKIP
-stringbytes-external-exceed-max/test-stringbytes-external-exceed-max-by-1-hex: SKIP
-stringbytes-external-exceed-max/test-stringbytes-external-exceed-max-by-1-utf8: SKIP
-stringbytes-external-exceed-max/test-stringbytes-external-exceed-max-by-2: SKIP
-stringbytes-external-exceed-max/test-stringbytes-external-exceed-max: SKIP
diff --git a/test/common/index.js b/test/common/index.js
index 13604d0..a627922 100644
--- a/test/common/index.js
+++ b/test/common/index.js
@@ -111,7 +111,7 @@ const isOpenBSD = process.platform === 'openbsd';
 const isLinux = process.platform === 'linux';
 const isOSX = process.platform === 'darwin';
 
-const enoughTestMem = os.totalmem() > 0x70000000; /* 1.75 Gb */
+const enoughTestMem = true;
 const cpus = os.cpus();
 const enoughTestCpu = Array.isArray(cpus) &&
                       (cpus.length > 1 || cpus[0].speed > 999);

It makes no difference however because the tests have additional checks that creating a big buffer is possible:

node/test/addons/stringbytes-external-exceed-max/test-stringbytes-external-at-max.js

Lines 15 to 22 in 462f438

	let buf;
	try {
	buf = Buffer.allocUnsafe(kStringMaxLength);
	} catch (e) {
	// If the exception is not due to memory confinement then rethrow it.
	if (e.message !== 'Array buffer allocation failed') throw (e);
	common.skip(skipMessage);
	}

And that causes the tests to bail out with a "Skipped:" message.

Not sure where to go from here. Maybe just re-enable the tests and see what happens?

[mhdawson]

If you create a PR that re-enables them we can run the CI on that PR a bunch of times to see if we see failures. It was very common before they were diabled.

[bnoordhuis]

Here's a trial run: https://ci.nodejs.org/job/node-test-commit/30163/

[bnoordhuis]

Another run with just AIX: https://ci.nodejs.org/job/node-test-commit-aix/24447/

edit: and another: https://ci.nodejs.org/job/node-test-commit-aix/24448/

[sam-github]

24448 failed:

04:35:29 ok 2424 addons/stringbytes-external-exceed-max/test-stringbytes-external-at-max
04:35:29   ---
04:35:29   duration_ms: 3.59
04:35:29   ...
04:38:07 FATAL: command execution failed
04:38:07 java.nio.channels.ClosedChannelException

This is an instance of nodejs/build#1849

[targos]

Closing as there was no recent activity