AVA finishes prematurely on Windows, failing with remaining tests

[matz3]

In our project https://github.com/UI5/linter we are observing issues in our CI pipeline on Windows for quite some time now.
I currently suspect this to be caused by a timing issue when running tests in AVA.

First of all, I would like to check with you, whether I understood the concept of how runners are communicating with the main thread.

Runners are sending messages about tests and whether they finish via an async communication port.
When the runner is done, it fires a final "finish" message, and the process eventually exits.

The main thread receives those messages and records the status of tests, e.g. whether they are finished or not.

What we could observe in the CI and also reproduce in a VM is that AVA finishes before the main thread receives all messages from the runner. This leads to the error × N tests remaining in test.ts, as AVA thinks the runner exited without running all the tests. But in fact, only the information that the tests ran was not received.

This seems to only appear when workerThreads is false, which we require in order to use loaders for TypeScript, coverage and mocking.

I am happy to contribute to a solution. I believe there is no mechanism that ensures all messages are fully sent to the main thread before the runner process exits. Maybe another event ping-pong at the end could help to ensure all messages have been received. That is at least the idea a got from debugging and looking at the code, but I'm not fully aware of the whole concept of AVA or whether this is actually already ensured.

Here is a very simple example with which I could reproduce the problem most of the time.

Config

export default {
	files: [
		"test.js",
	],
	workerThreads: false,
};

Test

import test from "ava";
for (let i = 0; i < 50; i++) {
	test("Test " + i, (t) => {
		t.is(true, true);
	});
}

Output

Note that the test sometimes succeeds and the number of remaining tests differs a lot.

> ava


  ✔ Test 0
  ✔ Test 1
  ✔ Test 2
  ✔ Test 3
  ✔ Test 4
  ✔ Test 5
  ✘ 44 tests remaining in test.js
  ─

  6 tests passed

Exit code: 1

System Information

We've encountered this in our project on GitHub Actions with both, windows-2022 and windows-2025.
I could also reproduce it in a virtual machine with Microsoft Windows 11 Enterprise (Version 10.0.26100 Build 26100).

• Node.js: 22.17.0
• npm: 10.9.2
• AVA: 6.4.0

[novemberborn]

It's odd that occurs with 50 synchronous tests!

The worker doesn't explicitly exit so it must be the child process terminating since it has nothing left to do. Sending the messages over IPC seems like a task it should complete.

Do you know if this is limited to specific Node.js versions?

I can't recall whether we still have a ping step to confirm all messages were received. Reintroducing that and keeping the event loop alive until that happens seems reasonable.

BTW, I am considering making worker threads mandatory. Could you elaborate on why that doesn't work for your use case?

[matz3]

Thank you for the quick response 👍🏻

Do you know if this is limited to specific Node.js versions?

It in fact does not appear in v18.20.8, so it's likely related to changes within Node.js.
With v20.19.3 I could also reproduce the error.

BTW, I am considering making worker threads mandatory. Could you elaborate on why that doesn't work for your use case?

Let me check the details again. One reason was that our code also uses worker threads, which might have been one reason, especially as the code within our worker again needs TypeScript transpiling and we use esmock. I will check again what exactly prevented us from using worker threads.

[matz3]

esmock has dropped support for AVA (iambumblehead/esmock@3bbff0e). It seems to still work with JavaScript, but not when using --import=tsx/esm with TypeScript code.

I briefly tried to use @ava/typescript, but also got errors. We also don't really want to switch to an upfront compilation, as the watch mode likely becomes less convenient.

[matz3]

The problem doesn't even seem to be related to esmock, but rather to tsx. Also without esmock I couldn't get --import=tsx/esm working with worker threads. It works with tsimp, but then I get the errors as reported in tapjs/tsimp#34, so this also isn't a great solution right now. But even then, some tests are failing for unknown reasons. Likely, because the code isn't designed to run in a worker thread.

BTW, I am considering making worker threads mandatory. Could you elaborate on why that doesn't work for your use case?

Could you please add some background for that decision? I do believe that there might be some advantages over child processes, but right now I can list more disadvantages:

• Unable to use worker threads in the code (see WorkerThreads in AVA #3337). This for me is the number 1 reason, why worker threads should not be mandatory, as it would prevent people from testing their code within worker threads. I would even opt for the opposite and make workerThreads an opt-in, as I see more issues with them enabled.
• The code runs in a slightly different setup than in production, where usually a normal Node process is spawned. This opens the risk of facing issues and bugs only in production. If the code is supposed to run in a worker thread in production, the test itself should ensure this environment and start the thread.

We're in general very happy with AVA and would really like to continue using it, especially with workerThreads: false.

I will now continue to investigate the actual root cause of this issue, which is likely related to a change in Node.js. After that I hope to find a solution that ensures the IPC communication works as expected, and will file a PR if I'm successful with that.

[novemberborn]

Could you please add some background for that decision? I do believe that there might be some advantages over child processes, but right now I can list more disadvantages:

See this "discussion": #3383

I'll briefly respond here, but let's continue the discussion outside of this issue.

• Unable to use worker threads in the code (see WorkerThreads in AVA #3337). This for me is the number 1 reason, why worker threads should not be mandatory, as it would prevent people from testing their code within worker threads. I would even opt for the opposite and make workerThreads an opt-in, as I see more issues with them enabled.

I just tried and you can launch a worker thread from a worker thread. Maybe you could provide more context in the aforementioned discussion?

• The code runs in a slightly different setup than in production, where usually a normal Node process is spawned. This opens the risk of facing issues and bugs only in production. If the code is supposed to run in a worker thread in production, the test itself should ensure this environment and start the thread.

Yes, but even a child process is a slightly different environment given that it has a parent it can IPC with.

[novemberborn]

I briefly tried to use @ava/typescript, but also got errors. We also don't really want to switch to an upfront compilation, as the watch mode likely becomes less convenient.

I've found it works quite well with the TypeScript compiler in watch mode. AVA's watch mode is smart enough to detect the *.ts change and then wait for the *.js file to recompile.