Skip to content

investigate flaky sequential/test-benchmark-child-process on Windows #12560

Closed
@Trott

Description

@Trott
  • Version: v8.0.0-pre
  • Platform: win2008r2
  • Subsystem: test

sequential/test-benchmark-child-process is still failing sometimes flaky on Windows in CI. I'll open a PR to mark it as flaky. This issue is for trying to locate the problem and a solution.

When the test succeeds, it seems to take just a few seconds.

https://ci.nodejs.org/job/node-test-binary-windows/RUN_SUBSET=3,VS_VERSION=vs2015,label=win2008r2/7865/console

ok 356 sequential/test-benchmark-child-process
  ---
  duration_ms: 1.764

When it fails, it's a timeout.

https://ci.nodejs.org/job/node-test-binary-windows/7867/RUN_SUBSET=3,VS_VERSION=vs2015,label=win2008r2/console

not ok 356 sequential/test-benchmark-child-process
  ---
  duration_ms: 60.70
  severity: fail
  stack: |-
    timeout

This would suggest a race condition or something else causing a child process to hang or something. And that might be the cause. But...

Interestingly, a stress test where the five benchmarks that this test calls were all split out into individual tests, succeeded but each test took around 30 seconds to run. Wha??!! I know! (Only other change in those tests is the dur option for the benchmarks was increased from 0 to 0.1. Well, that, and that this test was run on win2016 so maybe the results are completely irrelevant? I don't know.)

https://ci.nodejs.org/job/node-stress-single-test/1161/nodes=win2016/console:

ok 1 sequential/test-benchmark-child-process-exec-stdout
  ---
  duration_ms: 3.158
  ...
ok 2 sequential/test-benchmark-child-process-params
  ---
  duration_ms: 36.735
  ...
ok 3 sequential/test-benchmark-child-process-read-ipc
  ---
  duration_ms: 30.116
  ...
ok 4 sequential/test-benchmark-child-process-read
  ---
  duration_ms: 31.844
  ...
ok 5 sequential/test-benchmark-child-process-spawn-echo
  ---
  duration_ms: 31.661

So I'm not sure what's going on here. Maybe it can be worked out by someone more comfortable testing and debugging on Windows or someone more deeply familiar with child_process and/or our benchmarking code. @nodejs/platform-windows @nodejs/benchmarking @mscdex @cjihrig @bnoordhuis @nodejs/testing

Metadata

Metadata

Assignees

No one assigned

    Labels

    benchmarkIssues and PRs related to the benchmark subsystem.child_processIssues and PRs related to the child_process subsystem.testIssues and PRs related to the tests.windowsIssues and PRs related to the Windows platform.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions