`distributed/cli/tests/test_dask_scheduler.py::test_dashboard_port_zero`

[fjetter]

This test is evolving to be a frequent offender

Short test report (failures on main)

https://github.com/dask/distributed/actions/runs/2353756567
https://github.com/dask/distributed/actions/runs/2345977494

Failure on #6371
https://github.com/dask/distributed/runs/6517029607?check_suite_focus=true

self = <Popen: returncode: -9 args: ['/usr/share/miniconda3/envs/dask-distributed/b...>
timeout = 30

    def _wait(self, timeout):
        """Internal implementation of wait() on POSIX."""
        if self.returncode is not None:
            return self.returncode
    
        if timeout is not None:
            endtime = _time() + timeout
            # Enter a busy loop if we have a timeout.  This busy loop was
            # cribbed from Lib/threading.py in Thread.wait() at r71065.
            delay = 0.0005 # 500 us -> initial delay of 1 ms
            while True:
                if self._waitpid_lock.acquire(False):
                    try:
                        if self.returncode is not None:
                            break  # Another thread waited.
                        (pid, sts) = self._try_wait(os.WNOHANG)
                        assert pid == self.pid or pid == 0
                        if pid == self.pid:
                            self._handle_exitstatus(sts)
                            break
                    finally:
                        self._waitpid_lock.release()
                remaining = self._remaining_time(endtime)
                if remaining <= 0:
>                   raise TimeoutExpired(self.args, timeout)
E                   subprocess.TimeoutExpired: Command '['/usr/share/miniconda3/envs/dask-distributed/bin/dask-scheduler', '--dashboard-address', ':0']' timed out after 30 seconds

[gjoseph92]

We'll see what's happening in CI with #6461, but I'm wondering if this is just the typical thing where port 8787 is already in use. pytest --count 10 -n4 distributed/cli/tests/test_dask_scheduler.py::test_dashboard_port_zero will easily make this test fail. That's not with a TimeoutError, but maybe CI is so slow that the test times out before the process actually starts and stops?

[gjoseph92]

With #6461 in, we'll wait for a week to see if these subprocess.TimeoutExpired errors keep occurring; if not, we'll close this issue.

[crusaderky]

It doesn't seem to be effective: https://github.com/dask/distributed/runs/6825736211?check_suite_focus=true