Don't end computations until cluster is truly idle

[crusaderky]

• XREF Fine performance metrics: apportion to Computations #7776

Testing for Scheduler.total_occupancy to be zero is not a great pick for declaring a computation finished and starting the next one.
This PR will not create a new computation as long as there are

• any long-running tasks
• any tasks with unmet resource constraints
• zero workers, but queued tasks

This reduces (but doesn't fully eliminate) use cases where you have overlapping Computation objects (read #7776 for more).

[github-actions]

Unit Test Results

See test report for an extended history of previous test failures. This is useful for diagnosing flaky tests.

       26 files  ±  0         26 suites  ±0   15h 12m 20s ⏱️ - 43m 22s
  3 606 tests +  3    3 496 ✔️ +  2     105 💤 ±0  5 ❌ +1 
45 629 runs  +39  43 455 ✔️ +38  2 169 💤 +1  5 ❌ ±0 

For more details on these failures, see this check.

Results for commit 0cf5145. ± Comparison against base commit e887fde.

This pull request removes 2 and adds 5 tests. Note that renamed tests count towards both.

distributed.tests.test_scheduler ‑ test_computations
distributed.tests.test_scheduler ‑ test_computations_futures

distributed.tests.test_computations ‑ test_computations
distributed.tests.test_computations ‑ test_computations_futures
distributed.tests.test_computations ‑ test_computations_long_running
distributed.tests.test_computations ‑ test_computations_no_resources
distributed.tests.test_computations ‑ test_computations_no_workers

♻️ This comment has been updated with latest results.

[hendrikmakait]

Thanks, @crusaderky! This logic is much more sensible.

[hendrikmakait]

nit: Is there a way we can stress the semantic difference between idle and fully_idle (e.g., by renaming fully_idle -> is_idle or idle -> idle_workers)?

[crusaderky]

renamed fully_idle -> is_idle