sort the pending queue according to cost/priority

If multiple pieces of work are waiting in the pending queue, we can sort it according to
their priorities: higher priorities should be scheduled sooner.

They are more often than not wider than pure chains, and this should create more parallelism
opportunities earlier in the pipeline: a high priority piece of work represents more future
pieces of work down the line.

This is a scheduling tradeoff that behaves differently for each project, machine configuration,
amount of available parallelism at a given point in time, etc, but seems to help more often than
hinders, at low-core counts and with enough units of work to be done, so that there is jobserver
token contention where choosing a "better" piece of work to work on next is possible.

Author: lqd

src/cargo/core/compiler/job_queue.rs

@@ -583,6 +583,17 @@ impl<'cfg> DrainState<'cfg> {
             }
         }
 
+        // If multiple pieces of work are waiting in the pending queue, we can sort it according to
+        // their priorities: higher priorities should be scheduled sooner.
+        if self.pending_queue.len() > 1 {
+            self.pending_queue.sort_by(|(unit_a, _), (unit_b, _)| {
+                // We want to sort in descending order, as bigger values represent higher priorities.
+                let priority_a = self.queue.priority(unit_a);
+                let priority_b = self.queue.priority(unit_b);
+                priority_b.partial_cmp(&priority_a).unwrap()
+            });
+        }
+
         // Now that we've learned of all possible work that we can execute
         // try to spawn it so long as we've got a jobserver token which says
         // we're able to perform some parallel work.

src/cargo/util/dependency_queue.rs

@@ -170,6 +170,13 @@ impl<N: Hash + Eq + Clone, E: Eq + Hash + Clone, V> DependencyQueue<N, E, V> {
         self.dep_map.len()
     }
 
+    /// Returns the relative priority of a node. Higher priorities should be scheduled sooner.
+    /// Currently computed as the transitive cost of the given node: its own, plus the cost of its
+    /// reverse dependencies.
+    pub(crate) fn priority(&self, node: &N) -> usize {
+        self.priority[node]
+    }
+
     /// Indicate that something has finished.
     ///
     /// Calling this function indicates that the `node` has produced `edge`. All