build: increase snapshot publish job queue confidence threshold (#19843)

We recently added a new step to the snapshot publish job. The
step will block the job until all previous snapshot publish
jobs that have been launched before complete.

This step relies on the CircleCI API which does not always
refresh immediatiely. This means that we could run into
a race condition where the queue does not detect previous
jobs. This has been observed with:

https://app.circleci.com/pipelines/github/angular/components/9618/workflows/52fef2b9-8059-4c00-8c2e-347b83ad1031/jobs/158528/steps

We fix this by increasing the confidence threshold of the
queue step, so that it will perform a second CircleCI request
after a delay to avoid the race condition in the most common
cases. It's a trade-off between stability and delay. We could
certainly increase the threshold further, but that will slow
down CI on publish branches, so we start experimenting with lower
threshold values first.

Author: devversion

.circleci/config.yml

@@ -440,7 +440,13 @@ jobs:
       # Ensures that we do not push the snapshot artifacts upstream until all previous
       # snapshot build jobs have completed. This helps avoiding conflicts when multiple
       # commits have been pushed (resulting in multiple concurrent snapshot publish jobs).
-      - queue/until_front_of_line
+      # We increase the confidence threshold so that it queries the CircleCI API a second time
+      # after a delay. We do this as the CircleCI API does not refresh immediately when a job
+      # completes/starts, and this will improve stability of the queue step. See source:
+      # https://github.com/eddiewebb/circleci-queue/commit/5d42add5bbcff5e8ac7fe189448a61fea98b0839.
+      - queue/until_front_of_line:
+          confidence: "2"
+
       - run: ./scripts/circleci/publish-snapshots.sh
       - *slack_notify_on_failure