🐛 [firestore-counter] Unable to read shards during aggregation round, skipping... Error: 10 ABORTED: Aborted due to cross-transaction contention.

[mcarriere]

[READ] Step 1: Are you in the right place?

It might be related to something deeper in firestore but it's hard for me to know.

[REQUIRED] Step 2: Describe your configuration

• Extension name: firestore-counter
• Extension version: 0.2.11
• Configuration values (redact info where appropriate):
  • Document path for internal state: _firebase_ext_/sharded_counter
  • Frequency for controllerCore function to be run: 1

[REQUIRED] Step 3: Describe the problem

We've been using this extension without any problem for years and this week it started to become extremely slow. We're talking 10 minutes to update 2 counters with 80 and 40 increments respectively.

We've also saw a huge increase in the number of logs from ext-firestore-counter-worker

Unable to read shards during aggregation round, skipping... Error: 10 ABORTED: Aborted due to cross-transaction contention. This occurs when multiple transactions attempt to access the same data, requiring Firestore to abort at least one in order to enforce serializability.

Steps to reproduce:

In our case whether it's our dev or prod environment it seems like even around a 100 increments will trigger it.

Here's the history of this log in GCP, we can see we basically had 0 logs before and now we get hundreds.

Our traffic or volume of data has not increased recently.

Expected result

Working counters and no error logs

Actual result

A lot of contention errors slowing down/breaking the extension.