Skip to content

fix(memory-storage): don't increment handledRequestCount when updating a handled request#3827

Merged
barjin merged 1 commit into
apify:masterfrom
anxkhn:patch-7
Jul 7, 2026
Merged

fix(memory-storage): don't increment handledRequestCount when updating a handled request#3827
barjin merged 1 commit into
apify:masterfrom
anxkhn:patch-7

Conversation

@anxkhn

@anxkhn anxkhn commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

RequestQueueClient.updateRequest() in @crawlee/memory-storage keeps two sibling counters on the queue in sync: pendingRequestCount and handledRequestCount. It derives three booleans from the request's orderNo (null when handled, a number when pending):

  • isRequestHandledStateChanging = typeof existingRequest.orderNo !== typeof requestModel.orderNo — true only when the request actually crosses the pending/handled boundary.
  • requestWasHandledBeforeUpdate = existingRequest.orderNo === null
  • requestIsHandledAfterUpdate = requestModel.orderNo === null

pendingRequestCount is adjusted only inside if (isRequestHandledStateChanging), so it is correct. handledRequestCount, however, was incremented under if (requestIsHandledAfterUpdate) alone:

if (requestIsHandledAfterUpdate) {
    existingQueueById.handledRequestCount += 1;
}

Because that branch is not gated on the state actually changing, updating a request that was already handled (handled -> handled, no transition) counts it a second time and permanently inflates the value reported by getInfo() / toRequestQueueInfo().

This is reachable in normal use, not just in theory: RequestProvider.markRequestHandled() calls client.updateRequest({ ...request, handledAt }) unconditionally, and the higher-level provider only guards its own assumedHandledCount with if (!queueOperationInfo.wasAlreadyHandled) — it never guards the memory-storage counter. So a second markRequestHandled() (or any re-updateRequest()) on an already-handled request double-counts.

Fix

Guard the increment with isRequestHandledStateChanging as well, so it fires only on the pending -> handled transition, mirroring the pendingRequestCount block directly above:

if (isRequestHandledStateChanging && requestIsHandledAfterUpdate) {
    existingQueueById.handledRequestCount += 1;
}

One line changed. The legitimate paths are untouched: the pending -> handled transition still increments, and deleteRequest()'s handledRequestCount -= 1 (on a handled request) is unchanged.

Test

Extends the existing suite packages/memory-storage/test/request-queue/handledRequestCount-should-update.test.ts with a case that adds an already-handled request, records handledRequestCount, updates that request again, and asserts the count is unchanged. It fails without the fix (expected 4 to deeply equal 3) and passes with it. The assertion captures the count after the add, so it is self-contained and does not depend on the running total from the earlier tests in the file.

…ng a handled request

`RequestQueueClient.updateRequest()` incremented the queue's
`handledRequestCount` whenever the updated request ended up handled,
guarded only by `requestIsHandledAfterUpdate`. Updating a request that
was already handled (handled -> handled, no state transition) therefore
counted it a second time and permanently inflated the count reported by
`getInfo()`.

This is reachable in normal use: `RequestProvider.markRequestHandled()`
calls `updateRequest()` unconditionally, and the higher-level provider
only guards its own `assumedHandledCount`, not the memory-storage
counter.

Guard the increment with `isRequestHandledStateChanging` so it fires
only on the pending -> handled transition, mirroring how the sibling
`pendingRequestCount` is adjusted just above.

Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
@barjin barjin changed the title fix(memory-storage): don't inflate handledRequestCount when re-updating a handled request fix(memory-storage): don't increment handledRequestCount when updating a handled request Jul 7, 2026

@barjin barjin left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, thank you @anxkhn !

@barjin barjin merged commit 4739d48 into apify:master Jul 7, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants