fix(memory-storage): don't increment handledRequestCount when updating a handled request#3827
Merged
Conversation
…ng a handled request `RequestQueueClient.updateRequest()` incremented the queue's `handledRequestCount` whenever the updated request ended up handled, guarded only by `requestIsHandledAfterUpdate`. Updating a request that was already handled (handled -> handled, no state transition) therefore counted it a second time and permanently inflated the count reported by `getInfo()`. This is reachable in normal use: `RequestProvider.markRequestHandled()` calls `updateRequest()` unconditionally, and the higher-level provider only guards its own `assumedHandledCount`, not the memory-storage counter. Guard the increment with `isRequestHandledStateChanging` so it fires only on the pending -> handled transition, mirroring how the sibling `pendingRequestCount` is adjusted just above. Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
handledRequestCount when updating a handled request
barjin
approved these changes
Jul 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
RequestQueueClient.updateRequest()in@crawlee/memory-storagekeeps two sibling counters on the queue in sync:pendingRequestCountandhandledRequestCount. It derives three booleans from the request'sorderNo(nullwhen handled, a number when pending):isRequestHandledStateChanging = typeof existingRequest.orderNo !== typeof requestModel.orderNo— true only when the request actually crosses the pending/handled boundary.requestWasHandledBeforeUpdate = existingRequest.orderNo === nullrequestIsHandledAfterUpdate = requestModel.orderNo === nullpendingRequestCountis adjusted only insideif (isRequestHandledStateChanging), so it is correct.handledRequestCount, however, was incremented underif (requestIsHandledAfterUpdate)alone:Because that branch is not gated on the state actually changing, updating a request that was already handled (handled -> handled, no transition) counts it a second time and permanently inflates the value reported by
getInfo()/toRequestQueueInfo().This is reachable in normal use, not just in theory:
RequestProvider.markRequestHandled()callsclient.updateRequest({ ...request, handledAt })unconditionally, and the higher-level provider only guards its ownassumedHandledCountwithif (!queueOperationInfo.wasAlreadyHandled)— it never guards the memory-storage counter. So a secondmarkRequestHandled()(or any re-updateRequest()) on an already-handled request double-counts.Fix
Guard the increment with
isRequestHandledStateChangingas well, so it fires only on the pending -> handled transition, mirroring thependingRequestCountblock directly above:One line changed. The legitimate paths are untouched: the pending -> handled transition still increments, and
deleteRequest()'shandledRequestCount -= 1(on a handled request) is unchanged.Test
Extends the existing suite
packages/memory-storage/test/request-queue/handledRequestCount-should-update.test.tswith a case that adds an already-handled request, recordshandledRequestCount, updates that request again, and asserts the count is unchanged. It fails without the fix (expected 4 to deeply equal 3) and passes with it. The assertion captures the count after the add, so it is self-contained and does not depend on the running total from the earlier tests in the file.