Skip to content

fix(model): preserve unsaved documents after ordered bulkSave errors - #16533

Merged
vkarpov15 merged 3 commits into
Automattic:masterfrom
IbrahimHafez1:fix/ordered-bulk-save-state
Sep 28, 2026
Merged

vkarpov15 merged 3 commits into
Automattic:masterfrom
IbrahimHafez1:fix/ordered-bulk-save-state

Conversation

@IbrahimHafez1

@IbrahimHafez1 IbrahimHafez1 commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Fixes #16532.

After an ordered bulk write fails, MongoDB leaves later operations unattempted. bulkSave() currently treats every document without an individual write error as successful. It resets pending changes, marks unattempted inserts as no longer new, and runs their post-save hooks. The bulk promise rejects, but retrying those document instances can then omit writes that never reached the database.

Track the original input document index for each generated operation and handle ordered and unordered errors separately:

  • Ordered: map the first failed operation directly to its input index and use slice() without an indexOf() search to select the preceding documents for successful-write handling. Treat an omitted ordered option as ordered.
  • Unordered: exclude only the explicitly failed operations. Preserve the existing 25-document threshold: direct error lookup for smaller batches and a plain object keyed by input document index for larger batches.
  • No write errors: handle the input documents directly, without allocating a failure lookup.

Operation indexes must map back to documents because unchanged documents produce no operation and distinct instances can share an _id. The optional internal mapping array is populated while operations are built. No public API changes.

Examples

For new documents [A, B, C], where A and B violate the same unique index:

  • A is inserted.
  • B fails with a duplicate-key error.
  • MongoDB does not attempt C because writes are ordered by default.
  • Before: C becomes isNew === false, loses modified state, and receives a post-save hook despite being absent from the database. Retrying C does not insert it.
  • After: C retains its pending state, receives no success hook, and can be retried.

The linked issue contains a complete reproduction. Six new tests cover small and 25-document ordered batches, pending updates with unchanged inputs, repeated _id values, mixed inserts/updates, hooks, retries, and unordered failures followed by successful writes. The existing 25-document unordered regression remains covered. Tests use explicit declarations rather than a loop.

Validation

Current revision 780de3f, Windows, Node 22.17.1, MongoDB 8.2.3, MongoDB driver 7.6.0:

  • Full test/model.test.js: 403 passing, 15 pending.
  • Additional local validation script: 26 ordered/unordered scenarios passed, covering unchanged inputs, inserts, updates, mixed operations, repeated IDs, hooks, retries, and batch sizes around the 25-document threshold.
  • Direct mapping check confirms mixed insert/update operations preserve their original input indexes when unchanged inputs are skipped.
  • ESLint for both PR files and git diff --check: passed.
  • The full repository suite was not rerun locally for this revision.

Based on master 940de5d2c62dc29104b664a0e15be8d4aae38e91. Duplicate checks included issues and open/closed PRs about ordered bulkSave, skipped writes, retries, and duplicate IDs; related work is distinguished in the issue.

@vkarpov15 vkarpov15 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right that there's a bug in the ordered case, however the fix presented in this PR seems to break the unordered case. Also removes a performance optimization.

Ideal fix would be to handle the ordered and unordered cases separately. if (options.ordered) { /* successfulDocs are everything up to first write index error */ } else { /* check explicit indexes for unordered case */ }

Comment thread test/model.test.js Outdated
Comment thread lib/model.js Outdated
Comment thread lib/model.js Outdated
Comment thread lib/model.js Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

lib/model.js uses the wrong write-error index property, causing failed and unattempted documents to be treated as successful.

Review effort: Lite
Findings: 1 High severity

Open (1)
What changed in this PR

Fixes bulkSave() handling after ordered bulk-write errors so unattempted documents remain retryable.

Changes:

  • Tracks generated operations back to their source documents.
  • Preserves state and skips success hooks for failed or unattempted writes.
  • Adds regression coverage for ordered failures, retries, hooks, duplicate IDs, and mixed operations.
File Description
test/​model.test.js Adds ordered bulkSave() regression coverage.
lib/​model.js Maps bulk operation indexes to documents and classifies failures.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread lib/model.js Outdated

@vkarpov15 vkarpov15 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One minor improvement requested but otherwise looks good.

Comment thread lib/model.js Outdated
}
const writeOperation = { insertOne: { document } };
utils.injectTimestampsOption(writeOperation.insertOne, options.timestamps);
documentsToWrite?.push(document);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of returning the documentsToWrite, we should return the documentIndexes - documentIndexes.push(i) here. That way, the ordered case doesn't need an indexOf(), can just slice(0, documentIndexes[writeErrors[0].index]); and the unordered case can use a POJO instead of a Set() for faster initialization.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implemented in 780de3f. The mapping now stores original input indexes for both inserts and updates. Ordered handling slices directly at the mapped index, and unordered handling uses a plain object for larger batches while retaining the existing 25-document threshold.

Verified against MongoDB 8.2.3: the full model suite passes (403 passing, 15 pending), plus 26 additional ordered/unordered scenarios covering unchanged inputs, mixed writes, repeated IDs, hooks, retries, and threshold boundaries. ESLint and diff checks also pass.

A small local microbenchmark favored the object lookup for larger/error-heavy batches, but not every sparse small-batch case, so I am not claiming a universal speedup. The ordered path does eliminate the extra indexOf() scan.

@vkarpov15 vkarpov15 added this to the 9.10.3 milestone Sep 28, 2026
@vkarpov15
vkarpov15 merged commit 5ed18f4 into Automattic:master Sep 28, 2026
26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Ordered bulkSave marks unattempted documents as saved after a write error

3 participants