Added ignoreDuplicatesOnInsert configuration

[rozza]

Requires unordered inserts otherwise this configuration is ignored

SPARK-451

[Copilot]

Pull request overview

This PR adds support for ignoring duplicate key errors during insert operations through a new ignoreDuplicatesOnInsert configuration option. This feature requires unordered bulk inserts to function correctly; otherwise, the configuration is ignored with a warning logged.

• Adds new configuration ignoreDuplicatesOnInsert in WriteConfig with validation logic
• Implements exception handling in MongoDataWriter to catch and ignore duplicate key errors (error code 11000) when enabled
• Includes comprehensive unit and integration tests covering various scenarios

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File	Description
src/main/java/com/mongodb/spark/sql/connector/config/WriteConfig.java	Adds IGNORE_DUPLICATES_ON_INSERT_CONFIG constant, default value, and ignoreDuplicatesOnInsert() method with validation logic that ensures unordered inserts and INSERT operation type
src/main/java/com/mongodb/spark/sql/connector/write/MongoDataWriter.java	Adds exception handling to catch MongoBulkWriteException and ignore it when all errors are duplicate key errors (code 11000) and the feature is enabled
src/test/java/com/mongodb/spark/sql/connector/config/MongoConfigTest.java	Adds unit test testWriteConfigIgnoreDuplicatesOnInsert() to verify configuration behavior under different settings
src/integrationTest/java/com/mongodb/spark/sql/connector/write/MongoSparkConnectorWriteTest.java	Adds integration test testIgnoreDuplicates() to verify end-to-end functionality with both _id-based and unique index-based duplicates

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.