Skip to content

Model.insertMany with lean: true is very slow compared to native driver #14372

Closed
@berkay-dincer

Description

@berkay-dincer

Prerequisites

  • I have written a descriptive issue title

Mongoose version

6.6.1

Node.js version

16.x

MongoDB version

6.0.1

Operating system

macOS

Operating system version (i.e. 20.04, 11.3, 10)

No response

Issue

Hey 👋

I am currently working on a piece of functionality that requires bulk inserting of documents using Mongoose. In the process, I've noticed a considerable performance difference between Model.collection.insertMany(docs) method and the Model.insertMany(docs, {lean: true}) method with the {lean: true} flag. Based on the documentation, I understand that the {lean: true} option helps to skip the process of hydrating and validating the documents before insertion. I had thought that this would effectively make these two methods roughly equivalent in terms of performance.

However, upon performing benchmark tests, I found that Model.collection.insertMany is significantly faster than Model.insertMany({lean: true}). I am hoping to better understand why this is the case. I am also attaching a script to replicate the issue.

const mongoose = require('mongoose');

const Schema = mongoose.Schema;

const MyModelSchema = new Schema({
    field1: String,
    field2: Number,
    field3: Boolean,
    field4: Date,
    field5: Array
});

const MyModel = mongoose.model('MyModel', MyModelSchema);

async function clearCollection() {
    await MyModel.deleteMany({});
}
async function mongooseInsert(docs) {
    console.time("Mongoose Insert");
    await MyModel.insertMany(docs, { ordered: false, lean: true });
    console.timeEnd("Mongoose Insert");
}

async function nativeInsert(docs) {
    console.time("Native Insert");
    await MyModel.collection.insertMany(docs, { ordered: false });
    console.timeEnd("Native Insert");
}

async function chunkedNativeInsert(docs) {
    console.time("Chunked Native Insert");
    for (let i = 0; i < docs.length; i += 1000) {
        await MyModel.collection.insertMany(docs.slice(i, i + 1000), { ordered: false });
    }
    console.timeEnd("Chunked Native Insert");

}
async function main() {
    await mongoose.connect("mongodb://localhost:27017/test");
    let documents = [];

    for (let i = 0; i < 50000; i++) {
        documents.push({
            field1: `String${i}`,
            field2: i,
            field3: i % 2 === 0,
            field4: new Date(),
            field5: [i, i+1, i+2]
        });
    }
    await mongooseInsert(documents);
    await clearCollection();
    await chunkedNativeInsert(documents);
    await clearCollection();
    await nativeInsert(documents);
    await clearCollection();
}


main();

Thanks!

Metadata

Metadata

Assignees

No one assigned

    Labels

    helpThis issue can likely be resolved in GitHub issues. No bug fixes, features, or docs necessaryhelp wanted

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions