Saving 3000 objects in cloud code with saveAll takes more than 10 seconds

[SebC99]

Issue Description

For demo purposes, we need to clone 3000 objects and save them each time a demo session is started.
We use a cloudFunction to do this among other this and the saveAll operation of the newly cloned 3000 objects takes more than 10 seconds, without any impact of the batchSize parameter.

Steps to reproduce

1- query for 3000 objects
2- clone them
3- saveAll them

Expected Results

It should send a single (or a few batch) operation to the DB

Actual Outcome

It seems instead that the parse-server receives the requests in batch (from himself) but send them one by one to the DB, which then is very slow

Environment Setup

• Server

  • parse-server version: 3.1.3
  • Localhost or remote server? AWS ELB

• Database

  • MongoDB version: 3.6
  • Localhost or remote server? MongoAtlas

[dplewis]

BatchSize was added in 2.2.0 of JS SDK. Parse-Server 3.1.3 ships with 2.1.0. Please update your server to the latest version.

You might also want to try direct access. #5442

Post any performance improvements you see.

[SebC99]

Hello @dplewis, unfortunately upgrading to Parse-Server 3.2.3 didn't change anything. I've made some tests with batchSize (20, 100, 500, 1000) and it doesn't seems to have a lot of impact.

[dplewis]

How about direct access? Do you have any beforeSave / afterSave?

[SebC99]

beforeSave yes, but in that case they do nothing (I have a if (master) return condition in it

Direct Access is even worse as it's around 22 seconds instead of 10 seconds...

[SebC99]

In each cases the parse-server split the batch request in single operation rest command, so I guess it has something to do with this... In fact I don't really see the point of "batch" in that case
(either in parseServerRESTController handleRequest method, or in batch.js handleBatch method)

[dplewis]

I asked the same question, legacy from parse.com. parse-community/Parse-SDK-JS#701 (comment)

Try setting your batchSize to be greater than your number of objects so you don't hit the /batch endpoint like 5000. See if that works.

[SebC99]

It seems to help a little, but not that much (approx. 10%).
I can gain some time by removing all recursive unsaved object (I had 500 unsaved child in the 3000 objects, so saving them before helps a little)

[dplewis]

I'm going to reopen as a discussion. Maybe some DB experts can help or find some performance enhancement on the database layer.

You can checkout a previous discussion. #2654

[dplewis]

@SebC99 Do you have VERBOSE logs set on your server? Logs can slow down queries.

[SebC99]

No LOG_LEVEL is set on INFO in production
But we use Winston to log to AWS Cloudwatch too, I can try to deactivate it even if I'm pretty sure it's in background

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.