readPreference secondaryPreferred causes "Unable to create case insensitive username index: not master"

[sunshineo]

New Issue Checklist

• I am not disclosing a vulnerability.
• I am not just asking a question.
• I have searched through existing issues.
• I can reproduce the issue with the latest version of Parse Server.

Issue Description

When Mongo URI has readPreference set to secondaryPreferred (and connecting to a Mongo Atlas cluster) we get the following error:

[13973] parse-server running on http://localhost:1337/parse
warn: Unable to create case insensitive username index:  not master {"operationTime":"6903658739996295169","ok":0,"code":10107,"codeName":"NotMaster","$clusterTime":{"clusterTime":"6903658739996295169","signature":{"hash":"/wM8w3Ym3vb446s04YZqDHx5TTw=","keyId":"6847700517334810656"}},"name":"MongoError","stack":"MongoError: not master\n    at MessageStream.messageHandler (/usr/local/lib/node_modules/parse-server/node_modules/mongodb/lib/cmap/connection.js:268:20)\n    at MessageStream.emit (events.js:314:20)\n    at MessageStream.EventEmitter.emit (domain.js:486:12)\n    at processIncomingData (/usr/local/lib/node_modules/parse-server/node_modules/mongodb/lib/cmap/message_stream.js:144:12)\n    at MessageStream._write (/usr/local/lib/node_modules/parse-server/node_modules/mongodb/lib/cmap/message_stream.js:42:5)\n    at writeOrBuffer (_stream_writable.js:352:12)\n    at MessageStream.Writable.write (_stream_writable.js:303:10)\n    at TLSSocket.ondata (_stream_readable.js:717:22)\n    at TLSSocket.emit (events.js:314:20)\n    at TLSSocket.EventEmitter.emit (domain.js:486:12)"}
MongoError: not master
    at MessageStream.messageHandler (/usr/local/lib/node_modules/parse-server/node_modules/mongodb/lib/cmap/connection.js:268:20)
    at MessageStream.emit (events.js:314:20)
    at MessageStream.EventEmitter.emit (domain.js:486:12)
    at processIncomingData (/usr/local/lib/node_modules/parse-server/node_modules/mongodb/lib/cmap/message_stream.js:144:12)
    at MessageStream._write (/usr/local/lib/node_modules/parse-server/node_modules/mongodb/lib/cmap/message_stream.js:42:5)
    at writeOrBuffer (_stream_writable.js:352:12)
    at MessageStream.Writable.write (_stream_writable.js:303:10)
    at TLSSocket.ondata (_stream_readable.js:717:22)
    at TLSSocket.emit (events.js:314:20)
    at TLSSocket.EventEmitter.emit (domain.js:486:12) {
  operationTime: Timestamp { _bsontype: 'Timestamp', low_: 1, high_: 1607383308 },
  ok: 0,
  code: 10107,
  codeName: 'NotMaster',
  '$clusterTime': {
    clusterTime: Timestamp { _bsontype: 'Timestamp', low_: 1, high_: 1607383308 },
    signature: { hash: [Binary], keyId: [Long] }
  },
  [Symbol(errorLabels)]: Set(1) { 'RetryableWriteError' }
}

Steps to reproduce

Create a cluster with 3 nodes on MongoDB Atlas. I feel a cluster anywhere can reproduce

// This works
parse-server --appId myappid --masterKey masterkey --databaseURI "mongodb+srv:/<username>:<password>@cluster0.b2mxc.mongodb.net/<db-name>?retryWrites=true&w=3"

// Add &readPreference=secondaryPreferred&slaveOk=true to the end of the URI fails
parse-server --appId myappid --masterKey masterkey --databaseURI "mongodb+srv:/<username>:<password>@cluster0.b2mxc.mongodb.net/<db-name>?retryWrites=true&w=3&readPreference=secondaryPreferred&slaveOk=true"

Environment

Server

• Parse Server version: 4.4.0
• Operating system: Mac

Database

• System (MongoDB or Postgres): MongoDB
• Database version: 4.2.10
• Local or remote host (MongoDB Atlas, mLab, AWS, Azure, Google Cloud, etc): MongoDB Atlas

Logs

[13973] parse-server running on http://localhost:1337/parse
warn: Unable to create case insensitive username index:  not master {"operationTime":"6903658739996295169","ok":0,"code":10107,"codeName":"NotMaster","$clusterTime":{"clusterTime":"6903658739996295169","signature":{"hash":"/wM8w3Ym3vb446s04YZqDHx5TTw=","keyId":"6847700517334810656"}},"name":"MongoError","stack":"MongoError: not master\n    at MessageStream.messageHandler (/usr/local/lib/node_modules/parse-server/node_modules/mongodb/lib/cmap/connection.js:268:20)\n    at MessageStream.emit (events.js:314:20)\n    at MessageStream.EventEmitter.emit (domain.js:486:12)\n    at processIncomingData (/usr/local/lib/node_modules/parse-server/node_modules/mongodb/lib/cmap/message_stream.js:144:12)\n    at MessageStream._write (/usr/local/lib/node_modules/parse-server/node_modules/mongodb/lib/cmap/message_stream.js:42:5)\n    at writeOrBuffer (_stream_writable.js:352:12)\n    at MessageStream.Writable.write (_stream_writable.js:303:10)\n    at TLSSocket.ondata (_stream_readable.js:717:22)\n    at TLSSocket.emit (events.js:314:20)\n    at TLSSocket.EventEmitter.emit (domain.js:486:12)"}
MongoError: not master
    at MessageStream.messageHandler (/usr/local/lib/node_modules/parse-server/node_modules/mongodb/lib/cmap/connection.js:268:20)
    at MessageStream.emit (events.js:314:20)
    at MessageStream.EventEmitter.emit (domain.js:486:12)
    at processIncomingData (/usr/local/lib/node_modules/parse-server/node_modules/mongodb/lib/cmap/message_stream.js:144:12)
    at MessageStream._write (/usr/local/lib/node_modules/parse-server/node_modules/mongodb/lib/cmap/message_stream.js:42:5)
    at writeOrBuffer (_stream_writable.js:352:12)
    at MessageStream.Writable.write (_stream_writable.js:303:10)
    at TLSSocket.ondata (_stream_readable.js:717:22)
    at TLSSocket.emit (events.js:314:20)
    at TLSSocket.EventEmitter.emit (domain.js:486:12) {
  operationTime: Timestamp { _bsontype: 'Timestamp', low_: 1, high_: 1607383308 },
  ok: 0,
  code: 10107,
  codeName: 'NotMaster',
  '$clusterTime': {
    clusterTime: Timestamp { _bsontype: 'Timestamp', low_: 1, high_: 1607383308 },
    signature: { hash: [Binary], keyId: [Long] }
  },
  [Symbol(errorLabels)]: Set(1) { 'RetryableWriteError' }
}

[dplewis]

I believe this has been fixed in the latest version of the mongo driver.

https://jira.mongodb.org/browse/NODE-2784

We are looking into updating the driver.

[mtrezza]

Here are some workaround suggestions:

• Remove the readPreference from the connection string and instead set it in the Parse Server configuration as driver option. Untested.
• Set the readPreference as option of the Parse Query where you want to override the driver / connection string read preference. Untested.
• Manually create the index in the DB.

[sunshineo]

I tried that. Same problem.

…

On Wed, Dec 9, 2020 at 10:08 AM Manuel ***@***.***> wrote: A workaround may be - I haven't tested it - to remove the readPreference from the connection string and instead set it in the Parse Server configuration as driver option. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#7048 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIF47TTJGNMSGM5HCVEBL3ST64LRANCNFSM4URFMBNQ> .

[sunshineo]

@mtrezza manually create the index in the DB does not work. As a matter of fact, when I test removed the readPrefrence settings, it worked and created the index. But when add back the readPreference, code probably is trying to ensure index exists and still fails even the index is already there

parse-server/src/Controllers/DatabaseController.js

Line 1628 in ca1b782

this.adapter.ensureIndex(

[mtrezza]

manually create the index in the DB does not work.

I cannot reproduce this with an existing Parse Server deployment by adding readPreference=secondaryPreferred to the connection string. The server works just fine.

[sunshineo]

@mtrezza is your DB a cluster with 3 nodes (mine on MongoDB Atlas)?

[mtrezza]

Yes, I tried it with a 3 node (PSS) replica set. Just to be sure I also tried to use the exactly same connection string parameters as yours, still works. Maybe you want to post the created index?

You could also try to upgrade the Parse Server mongodb dependency to 3.6.3.

[sunshineo]

Upgrade to 3.6.3 definitely works. I guess I have to install from my repo for a while. I'm creating a PR