Nightscout crash: "MongoDB connection failed! Double check the MONGODB_URI setting in Heroku"

[philipgo]

Describe the bug
Every day or so my Nightscout crashes with the error message "MongoDB connection failed! Double check the MONGODB_URI setting in Heroku."

The problem appears to be with Nightscout (or AndroidAPS??), I do not need to restart the MongoDB instance and other Nightscout instances continue to work with the same MONGODB_URI without any issues. However I do need to manually restart the Nightscout instance.

Expected behavior
Nightscout should not crash or if it does it should recover automatically.

Screenshots

Your setup information
MongoDB self hosted on a private server, Nightscout Docker (latest dev, but same issue on master) hosted on a different private server. My uploader is AndroidAPS 2.8.1.1

Logs

nightscout_1  | DEDUPE: Devicestatus exact match
nightscout_1  | /opt/app/node_modules/mongodb/lib/utils.js:668
nightscout_1  |           throw error;
nightscout_1  |           ^
nightscout_1  |
nightscout_1  | TypeError: Cannot read property '0' of undefined
nightscout_1  |     at findResult (/opt/app/lib/server/websocket.js:414:32)
nightscout_1  |     at /opt/app/node_modules/mongodb/lib/utils.js:665:9
nightscout_1  |     at handleCallback (/opt/app/node_modules/mongodb/lib/utils.j                                                                                                                                                             s:102:55)
nightscout_1  |     at /opt/app/node_modules/mongodb/lib/cursor.js:834:20
nightscout_1  |     at /opt/app/node_modules/mongodb/lib/core/cursor.js:736:9
nightscout_1  |     at Object.callback (/opt/app/node_modules/mongodb/lib/core/c                                                                                                                                                             ursor.js:432:11)
nightscout_1  |     at Timeout._onTimeout (/opt/app/node_modules/mongodb/lib/cor                                                                                                                                                             e/sdam/topology.js:443:25)
nightscout_1  |     at listOnTimeout (internal/timers.js:554:17)
nightscout_1  |     at processTimers (internal/timers.js:497:7)
nightscout_1  | Executing startBoot
nightscout_1  | Executing checkNodeVersion
nightscout_1  | Node LTS version v14.15.3 is supported
nightscout_1  | Executing checkEnv
nightscout_1  | Executing augmentSettings
nightscout_1  | Parsing config URL from IMPORT_CONFIG failed
nightscout_1  | Executing checkSettings
nightscout_1  | Checking settings
nightscout_1  | Executing setupStorage
nightscout_1  | Setting up new connection to MongoDB
nightscout_1  | Error connecting to MongoDB: {"name":"MongoServerSelectionError"                                                                                                                                                             ,"reason":{"type":"Single","setName":null,"maxSetVersion":null,"maxElectionId":n                                                                                                                                                             ull,"servers":{},"stale":false,"compatible":true,"compatibilityError":null,"logi                                                                                                                                                             calSessionTimeoutMinutes":null,"heartbeatFrequencyMS":10000,"localThresholdMS":1                                                                                                                                                             5,"commonWireVersion":null}} - retrying in 3 sec
nightscout_1  | ERROR CONNECTING TO MONGO Error: MongoDB connection failed! Doub                                                                                                                                                             le check the MONGODB_URI setting in Heroku.
nightscout_1  |     at connect_with_retry (/opt/app/lib/storage/mongo-storage.js                                                                                                                                                             :70:28)
nightscout_1  |     at processTicksAndRejections (internal/process/task_queues.j                                                                                                                                                             s:93:5)
nightscout_1  | Mongo Storage system ready
nightscout_1  | Executing setupAuthorization
nightscout_1  | Executing setupInternals
nightscout_1  | Executing ensureIndexes
nightscout_1  | Executing setupListeners
nightscout_1  | Executing setupBridge
nightscout_1  | Executing setupMMConnect
nightscout_1  | Executing finishBoot
nightscout_1  | Boot event processing completed
nightscout_1  | Security settings: INSECURE_USE_HTTP= true , SECURE_HSTS_HEADER=                                                                                                                                                              false
nightscout_1  | Listening on port 1337 b2d084333cfc
nightscout_1  | Error connecting to MongoDB: {"name":"MongoServerSelectionError"                                                                                                                                                             ,"reason":{"type":"Single","setName":null,"maxSetVersion":null,"maxElectionId":n                                                                                                                                                             ull,"servers":{},"stale":false,"compatible":true,"compatibilityError":null,"logi                                                                                                                                                             calSessionTimeoutMinutes":null,"heartbeatFrequencyMS":10000,"localThresholdMS":1                                                                                                                                                             5,"commonWireVersion":null}} - retrying in 6 sec
nightscout_1  | Error connecting to MongoDB: {"name":"MongoServerSelectionError"                                                                                                                                                             ,"reason":{"type":"Single","setName":null,"maxSetVersion":null,"maxElectionId":n                                                                                                                                                             ull,"servers":{},"stale":false,"compatible":true,"compatibilityError":null,"logi                                                                                                                                                             calSessionTimeoutMinutes":null,"heartbeatFrequencyMS":10000,"localThresholdMS":1                                                                                                                                                             5,"commonWireVersion":null}} - retrying in 9 sec
nightscout_1  | Error connecting to MongoDB: {"name":"MongoServerSelectionError"                                                                                                                                                             ,"reason":{"type":"Single","setName":null,"maxSetVersion":null,"maxElectionId":n                                                                                                                                                             ull,"servers":{},"stale":false,"compatible":true,"compatibilityError":null,"logi                                                                                                                                                             calSessionTimeoutMinutes":null,"heartbeatFrequencyMS":10000,"localThresholdMS":1                                                                                                                                                             5,"commonWireVersion":null}} - retrying in 12 sec
nightscout_1  | Error connecting to MongoDB: {"name":"MongoServerSelectionError"                                                                                                                                                             ,"reason":{"type":"Single","setName":null,"maxSetVersion":null,"maxElectionId":n                                                                                                                                                             ull,"servers":{},"stale":false,"compatible":true,"compatibilityError":null,"logi                                                                                                                                                             calSessionTimeoutMinutes":null,"heartbeatFrequencyMS":10000,"localThresholdMS":1                                                                                                                                                             5,"commonWireVersion":null}} - retrying in 15 sec
nightscout_1  | Error connecting to MongoDB: {"name":"MongoServerSelectionError"                                                                                                                                                             ,"reason":{"type":"Single","setName":null,"maxSetVersion":null,"maxElectionId":n                                                                                                                                                             ull,"servers":{},"stale":false,"compatible":true,"compatibilityError":null,"logi                                                                                                                                                             calSessionTimeoutMinutes":null,"heartbeatFrequencyMS":10000,"localThresholdMS":1                                                                                                                                                             5,"commonWireVersion":null}} - retrying in 18 sec
nightscout_1  | Error connecting to MongoDB: {"name":"MongoServerSelectionError"                                                                                                                                                             ,"reason":{"type":"Single","setName":null,"maxSetVersion":null,"maxElectionId":n                                                                                                                                                             ull,"servers":{},"stale":false,"compatible":true,"compatibilityError":null,"logi                                                                                                                                                             calSessionTimeoutMinutes":null,"heartbeatFrequencyMS":10000,"localThresholdMS":1                                                                                                                                                             5,"commonWireVersion":null}} - retrying in 21 sec
nightscout_1  | Error connecting to MongoDB: {"name":"MongoServerSelectionError"                                                                                                                                                             ,"reason":{"type":"Single","setName":null,"maxSetVersion":null,"maxElectionId":n                                                                                                                                                             ull,"servers":{},"stale":false,"compatible":true,"compatibilityError":null,"logi                                                                                                                                                             calSessionTimeoutMinutes":null,"heartbeatFrequencyMS":10000,"localThresholdMS":1                                                                                                                                                             5,"commonWireVersion":null}} - retrying in 24 sec
nightscout_1  | Error connecting to MongoDB: {"name":"MongoServerSelectionError"                                                                                                                                                             ,"reason":{"type":"Single","setName":null,"maxSetVersion":null,"maxElectionId":n                                                                                                                                                             ull,"servers":{},"stale":false,"compatible":true,"compatibilityError":null,"logi                                                                                                                                                             calSessionTimeoutMinutes":null,"heartbeatFrequencyMS":10000,"localThresholdMS":1                                                                                                                                                             5,"commonWireVersion":null}} - retrying in 27 sec
nightscout_1  | Successfully established a connected to MongoDB
nightscout_1  | Mongo user role seems ok: [ { role: 'readWrite', db: 'nightscout'                                                                                                                                                              } ]
nightscout_1  | Mongo Storage system ready

[sulkaharo]

Right - would it be possible for you to deploy this branch, so we'll see what the error is. Looks like you're hitting a previously unreported bug and sadly the current build doesn't log the error that's happening. #6777

[sulkaharo]

The branch is here: https://github.com/nightscout/cgm-remote-monitor/tree/websocket_dedupe_error

[philipgo]

I have deployed the websocket_dedupe_error branch and will wait for it to crash again and get back to you. Thank you for looking into this!

[philipgo]

Here are the logs of a crash running your websocket_dedupe_errorbranch:
(The MongoDB logs show nothing exceptional)

nightscout_1  | devicestatus insertion error:  getaddrinfo EAI_AGAIN mongo.xyz.de
nightscout_1  | DEDUPE: Exact match
nightscout_1  | /opt/app/node_modules/mongodb/lib/utils.js:668
nightscout_1  |           throw error;
nightscout_1  |           ^
nightscout_1  |
nightscout_1  | TypeError: Cannot read property '0' of undefined
nightscout_1  |     at findResult (/opt/app/lib/server/websocket.js:326:32)
nightscout_1  |     at /opt/app/node_modules/mongodb/lib/utils.js:665:9
nightscout_1  |     at handleCallback (/opt/app/node_modules/mongodb/lib/utils.j                                                                                                                                                             s:102:55)
nightscout_1  |     at /opt/app/node_modules/mongodb/lib/cursor.js:834:20
nightscout_1  |     at /opt/app/node_modules/mongodb/lib/core/cursor.js:736:9
nightscout_1  |     at Object.callback (/opt/app/node_modules/mongodb/lib/core/c                                                                                                                                                             ursor.js:432:11)
nightscout_1  |     at Timeout._onTimeout (/opt/app/node_modules/mongodb/lib/cor                                                                                                                                                             e/sdam/topology.js:443:25)
nightscout_1  |     at listOnTimeout (internal/timers.js:554:17)
nightscout_1  |     at processTimers (internal/timers.js:497:7)
nightscout_1  | Executing startBoot
nightscout_1  | Executing checkNodeVersion
nightscout_1  | Node LTS version v14.15.3 is supported
nightscout_1  | Executing checkEnv
nightscout_1  | Executing augmentSettings
nightscout_1  | Parsing config URL from IMPORT_CONFIG failed
nightscout_1  | Executing checkSettings
nightscout_1  | Checking settings
nightscout_1  | Executing setupStorage
nightscout_1  | Setting up new connection to MongoDB
nightscout_1  | Error connecting to MongoDB: {"name":"MongoServerSelectionError"                                                                                                                                                             ,"reason":{"type":"Single","setName":null,"maxSetVersion":null,"maxElectionId":n                                                                                                                                                             ull,"servers":{},"stale":false,"compatible":true,"compatibilityError":null,"logi                                                                                                                                                             calSessionTimeoutMinutes":null,"heartbeatFrequencyMS":10000,"localThresholdMS":1                                                                                                                                                             5,"commonWireVersion":null}} - retrying in 3 sec
nightscout_1  | ERROR CONNECTING TO MONGO Error: MongoDB connection failed! Doub                                                                                                                                                             le check the MONGODB_URI setting in Heroku.
nightscout_1  |     at connect_with_retry (/opt/app/lib/storage/mongo-storage.js                                                                                                                                                             :70:28)
nightscout_1  |     at processTicksAndRejections (internal/process/task_queues.j                                                                                                                                                             s:93:5)
nightscout_1  | Mongo Storage system ready
nightscout_1  | Executing setupAuthorization
nightscout_1  | Executing setupInternals
nightscout_1  | Executing ensureIndexes
nightscout_1  | Executing setupListeners
nightscout_1  | Executing setupBridge
nightscout_1  | Executing setupMMConnect
nightscout_1  | Executing finishBoot
nightscout_1  | Boot event processing completed
nightscout_1  | Security settings: INSECURE_USE_HTTP= true , SECURE_HSTS_HEADER=                                                                                                                                                              false
nightscout_1  | Listening on port 1337 dace99d2d2ce
nightscout_1  | Error connecting to MongoDB: {"name":"MongoServerSelectionError"                                                                                                                                                             ,"reason":{"type":"Single","setName":null,"maxSetVersion":null,"maxElectionId":n                                                                                                                                                             ull,"servers":{},"stale":false,"compatible":true,"compatibilityError":null,"logi                                                                                                                                                             calSessionTimeoutMinutes":null,"heartbeatFrequencyMS":10000,"localThresholdMS":1                                                                                                                                                             5,"commonWireVersion":null}} - retrying in 6 sec
nightscout_1  | Error connecting to MongoDB: {"name":"MongoServerSelectionError"                                                                                                                                                             ,"reason":{"type":"Single","setName":null,"maxSetVersion":null,"maxElectionId":n                                                                                                                                                             ull,"servers":{},"stale":false,"compatible":true,"compatibilityError":null,"logi                                                                                                                                                             calSessionTimeoutMinutes":null,"heartbeatFrequencyMS":10000,"localThresholdMS":1                                                                                                                                                             5,"commonWireVersion":null}} - retrying in 9 sec
nightscout_1  | Error connecting to MongoDB: {"name":"MongoServerSelectionError"                                                                                                                                                             ,"reason":{"type":"Single","setName":null,"maxSetVersion":null,"maxElectionId":n                                                                                                                                                             ull,"servers":{},"stale":false,"compatible":true,"compatibilityError":null,"logi                                                                                                                                                             calSessionTimeoutMinutes":null,"heartbeatFrequencyMS":10000,"localThresholdMS":1                                                                                                                                                             5,"commonWireVersion":null}} - retrying in 12 sec
nightscout_1  | Error connecting to MongoDB: {"name":"MongoServerSelectionError"                                                                                                                                                             ,"reason":{"type":"Single","setName":null,"maxSetVersion":null,"maxElectionId":n                                                                                                                                                             ull,"servers":{},"stale":false,"compatible":true,"compatibilityError":null,"logi                                                                                                                                                             calSessionTimeoutMinutes":null,"heartbeatFrequencyMS":10000,"localThresholdMS":1                                                                                                                                                             5,"commonWireVersion":null}} - retrying in 15 sec
nightscout_1  | Error connecting to MongoDB: {"name":"MongoServerSelectionError"                                                                                                                                                             ,"reason":{"type":"Single","setName":null,"maxSetVersion":null,"maxElectionId":n                                                                                                                                                             ull,"servers":{},"stale":false,"compatible":true,"compatibilityError":null,"logi                                                                                                                                                             calSessionTimeoutMinutes":null,"heartbeatFrequencyMS":10000,"localThresholdMS":1                                                                                                                                                             5,"commonWireVersion":null}} - retrying in 18 sec
nightscout_1  | Error connecting to MongoDB: {"name":"MongoServerSelectionError"                                                                                                                                                             ,"reason":{"type":"Single","setName":null,"maxSetVersion":null,"maxElectionId":n                                                                                                                                                             ull,"servers":{},"stale":false,"compatible":true,"compatibilityError":null,"logi                                                                                                                                                             calSessionTimeoutMinutes":null,"heartbeatFrequencyMS":10000,"localThresholdMS":1                                                                                                                                                             5,"commonWireVersion":null}} - retrying in 21 sec
nightscout_1  | Error connecting to MongoDB: {"name":"MongoServerSelectionError"                                                                                                                                                             ,"reason":{"type":"Single","setName":null,"maxSetVersion":null,"maxElectionId":n                                                                                                                                                             ull,"servers":{},"stale":false,"compatible":true,"compatibilityError":null,"logi                                                                                                                                                             calSessionTimeoutMinutes":null,"heartbeatFrequencyMS":10000,"localThresholdMS":1                                                                                                                                                             5,"commonWireVersion":null}} - retrying in 24 sec
nightscout_1  | Error connecting to MongoDB: {"name":"MongoServerSelectionError"                                                                                                                                                             ,"reason":{"type":"Single","setName":null,"maxSetVersion":null,"maxElectionId":n                                                                                                                                                             ull,"servers":{},"stale":false,"compatible":true,"compatibilityError":null,"logi                                                                                                                                                             calSessionTimeoutMinutes":null,"heartbeatFrequencyMS":10000,"localThresholdMS":1                                                                                                                                                             5,"commonWireVersion":null}} - retrying in 27 sec
nightscout_1  | Error connecting to MongoDB: {"name":"MongoServerSelectionError"                                                                                                                                                             ,"reason":{"type":"Single","setName":null,"maxSetVersion":null,"maxElectionId":n                                                                                                                                                             ull,"servers":{},"stale":false,"compatible":true,"compatibilityError":null,"logi                                                                                                                                                             calSessionTimeoutMinutes":null,"heartbeatFrequencyMS":10000,"localThresholdMS":1                                                                                                                                                             5,"commonWireVersion":null}} - retrying in 30 sec
nightscout_1  | Error connecting to MongoDB: {"name":"MongoServerSelectionError"                                                                                                                                                             ,"reason":{"type":"Single","setName":null,"maxSetVersion":null,"maxElectionId":n                                                                                                                                                             ull,"servers":{},"stale":false,"compatible":true,"compatibilityError":null,"logi                                                                                                                                                             calSessionTimeoutMinutes":null,"heartbeatFrequencyMS":10000,"localThresholdMS":1                                                                                                                                                             5,"commonWireVersion":null}} - retrying in 33 sec
nightscout_1  | Successfully established a connected to MongoDB
nightscout_1  | Mongo user role seems ok: [ { role: 'readWrite', db: 'nightscout'                                                                                                                                                              } ]
nightscout_1  | Mongo Storage system ready

I have just deployed the latest and got the same error straight away.

[sulkaharo]

Dammit, @philipgo can you update to latest in the branch, there were two additional error spots like this, which now also capture the error. Btw how do you host the site? Based on the URL this is your own setup? Which Mongo version do you have? @lukebrogan Same to you - how are you hosting the site?

Hello!
I am deploying to Heroku 20 stack.
URL is https://cgm-luke.herokuapp.com/
Let me know if you need anything else.
Many thanks.

[philipgo]

@sulkaharo Thank you, Sulka, already deployed.
My MongoDB version is v4.2.11. MongoDB is self hosted on a private server, Nightscout Docker hosted on a different private server behind an nginx reverse proxy. My uploader is AndroidAPS 2.8.1.1. I have never found anything in the MongoDB logs which could point to the cause.

@lukebrogan Your error may not be the same as mine. If your site never works: are you sure your MongoDB_URI is correct? My site works fine for hours or sometimes days, then it crashes.

Yeah you are right - it was an error in the MongoDB_URI string, all working for me now 👍

[philipgo]

Here are the logs of another crash running the latest websocket_dedupe_error branch.

Some googling showed others are seeing similar errors in connection with useUnifiedTopology: true which was changed here recently.

�[36mnightscout_1  |�[0m �[32mWS: �[0mdbAdd client ID:  PbvoXQ5Gg6ZgtauSAABP  data:  {
�[36mnightscout_1  |�[0m   collection: 'treatments',
�[36mnightscout_1  |�[0m   data: {
�[36mnightscout_1  |�[0m     eventType: 'Temp Basal',
�[36mnightscout_1  |�[0m     duration: 60,
�[36mnightscout_1  |�[0m     absolute: 0,
�[36mnightscout_1  |�[0m     rate: 0,
�[36mnightscout_1  |�[0m     created_at: '2021-01-23T09:00:13Z',
�[36mnightscout_1  |�[0m     enteredBy: 'openaps://AndroidAPS',
�[36mnightscout_1  |�[0m     NSCLIENT_ID: '1611392413159'
�[36mnightscout_1  |�[0m   }
�[36mnightscout_1  |�[0m }
�[36mnightscout_1  |�[0m MongoServerSelectionError: getaddrinfo EAI_AGAIN mongo.xyz.de
�[36mnightscout_1  |�[0m     at Timeout._onTimeout (/opt/app/node_modules/mongodb/lib/core/sdam/topology.js:438:30)
�[36mnightscout_1  |�[0m     at listOnTimeout (internal/timers.js:554:17)
�[36mnightscout_1  |�[0m     at processTimers (internal/timers.js:497:7) {
�[36mnightscout_1  |�[0m   reason: TopologyDescription {
�[36mnightscout_1  |�[0m     type: 'Single',
�[36mnightscout_1  |�[0m     setName: null,
�[36mnightscout_1  |�[0m     maxSetVersion: null,
�[36mnightscout_1  |�[0m     maxElectionId: null,
�[36mnightscout_1  |�[0m     servers: Map(1) {
�[36mnightscout_1  |�[0m       'mongo.xyz.de:27017' => [ServerDescription]
�[36mnightscout_1  |�[0m     },
�[36mnightscout_1  |�[0m     stale: false,
�[36mnightscout_1  |�[0m     compatible: true,
�[36mnightscout_1  |�[0m     compatibilityError: null,
�[36mnightscout_1  |�[0m     logicalSessionTimeoutMinutes: null,
�[36mnightscout_1  |�[0m     heartbeatFrequencyMS: 10000,
�[36mnightscout_1  |�[0m     localThresholdMS: 15,
�[36mnightscout_1  |�[0m     commonWireVersion: 8
�[36mnightscout_1  |�[0m   }
�[36mnightscout_1  |�[0m }
�[36mnightscout_1  |�[0m devicestatus insertion error:  getaddrinfo EAI_AGAIN mongo.xyz.de
�[36mnightscout_1  |�[0m got data-received event, requesting reload
�[36mnightscout_1  |�[0m Problem loading entries
�[36mnightscout_1  |�[0m Problem loading activity data
�[36mnightscout_1  |�[0m Problem loading database stats
�[36mnightscout_1  |�[0m Load Complete:
�[36mnightscout_1  |�[0m 	 sgvs:576, profiles:1, food:2
�[36mnightscout_1  |�[0m data loaded: reloading sandbox data and updating plugins
�[36mnightscout_1  |�[0m OpenAPS hasn't reported a loop yet
�[36mnightscout_1  |�[0m MongoServerSelectionError: getaddrinfo EAI_AGAIN mongo.xyz.de
�[36mnightscout_1  |�[0m     at Timeout._onTimeout (/opt/app/node_modules/mongodb/lib/core/sdam/topology.js:438:30)
�[36mnightscout_1  |�[0m     at listOnTimeout (internal/timers.js:554:17)
�[36mnightscout_1  |�[0m     at processTimers (internal/timers.js:497:7) {
�[36mnightscout_1  |�[0m   reason: TopologyDescription {
�[36mnightscout_1  |�[0m     type: 'Single',
�[36mnightscout_1  |�[0m     setName: null,
�[36mnightscout_1  |�[0m     maxSetVersion: null,
�[36mnightscout_1  |�[0m     maxElectionId: null,
�[36mnightscout_1  |�[0m     servers: Map(1) {
�[36mnightscout_1  |�[0m       'mongo.xyz.de:27017' => [ServerDescription]
�[36mnightscout_1  |�[0m     },
�[36mnightscout_1  |�[0m     stale: false,
�[36mnightscout_1  |�[0m     compatible: true,
�[36mnightscout_1  |�[0m     compatibilityError: null,
�[36mnightscout_1  |�[0m     logicalSessionTimeoutMinutes: null,
�[36mnightscout_1  |�[0m     heartbeatFrequencyMS: 10000,
�[36mnightscout_1  |�[0m     localThresholdMS: 15,
�[36mnightscout_1  |�[0m     commonWireVersion: 8
�[36mnightscout_1  |�[0m   }
�[36mnightscout_1  |�[0m }
�[36mnightscout_1  |�[0m devicestatus insertion error:  getaddrinfo EAI_AGAIN mongo.xyz.de
�[36mnightscout_1  |�[0m MongoServerSelectionError: getaddrinfo EAI_AGAIN mongo.xyz.de
�[36mnightscout_1  |�[0m     at Timeout._onTimeout (/opt/app/node_modules/mongodb/lib/core/sdam/topology.js:438:30)
�[36mnightscout_1  |�[0m     at listOnTimeout (internal/timers.js:554:17)
�[36mnightscout_1  |�[0m     at processTimers (internal/timers.js:497:7) {
�[36mnightscout_1  |�[0m   reason: TopologyDescription {
�[36mnightscout_1  |�[0m     type: 'Single',
�[36mnightscout_1  |�[0m     setName: null,
�[36mnightscout_1  |�[0m     maxSetVersion: null,
�[36mnightscout_1  |�[0m     maxElectionId: null,
�[36mnightscout_1  |�[0m     servers: Map(1) {
�[36mnightscout_1  |�[0m       'mongo.xyz.de:27017' => [ServerDescription]
�[36mnightscout_1  |�[0m     },
�[36mnightscout_1  |�[0m     stale: false,
�[36mnightscout_1  |�[0m     compatible: true,
�[36mnightscout_1  |�[0m     compatibilityError: null,
�[36mnightscout_1  |�[0m     logicalSessionTimeoutMinutes: null,
�[36mnightscout_1  |�[0m     heartbeatFrequencyMS: 10000,
�[36mnightscout_1  |�[0m     localThresholdMS: 15,
�[36mnightscout_1  |�[0m     commonWireVersion: 8
�[36mnightscout_1  |�[0m   }
�[36mnightscout_1  |�[0m }
�[36mnightscout_1  |�[0m �[32mWS: �[0mdbAdd client ID:  PbvoXQ5Gg6ZgtauSAABP  data:  {
�[36mnightscout_1  |�[0m   collection: 'devicestatus',
�[36mnightscout_1  |�[0m   data: {
�[36mnightscout_1  |�[0m     device: 'openaps://samsung SM-G960F',
�[36mnightscout_1  |�[0m     pump: {
�[36mnightscout_1  |�[0m       clock: '2021-01-23T08:55:30Z',
�[36mnightscout_1  |�[0m       reservoir: 187,
�[36mnightscout_1  |�[0m       status: [Object],
�[36mnightscout_1  |�[0m       extended: [Object],
�[36mnightscout_1  |�[0m       battery: [Object]
�[36mnightscout_1  |�[0m     },
�[36mnightscout_1  |�[0m     openaps: { suggested: [Object], iob: [Object] },
�[36mnightscout_1  |�[0m     uploaderBattery: 88,
�[36mnightscout_1  |�[0m     created_at: '2021-01-23T08:59:58Z',
�[36mnightscout_1  |�[0m     configuration: {},
�[36mnightscout_1  |�[0m     NSCLIENT_ID: '1611392398671'
�[36mnightscout_1  |�[0m   }
�[36mnightscout_1  |�[0m }
�[36mnightscout_1  |�[0m �[32mWS: �[0mdbAdd client ID:  PbvoXQ5Gg6ZgtauSAABP  data:  {
�[36mnightscout_1  |�[0m   collection: 'treatments',
�[36mnightscout_1  |�[0m   data: {
�[36mnightscout_1  |�[0m     eventType: 'Temp Basal',
�[36mnightscout_1  |�[0m     duration: 60,
�[36mnightscout_1  |�[0m     absolute: 0,
�[36mnightscout_1  |�[0m     rate: 0,
�[36mnightscout_1  |�[0m     created_at: '2021-01-23T09:00:13Z',
�[36mnightscout_1  |�[0m     enteredBy: 'openaps://AndroidAPS',
�[36mnightscout_1  |�[0m     NSCLIENT_ID: '1611392413159'
�[36mnightscout_1  |�[0m   }
�[36mnightscout_1  |�[0m }
�[36mnightscout_1  |�[0m tick 2021-01-23T08:58:41.228Z
�[36mnightscout_1  |�[0m Problem loading entries
�[36mnightscout_1  |�[0m Problem loading activity data
�[36mnightscout_1  |�[0m Problem loading database stats
�[36mnightscout_1  |�[0m Load Complete:
�[36mnightscout_1  |�[0m 	 sgvs:576, profiles:1, food:2
�[36mnightscout_1  |�[0m data loaded: reloading sandbox data and updating plugins
�[36mnightscout_1  |�[0m OpenAPS hasn't reported a loop yet
�[36mnightscout_1  |�[0m �[32mWS: �[0mdbAdd client ID:  PbvoXQ5Gg6ZgtauSAABP  data:  {
�[36mnightscout_1  |�[0m   collection: 'devicestatus',
�[36mnightscout_1  |�[0m   data: {
�[36mnightscout_1  |�[0m     device: 'openaps://samsung SM-G960F',
�[36mnightscout_1  |�[0m     pump: {
�[36mnightscout_1  |�[0m       clock: '2021-01-23T08:55:30Z',
�[36mnightscout_1  |�[0m       reservoir: 187,
�[36mnightscout_1  |�[0m       status: [Object],
�[36mnightscout_1  |�[0m       extended: [Object],
�[36mnightscout_1  |�[0m       battery: [Object]
�[36mnightscout_1  |�[0m     },
�[36mnightscout_1  |�[0m     openaps: { suggested: [Object], iob: [Object] },
�[36mnightscout_1  |�[0m     uploaderBattery: 88,
�[36mnightscout_1  |�[0m     created_at: '2021-01-23T08:59:58Z',
�[36mnightscout_1  |�[0m     configuration: {},
�[36mnightscout_1  |�[0m     NSCLIENT_ID: '1611392398671'
�[36mnightscout_1  |�[0m   }
�[36mnightscout_1  |�[0m }
�[36mnightscout_1  |�[0m �[32mWS: �[0mdbAdd client ID:  PbvoXQ5Gg6ZgtauSAABP  data:  {
�[36mnightscout_1  |�[0m   collection: 'treatments',
�[36mnightscout_1  |�[0m   data: {
�[36mnightscout_1  |�[0m     eventType: 'Temp Basal',
�[36mnightscout_1  |�[0m     duration: 60,
�[36mnightscout_1  |�[0m     absolute: 0,
�[36mnightscout_1  |�[0m     rate: 0,
�[36mnightscout_1  |�[0m     created_at: '2021-01-23T09:00:13Z',
�[36mnightscout_1  |�[0m     enteredBy: 'openaps://AndroidAPS',
�[36mnightscout_1  |�[0m     NSCLIENT_ID: '1611392413159'
�[36mnightscout_1  |�[0m   }
�[36mnightscout_1  |�[0m }
�[36mnightscout_1  |�[0m MongoServerSelectionError: getaddrinfo EAI_AGAIN mongo.xyz.de
�[36mnightscout_1  |�[0m     at Timeout._onTimeout (/opt/app/node_modules/mongodb/lib/core/sdam/topology.js:438:30)
�[36mnightscout_1  |�[0m     at listOnTimeout (internal/timers.js:554:17)
�[36mnightscout_1  |�[0m     at processTimers (internal/timers.js:497:7) {
�[36mnightscout_1  |�[0m   reason: TopologyDescription {
�[36mnightscout_1  |�[0m     type: 'Single',
�[36mnightscout_1  |�[0m     setName: null,
�[36mnightscout_1  |�[0m     maxSetVersion: null,
�[36mnightscout_1  |�[0m     maxElectionId: null,
�[36mnightscout_1  |�[0m     servers: Map(1) {
�[36mnightscout_1  |�[0m       'mongo.xyz.de:27017' => [ServerDescription]
�[36mnightscout_1  |�[0m     },
�[36mnightscout_1  |�[0m     stale: false,
�[36mnightscout_1  |�[0m     compatible: true,
�[36mnightscout_1  |�[0m     compatibilityError: null,
�[36mnightscout_1  |�[0m     logicalSessionTimeoutMinutes: null,
�[36mnightscout_1  |�[0m     heartbeatFrequencyMS: 10000,
�[36mnightscout_1  |�[0m     localThresholdMS: 15,
�[36mnightscout_1  |�[0m     commonWireVersion: 8
�[36mnightscout_1  |�[0m   }
�[36mnightscout_1  |�[0m }
�[36mnightscout_1  |�[0m devicestatus insertion error:  getaddrinfo EAI_AGAIN mongo.xyz.de
�[36mnightscout_1  |�[0m MongoServerSelectionError: getaddrinfo EAI_AGAIN mongo.xyz.de
�[36mnightscout_1  |�[0m     at Timeout._onTimeout (/opt/app/node_modules/mongodb/lib/core/sdam/topology.js:438:30)
�[36mnightscout_1  |�[0m     at listOnTimeout (internal/timers.js:554:17)
�[36mnightscout_1  |�[0m     at processTimers (internal/timers.js:497:7) {
�[36mnightscout_1  |�[0m   reason: TopologyDescription {
�[36mnightscout_1  |�[0m     type: 'Single',
�[36mnightscout_1  |�[0m     setName: null,
�[36mnightscout_1  |�[0m     maxSetVersion: null,
�[36mnightscout_1  |�[0m     maxElectionId: null,
�[36mnightscout_1  |�[0m     servers: Map(1) {
�[36mnightscout_1  |�[0m       'mongo.xyz.de:27017' => [ServerDescription]
�[36mnightscout_1  |�[0m     },
�[36mnightscout_1  |�[0m     stale: false,
�[36mnightscout_1  |�[0m     compatible: true,
�[36mnightscout_1  |�[0m     compatibilityError: null,
�[36mnightscout_1  |�[0m     logicalSessionTimeoutMinutes: null,
�[36mnightscout_1  |�[0m     heartbeatFrequencyMS: 10000,
�[36mnightscout_1  |�[0m     localThresholdMS: 15,
�[36mnightscout_1  |�[0m     commonWireVersion: 8
�[36mnightscout_1  |�[0m   }
�[36mnightscout_1  |�[0m }
�[36mnightscout_1  |�[0m �[32mWS: �[0mdbAdd client ID:  PbvoXQ5Gg6ZgtauSAABP  data:  {
�[36mnightscout_1  |�[0m   collection: 'devicestatus',
�[36mnightscout_1  |�[0m   data: {
�[36mnightscout_1  |�[0m     device: 'openaps://samsung SM-G960F',
�[36mnightscout_1  |�[0m     pump: {
�[36mnightscout_1  |�[0m       clock: '2021-01-23T08:55:30Z',
�[36mnightscout_1  |�[0m       reservoir: 187,
�[36mnightscout_1  |�[0m       status: [Object],
�[36mnightscout_1  |�[0m       extended: [Object],
�[36mnightscout_1  |�[0m       battery: [Object]
�[36mnightscout_1  |�[0m     },
�[36mnightscout_1  |�[0m     openaps: { suggested: [Object], iob: [Object] },
�[36mnightscout_1  |�[0m     uploaderBattery: 88,
�[36mnightscout_1  |�[0m     created_at: '2021-01-23T08:59:58Z',
�[36mnightscout_1  |�[0m     configuration: {},
�[36mnightscout_1  |�[0m     NSCLIENT_ID: '1611392398671'
�[36mnightscout_1  |�[0m   }
�[36mnightscout_1  |�[0m }
�[36mnightscout_1  |�[0m �[32mWS: �[0mdbAdd client ID:  PbvoXQ5Gg6ZgtauSAABP  data:  {
�[36mnightscout_1  |�[0m   collection: 'treatments',
�[36mnightscout_1  |�[0m   data: {
�[36mnightscout_1  |�[0m     eventType: 'Temp Basal',
�[36mnightscout_1  |�[0m     duration: 60,
�[36mnightscout_1  |�[0m     absolute: 0,
�[36mnightscout_1  |�[0m     rate: 0,
�[36mnightscout_1  |�[0m     created_at: '2021-01-23T09:00:13Z',
�[36mnightscout_1  |�[0m     enteredBy: 'openaps://AndroidAPS',
�[36mnightscout_1  |�[0m     NSCLIENT_ID: '1611392413159'
�[36mnightscout_1  |�[0m   }
�[36mnightscout_1  |�[0m }
�[36mnightscout_1  |�[0m Problem loading entries
�[36mnightscout_1  |�[0m Problem loading activity data
�[36mnightscout_1  |�[0m Problem loading database stats
�[36mnightscout_1  |�[0m Load Complete:
�[36mnightscout_1  |�[0m 	 sgvs:576, profiles:1, food:2
�[36mnightscout_1  |�[0m data loaded: reloading sandbox data and updating plugins
�[36mnightscout_1  |�[0m OpenAPS hasn't reported a loop yet
�[36mnightscout_1  |�[0m MongoServerSelectionError: getaddrinfo EAI_AGAIN mongo.xyz.de
�[36mnightscout_1  |�[0m     at Timeout._onTimeout (/opt/app/node_modules/mongodb/lib/core/sdam/topology.js:438:30)
�[36mnightscout_1  |�[0m     at listOnTimeout (internal/timers.js:554:17)
�[36mnightscout_1  |�[0m     at processTimers (internal/timers.js:497:7) {
�[36mnightscout_1  |�[0m   reason: TopologyDescription {
�[36mnightscout_1  |�[0m     type: 'Single',
�[36mnightscout_1  |�[0m     setName: null,
�[36mnightscout_1  |�[0m     maxSetVersion: null,
�[36mnightscout_1  |�[0m     maxElectionId: null,
�[36mnightscout_1  |�[0m     servers: Map(1) {
�[36mnightscout_1  |�[0m       'mongo.xyz.de:27017' => [ServerDescription]
�[36mnightscout_1  |�[0m     },
�[36mnightscout_1  |�[0m     stale: false,
�[36mnightscout_1  |�[0m     compatible: true,
�[36mnightscout_1  |�[0m     compatibilityError: null,
�[36mnightscout_1  |�[0m     logicalSessionTimeoutMinutes: null,
�[36mnightscout_1  |�[0m     heartbeatFrequencyMS: 10000,
�[36mnightscout_1  |�[0m     localThresholdMS: 15,
�[36mnightscout_1  |�[0m     commonWireVersion: 8
�[36mnightscout_1  |�[0m   }
�[36mnightscout_1  |�[0m }
�[36mnightscout_1  |�[0m devicestatus insertion error:  getaddrinfo EAI_AGAIN mongo.xyz.de
�[36mnightscout_1  |�[0m MongoServerSelectionError: getaddrinfo EAI_AGAIN mongo.xyz.de
�[36mnightscout_1  |�[0m     at Timeout._onTimeout (/opt/app/node_modules/mongodb/lib/core/sdam/topology.js:438:30)
�[36mnightscout_1  |�[0m     at listOnTimeout (internal/timers.js:554:17)
�[36mnightscout_1  |�[0m     at processTimers (internal/timers.js:497:7) {
�[36mnightscout_1  |�[0m   reason: TopologyDescription {
�[36mnightscout_1  |�[0m     type: 'Single',
�[36mnightscout_1  |�[0m     setName: null,
�[36mnightscout_1  |�[0m     maxSetVersion: null,
�[36mnightscout_1  |�[0m     maxElectionId: null,
�[36mnightscout_1  |�[0m     servers: Map(1) {
�[36mnightscout_1  |�[0m       'mongo.xyz.de:27017' => [ServerDescription]
�[36mnightscout_1  |�[0m     },
�[36mnightscout_1  |�[0m     stale: false,
�[36mnightscout_1  |�[0m     compatible: true,
�[36mnightscout_1  |�[0m     compatibilityError: null,
�[36mnightscout_1  |�[0m     logicalSessionTimeoutMinutes: null,
�[36mnightscout_1  |�[0m     heartbeatFrequencyMS: 10000,
�[36mnightscout_1  |�[0m     localThresholdMS: 15,
�[36mnightscout_1  |�[0m     commonWireVersion: 8
�[36mnightscout_1  |�[0m   }
�[36mnightscout_1  |�[0m }

[sulkaharo]

@philipgo Given the logged error is simply that the connection has failed, I'm assuming this has to be something in your Mongo setup, as this hasn't been reported by others. Not the Mongo Driver in Nightscout has been configured to auto-reconnect and this seems to be also fail in this case. FWIW I have a similar setup in Digital Ocean and have never had the Mongo connection fail in this manner. Wonder if there's a Mongo forum somewhere that could help you?

[philipgo]

I am still experimenting with different setups.

• a Nightscout version without Refactor mongo-storage.js  #6589 worked fine
• a current dev Nightscout version removing both timeouts at

  cgm-remote-monitor/lib/storage/mongo-storage.js

  Line 29 in c91ec78

  connectTimeoutMS: timeout,

  worked fine as well

My MongoDB instance is super fast, so I would have no idea where timeouts might come from. However, is there a reason setting them to much shorter values than their defaults? Just removing the two lines will let them default to socketTimeoutMS = 360000 and connectTimeoutMS=10000.

One note: One of your changes in https://github.com/nightscout/cgm-remote-monitor/pull/6777/files causes the MONGODB_URI string to be displayed in full in the html error message on the web, including user name and password.

[sulkaharo]

Right, increasing the timeouts makes sense. Note the connection time can be different from the query performance and if what you're seeing is caused by a connection or socket timing out, that indicates there's something odd either in the networking between the servers or the Mongo server is misbehaving and sleeping, or some other issue that slows down the handshake.

[inventor96]

I was having a very similar issue on my local machine doing development work. The error message said the type was ReplicaSetNoPrimary. I had it pointing to my live database on MongoDB Cloud. While troubleshooting, I determined that the DNS record for my MongoDB instance didn't exist (I checked locally and by using some DNS lookup services online). This missing DNS record issue has come and gone periodically.

I chatted MongoDB support, and the support rep gave me another connection string to work with instead of the one I was using on my live Nightscout instance on Heroku, which specifically set the name of each shard (rather than referencing the cluster as a whole) and changed a couple options.

Original connection string:

mongodb+srv://username:password@cluster-bp7z8gq4.ngvbd.mongodb.net/heroku_bp7z8gq4?retryWrites=true&w=majority

New connection string:

mongodb://username:password@cluster-bp7z8gq4-shard-00-00.ngvbd.mongodb.net:27017,cluster-bp7z8gq4-shard-00-01.ngvbd.mongodb.net:27017,cluster-bp7z8gq4-shard-00-02.ngvbd.mongodb.net:27017/heroku_bp7z8gq4?ssl=true&replicaSet=atlas-llw8hn-shard-0&authSource=admin&retryWrites=true&w=majority

Using this new connection string, I have been able to connect to my database on MongoDB Cloud from my local machine.