ReceiveSMS sketch hangs after SMS received

[joshbober]

In the Arduino example sketch ReceiveSMS.ino, when a message is received by the device, the response gets cut off and the sketch hangs. This was discussed partially in #50, but is a separate issue from what has been discussed there. #80 hypothesized that it was an issue with the '/' character, but the issue was closed without clear resolution (using another library may have solved this). With Serial debugging active, I see that the response returned over the serial monitor is cut off abruptly, like this:

14:32:05.683 -> AT+CMGL="REC UNREAD"
14:32:05.683 -> +CMGL: 6,"REC UNREAD","+1##########",,"21

After this, the code gets stuck in a while() loop, waiting for MODEM.ready() to return something other than 0. This is where the problem becomes more clear. Incoming uart data is stored in the RX ring buffer, which has a size of 64 ints. The first 64 characters of the response are being stored in this buffer, and the rest seems to be dropped. I verified this by adding a Serial.print statement with the size of the RX buffer after each character is read. This means that the response code at the end is not stored or read in the modem's poll() function, causing the return value for "ready" to always remain 0.

I have verified that this is NOT an issue with processing the '/' character; if you change the AT command in the library from "AT+CMGL="REC UNREAD"" to just "AT+CGML" (identical commands, see AT commands reference), the serial monitor will print more of the date/time, like this:

15:45:05.700 -> AT+CMGL
15:45:05.746 -> +CMGL: 11,"REC UNREAD","+1##########",,"21/09/08,12:45

I see that buffering is handled differently for the NBClient functions (the NBSocketBuffer files), where the buffer is extended to 512 ints. Does a similar solution need to be applied here to handle SMS message reading? Is there something that I am missing in my understanding?

Steps to replicate:

1. Upload ReceiveSMS.ino to an NB1500 with an activated SIM card that has SMS service. Set debugging to true in the NB constructor- nbAccess(true)
2. Wait for the device to initialize and be in the "Waiting for messages" state
3. Send an SMS message to the device

Link to gist of Serial Monitor output following the 3 steps above (with phone number redacted)
https://gist.github.com/joshbober/9a9a01e35c540c0f4d4771276bfee0be

Other relevant information:

• My device firmware is updated to the latest release 05.12
• Using the SerialSARAPassthrough sketch and u-blox tool m-center, the AT+CGML command returns the full expected response

[CptHolzschnauz]

Just a short reply. Sorry, i was unclear with "other library". I meant i compiled my sketch with another computer with a pristine MKRNB lib (without any changes made by me ;=) and then it worked. Thats what i meant with "another" lib.

You are right, the / char was coincidence.

I just replicate the error, but with another board (and SIM) with the same sketch it works perfektly.

There is a bug and it's going to be a tough one...

PS: I loaded the serialPasstrough (the modem keeping powered on), with
AT+CMGL="REC UNREAD" there was nothing
after AT+CMGL="ALL" it listed 13 sms, i deleted all by hand, after that AT+CMGL="REC UNREAD" worked.
Loaded back the sketch, the sms lib failed with the same behaviour.

Joshbober, you wrote
After this, the code gets stuck in a while() loop, waiting for MODEM.ready() to return something other than 0.
Where do you mean? Is the buffer handling not within NB_SMS.cpp, 350 ?
Let's track down that bug!

[CptHolzschnauz]

PPS: I reproduced the bug with
07:57:59.400 -> AT+CMGL="REC UNREAD"
07:57:59.400 -> +CMGL: 1,"REC UNREAD","+41xxxxxxxxx",,"21

Manually, it works perfectly:
08:08:19.720 -> AT+CMGL="REC UNREAD"
08:08:19.720 -> +CMGL: 2,"REC UNREAD","+41xxxxxxxxx",,"21/09/10,08:08:16+08"
08:08:19.756 -> Test
08:08:19.756 ->
08:08:19.756 -> OK

Where do you count 64 characters?
And again, stoped at the "/" char, despite this is more then coincidence? Different results with the same AT command AT+CMGL="REC UNREAD" send by serialPASS and by the lib, it must be some issue with the buffer handler of the lib!?

This issue comes out of a sudden and vanish the same way...strange flaky thing.
Or is it just happend to Josh and me?

[joshbober]

Thanks for looking into this, Cpt.

First off, I have also seen that manually sending the AT+CMGL command using the passthrough sketch works perfectly.

Where do you mean? Is the buffer handling not within NB_SMS.cpp, 350?
I think the issue actually lies in the handling of AT responses in Modem.cpp, poll() function (starting at line 274). I did some digging into the "_uart->available()" code, which took me to the Arduino files Uart.cpp, Uart.h, and RingBuffer.h. The Uart class uses a "ring buffer" to store incoming data, and the available() command returns the number of elements stored in the buffer. This check is done in Modem.cpp, 276, where it tests to see if there is data in the ring buffer that can be read. If there is data, the next char is read from the uart buffer, printed to Serial monitor (if debug enabled), then added to the Modem's "_buffer" variable (lines 277-283).

Where do you count 64 characters?
In the file RingBuffer.h (C:\Users\[user]\AppData\Local\Arduino15\packages\arduino\hardware\samd\1.8.11\cores\arduino\api\RingBuffer.h), line 33, the defined SERIAL_BUFFER_SIZE is 64. This could be modified elsewhere, or not even used, but I'm not sure, it just confirmed my assumption.
Using the information above, I added a Serial.print command to return the number of elements in the Uart buffer as each character gets read and printed to Serial monitor. Here's a link to the Gist of that code, and the output (it is a little hard to read since it prints the number after each character).

You can see from here that the "AT+CGML="REC UNREAD"" command is also stored in this buffer, which explains why the total characters printed is 64 and not only 40. A way to test this is to remove "="REC UNREAD"" from "NB_SMS.cpp", line 226; the "AT+CGML" command by default will return the unread messages (see section 9.12.4 of SARA-R4 series AT commands manual). 13 more characters of the response get printed since the part of command that is removed is 13 characters long.

I think the response for the AT+CGML command is longer than expected for most AT responses, and thus the Modem.cpp code is not equipped to process it. Only the first 64 characters of the total command+response get stored in the buffer, so the result code (OK, ERROR, NO CARRIER, or CME ERROR) is not found. The value of _ready remains 0, and the code remains forever in the while loop found in NB_SMS.cpp, line 283 (this can be verified by putting a Serial.print statement within the while loop and watching on the Serial monitor).

Sorry for the wall of text, hopefully this gets us closer to an understanding of the issue.

[CptHolzschnauz]

No worries! I'm glad i found a mate with this issue!
I got a feeling the cause is much simpler (just a feeling i know ;=)
Main question: Why was it working perfectly the other day?

[joshbober]

I fully support the theory that it's a simple fix that I'm just not understanding, hopefully someone who understands or contributed to this library can chime in.

As for your question- I have no clue. I have not gotten it to work even once yet. I tried using a fresh library install on the same computer, with a fresh compile (temp folder emptied), to no avail.

I noticed you said in another comment that it worked for you on a different computer; I will attempt that as well and report my results.

[joshbober]

UPDATE:

Tested on different computer, but I have found the real culprit: the Arduino SAMD boards library.

Versions 1.8.10 and 1.8.11 exhibit this buffer/hanging issue.
Downgrading to version 1.8.9 or below makes it work (except one of the characters gets eaten in the sketch Serial monitor output on a different line). Link to SM output

@CptHolzschnauz please test with the downgraded SAMD library and let me know if you see the same result.

If this is the case, and the issue is not with the MKRNB library, but with the SAMD, should there be a disclaimer or something to not use v1.8.10+ of the SAMD core until they fix it? I will pursue submitting this issue over there if it hasn't already been brought up.

[CptHolzschnauz]

Oh man, you are right! I can confirm.
I was the discoverer of the big SPI bug in 1.8.10, so i never updated. That's why it works compiled on my old computer.
With my new one, the 1.8.11 was automatically installed on so compiled on that machine it didn't work.
My gut feeling was right.. ;=)

[joshbober]

Thank you for confirming! I'm glad that we got to the bottom of this, and that your gut feeling was right.

I just created an issue over on the SAMD repo, hopefully they can provide some insight. @per1234, not sure if this should be closed here, with the stipulation that this is an issue with dependencies rather than the MKRNB lib itself.

[CptHolzschnauz]

Thank you for finding! My gut was half right, i think you were on the right path to the bug deep down in the core...

[CptHolzschnauz]

Update: I had also a strange behaviour with the same symptons with the nbscanner class, call getSignalStrength() with core 1.8.9. I downgraded the core lib to 1.6.12, the last version of the mighty Sandeep Mistry. The age of the mkrnb and the core are now more equal. I will let u know.

[rhowell209]

I have been following this thread (and all the precursors to this thread on Arduino forums including the board soldering to get the R410M firmware upgraded on the MKRNB) and am very interested in the progress on this. In my particular experience, I have been trying to get SMS messages to send and receive, but have achieved neither of these even once. I seem to be able to run all of the other examples (SATA pass through, NTP time pull from a server, Network type configuration (LTE-M, NB, then the ones with failover order)) but the SMS side of it has never worked. My AT&T account swears SMS is enabled for this SIM, although I never took it out and tried it on a cell phone because I wasn't sure if the SIM had to be tied to a device IMEI to work and feared that may alias the truth.

I'm having a bit of an embarassing moment here, but I can't seem to find this library in my Arduino Library Manager. What can I search for specifically to find it and ensure it is downgraded? I feel like I've seen it before but it's just not showing up.

Thanks,
Richard

EDIT: I realized the board manager core firmwares are in separate managers than the code libraries. All good here, will post any unique results that improve further guidance. When I get a moment, I'll keep downgrading them until I can possibly avoid the character loss issue described above.

EDIT 2: Still can not send or receive SMS using either MKR NB 1500 included examples. m-center shows "success" when sent manually over AT commands, but it's never received on my device, nor vice versa.

[CptHolzschnauz]

The issue is between the modem and the mobile network, nothing with the lib or your sketch.
You have to update the firmware of the modem first of all. The last release solves some sms problems and some issues with AT&T as far as i know.
https://forum.arduino.cc/t/firmware-upgrade-for-ublox-sara-r410m-02b-on-the-mkr-nb-1500-2/699292/12
It looks a bit scary for the first time, but you can do this. With some experience it's a 5 minute task.
You need a small solder iron tip. First bring some fresh solder to the mkr, take a good pretinned cable, start the EasyFlash as Admin, try several times when it does not suceed at first time. Good luck!