Enhance Serial RX Buffer Reuse and Optimize Memory Usage for Parsing Received Data With `scanf`

[DjordjeMandic]

Hi everyone,

I'm currently working on a project that involves handling large RX buffers and parsing incoming data. At the moment, I have a setup that involves creating an additional buffer in memory, which is just one byte larger than the main RX buffer. Here's how it works:

1. The additional buffer is cleared before expecting new data.
2. Incoming data is copied into this buffer using the readBytes function.
3. The buffer is then null-terminated.
4. Finally, the null-terminated buffer is used with vfscanf or vfscanf_P for parsing.

While this approach works, I believe there is room for optimization. Specifically, the current design requires additional memory for the secondary buffer. My idea is to reuse the existing RX buffer instead of allocating an additional one. This would reduce memory consumption significantly.

---

Proposed Implementation:

Here’s an example implementation of how scanf could work directly with the RX buffer:

/* Reset buffer to initial state */
_buffer_overflow = false;
_receive_buffer_head = _receive_buffer_tail = 0;

/* Wait for new data */
do {
  /* If stream timeout occurs, return EOF */
  if (timedPeek() < 0) {
    return EOF;
  }
} while (available() > 0);

/* Check for buffer overflow or insufficient room for a null terminator */
if (_buffer_overflow || _receive_buffer_tail == (sizeof(_receive_buffer) - 1)) {
  return EOF;
}

/* Null-terminate the buffer at the position after the tail */
_receive_buffer[_receive_buffer_tail + 1] = '\0';

/* Use the RX buffer directly with vfscanf or vfscanf_P */
return vfscanf(_receive_buffer, ......);

---

Benefits:

• Memory Efficiency: By reusing the RX buffer, the additional buffer is eliminated, reducing overall memory usage.
• Simplicity: Reduces code complexity by working directly with the RX buffer for parsing.
• Performance: Avoids redundant memory copying, leading to faster data handling.

---

Request:

I’d like to request help with refining and implementing this idea.

This would be a huge improvement for projects with limited memory or those handling large amounts of data.

Thank you for your time and consideration. Any feedback, suggestions, or contributions are greatly appreciated!