Use of Regex

[TheBakedPotato]

In my opinion, regex should not be used as a means to represent the protocol. First and foremost regex is genuinely a pain to decifer and to maintain. As well, regex is most powerful when using it to parse unvalidated input or searching unstructured data. For example, when a user enters something like an email address in a text box, regex is VERY good for ensuring it is of the correct form. When searching through files on your filesystem or text in a text file regex is as well extremely powerful there to help with your search. It is not a tool for ensuring a protocol payload is of the right form.

To make it easy for even MCU to parse, it can be newline or like CSV. The protocol message could then be of the form:

a
1
# this would be an empty line to accommodate the space for an ID
Your message here

Or CSV:

a,1,,My hair's on fire
a,1,18afbc,shelving message

That presents a MUCH more structured layout for the protocol.

[RobertLRead]

Well, RegEx will be valuable for the Python team. And it is very precise, but you are correct they are very hard to decipher. I don't believe they should be used exclusively, but in combination with examples they are okay.

The use of newlines is a nice idea, but makes the message less human readable. I like the CSV idea better.

If @nk25719 wants to switch to CSV, we could do that. Now would be the time to do it. @nk25719 if you want to discuss this, let's have a meeting.

[TheBakedPotato]

It is a protocol, afterall. It should not strive to be human readable. The protocol's goal is to get two machines to communicate with each other. Human readability should not REALLY be a factor in that. The goal should be to make the protocol as robust as possible.

[RobertLRead]

No, that is not right. The machines serve us. Human readability is VERY valuable in all cases, even if only for debugging---debugging is very important. I had this argument with my father 40 years ago, and I'm convinced I'm right. You don't want a cryptic protocol.

…

On Wed, Feb 25, 2026 at 10:11 AM TheBakedPotato ***@***.***> wrote: *TheBakedPotato* left a comment (PubInv/gpap#8) <#8 (comment)> It is a protocol, afterall. It should not strive to be human readable. The protocol's goal is to get two machines to communicate with each other. Human readability should not REALLY be a factor in that. The goal should be to make the protocol as robust as possible. — Reply to this email directly, view it on GitHub <#8 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABINEH72YTYDXLBFW2ZK34T4NXCRRAVCNFSM6AAAAACV7L2WBKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTSNRQGM3DONRSGA> . You are receiving this because you commented.Message ID: ***@***.***>

-- Robert L. Read, PhD Twitter: @RobertLeeRead @pubinvention Public Invention: https://www.pubinv.org Join Our Mailing list: ***@***.*** YouTube: https://www.youtube.com/channel/UCJQg_dkDY3KTP1ybugYwReg Medium: ***@***.***

[TheBakedPotato]

The best example I have is network protocols, like TCP, IP, ethernet. Those are not human readable in the SLIGHTEST. I am not proposing this should be a full binary protocol but human readability should not be a main concern.

With regards to debugging, that's why there are tools specific for packet sniffing and such, like Wireshark, tcpdump, and tools to sniff USB packets. USB is also very much a non-human readable protocol.

Other meesage passing formats, like XML and JSON, while human readable, add a lot of structure and "control" characters and symbols to indicate the data and their respective values. Those specs also pass around payloads in plain text and are human readable but they are HARDER to read due to the necessary data in the payloads to make it robust.

Also, as an aside, obfuscation != encryption. At WORST this would add some minor obfuscation. That is very different from encryption which the intent is to make it undecipherable.

[RobertLRead]

This email was sent to @nk25719 @TheBakedPotato @ForrestErickson @saivishwanatha and @mohamadMZ98. Juhandre asked me to put this here.

Dear Nagham and Juhandré,

Juhandre has raised an interesting change to my suggestions. I think we need to work this out. I've empowered Nagham to lead this, but she may not be prepared adjudicate a friendly disagreement between Juhandre and myself. Juhandre has raised this (thank you) in this issue:

#8

There are two issues here:

1. Should we use an RegEx? and
2. Should we use a different format.

I agree with Juhandre that a Regular Expression should not be the formal definition of the protocol. However, I think having one is valuable, because it is executable -- especially in Python, for the ADaMServer team for both parsing and validation. So I suggest we keep it. If we want to move it "below the fold", that is to move down below in the text that is fine. Hoping that is a good compromise.

Second, should we use "comma separated values" for this protocol?
I am somewhat hesitant to consider this because it is a "breaking change" of the protocol that is already in place, implemented in Mohammad and Saicharans python and test code, and the C++ that I wrote. However, if it is better, then we could change.

Arguments for (Rob):
CSV is something everyone understands. If you use a backslash or a doubled comma to escape "commas", so they can be used in the text, it is easy for a weak microcontoller to parse it.

Arguments against (Rob):
It is easy to be "off by one" in your commas. If you have optional fields, you indeed having empty commas, which are somewhat hard to understand and easy to misinterpret. I find it unattractive.

Possibly Juhandre wants to present, or re-present his own arguments.

This should be a team decision; others might have input. But in the end, I would like Nagham to make a decision; but if she wants she can delegate it to me, or Lee, or Juhandre, or Saicharan or Mohammad. Possibly Lee is the most experienced with this kind of thing beside myself.

[RobertLRead]

This email is from @nk25719 👍
Hello all,

I apologize for the late reply.. I am taking some time to review the protocol while considering the transports we expect to support (Wi-Fi/TCP, UART, and SPI)… still evaluating the options against the following criteria (and I decided to bring this up to you so you can help): Stability, parsing reliability , human readability ...

From a conceptual standpoint, I agree that a Regular Expression should not serve as the formal definition of the protocol, but it is valuable as an executable reference for validation and parsing in ADaM…

One important consideration is that the protocol should remain reusable across transports (UART and SPI) without requiring us to redefine framing or parsing rules each time (we should ensure that whichever framing approach we choose remains consistent across transports while maintaining reliability and simplicity).

Please advise on this.

We need to make a decision and move forward as soon as we can.. but we better check all the cons and evaluate our options first.

[RobertLRead]

Unless @TheBakedPotato objects, we have a compromise on the RegEx issue---it will not be used for the official definition but will be used to add precision.

The protocol as it stands is easily parseable by computers, even the weakest microcontrollers, because the hardest action it requires is to find an opening square bracket or curly brace and then looking for a closing square bracket or closing curly brace. JSON parsing in general is much more difficult, for example. In fact, from a theoretical computer science point of view, JSON is a "context-free language" and this protocol is a "regular language" (https://en.wikipedia.org/wiki/Regular_language)[https://en.wikipedia.org/wiki/Regular_language]. Or course, so a CSV-based protocol. If we allowed structure within the [] or {}, then would be more difficult. By limiting the content to 80 characters, we have made the overall message length limited, which helps with allocating buffers on small processors.

[TheBakedPotato]

@RobertLRead I was never suggesting to use JSON. That would be overkill in this case. Lol Simply wanted to highlight a different text based protocol is all.

But that is a fair compromise. I can work with that.