Suggestion: special cases

[wowczarek]

Hi there,

I am using a variant of this encoding in a project that runs on an ancient sub-1 MHz 8-bit machine, with data over serial port and extremely low bandwidth. I say a variant because I keep all excluded symbols in the 0...n range and exploit this fact.

I needed to make some optimisations and one applies to the general format of this encoding scheme and thus this project:

When marking the symbols used in a block, I keep track of the maximum byte found. If max < (0xff - n), you can safely pick the last n possible bytes as substitutions and move on without checking if they're taken. Obviously this is content-dependent, but it does trigger.

The other optimisation will not work with your code, but in case anyone else might want to do what I'm doing:

Because all of my "take outs" are in 0...n, when encoding I also find the minimum byte in a block, and if min > n, then there is no need for substitution at all - in this case I set at least the two first bytes of the replacement list to the same value. The receiver then checks if first two replacements are the same byte, and if they are, it knows that there is no substitution required.

This encoding scheme is a great find by the way, my project would have been impossible to implement if it wasn't for this. It's a case where I need to send IP packets via a serial link that does not support hardware flow control (so SLIP will not work), but it still requires explicit flow control that must be controlled by the application (XON/XOFF will not work either), plus extra control characters (block acknowledgment and separator / end of packet = forced end of block). With escapeless I could easily implement my own flow control and still send arbitrary data using escapeless 128/4.