Add Mobitex-NX deframer

[kerel-fs]

The TUBIN satellite will re-enter later this year and NanoFF will be active for even longer, so there is interest in the Mobitex-NX protocol. This MR replaces the unmaintained gr-tnc_nx module by implementing necessary components as blocks inside gr-satellites. For backward-compatibility, the gr-tnc_nx OOT module can still be used when --use_tnc_nx is passed as parameter.

This MR is targeting the Mobitex-NX framing as used by BEESAT-2,-3,-4, -9, TechnoSat, TUBIN and NanoFF. The satellite BEESAT-1 is using a slightly different framing as described in #670. It should be easy to adapt the new blocks to this frame structure, but for the sake of not increasing the size of this MR even further I've left this exercise for a future patch.

General structure

1. From a stream of symbols, sync to the frame sync with sync_to_pdu_packed and create PDUs of frames of the maximum expected length
2. Crop those frames into N blocks, preserve Mobitex header in a tag added to the first data block
3. Convert the PDUs into a stream of symbols (individual bits)
4. De-Interleave symbols
5. (De-)Scramble symbols
6. Convert stream of symbols back into PDUs of encoded Mobitex data blocks (20 code words = 30 bytes)
7. Perform error-correction, return decoded Mobitex data blocks (20 words = 20 bytes)
8. Re-assemble frames following the "TUBiX20" protocol

Callsign decoding

If the callsign is known, it calculates the bit-wise Manhattan distance between known and received callsign+CRC, to ensure correct decodes even under multiple bit errors.
If the callsign is unknown, it tries to correct the callsign based on the CRC (motivated by [3]). This can quickly produce false decodes with valid CRCs when there are more than 2 bit errors, thus the first option was introduced.

Scrambler

While the scrambler looks quite similar to an ordinary additive scrambler that could be implemented by additive_scrambler_bb, the feedback in mobitex_coding.cc#L121-123 is dependent on two bits simultaneously. Thus, mobitex_scrambler was added as a sync block, handling a stream of symbols, with reset by the frame_header tag.

De-Interleaver

Finally something boring: using the gnuradio matrix_interleaver block

FEC

Uses crc16_ccitt_zero to decode this (12, 8, 3) linear code, implemented in the mobitex_fec block. This block consumes PDUs consisting of individual encoded Mobitex blocks (20 * 12 bits) and returns decoded Mobitex blocks (20 * 8 bits).

TODO: Add CRC check, return 18 bytes instead of 20 bytes.

TUBiX20

The gr-tnc_nx module established a frame format which consist of a header (control bytes, callsign and callsign CRC; NOT the control bytes CRC), the decoded Mobitex data blocks (N*18 bytes) and 6 trailing bytes (0xaa, the errorcode, 0xbb). This block consumes PDUs with data blocks and assembles such "TNC-NX master frames".

Testing / QA

This patch was tested with the linked recording of TUBIN (with a lot of bit errors), as well as test data generated with the reference implementation (the mobitub-2 repo). During development the test data was injected in mobitex_deframer, but this is not acceptable in the final code. Instead, I'd like to add some basic test cases (each one data block = 30 bytes):

• Interleaved, then scrambled
• Interleaved, no scrambling
• Scrambling, no interleaving

Writing those test cases turned out to be more complicated than expected, thus I publish this version for review without them.

Telemetry decoder

I couldn't resist adding a rudimentary telemetry parser for TUBiX20 frames. That said, I consider it of secondary priority and would be happy to drop it from this MR if there are any problems with it, to make it easier to review the rest.

References

• mobitub-2 repository
• MIT lecture 6.02 chapter 5: error correction, syndrome decoding
• blog: Some notes on BEESAT and Mobitex-NX
• SatNOGS IQ recording of TUBIN: https://pe0sat.vgnet.nl/download/TUBIN/

[kerel-fs]

ToDo

• Fix pycodestyle errors
• mobitex_fec: Add CRC check and return 18 bytes instead of 20 bytes

[daniestevez]

Thanks! Ping me when this is ready for review.

[kerel-fs]

This is ready for review now.

Changes since my initial draft:

• Added missing per-block CRC checks
• Added QA tests for the mobitex_scrambler block, the mobitex_fec block, as well as the whole mobitex_deframer hierarchical block
• Added "custom" framings for TUBIN, BEESAT-1 & -9. Those are variants of the "Mobitex-NX" framing, with the difference that the callsign is known and thus it's reasonable to accept more callsign errors. When the callsign is unknown, we need to rely on the FEC, which strongly limits the maximum number of reasonably acceptable bit errors in the callsign
• Dropped unrelated commits (moved to other & future PRs)
• Fix all style errors
• Removed debugging remnants
• Fixed tag propagation for unknown tags

[daniestevez]

Thanks for contributing this! It is greatly appreciated. Generally the code looks good, but I have written lots of comments (since it's a lot of new code).

[kerel-fs]

Thanks for the extensive review! It will take some time for me to resolve all the comments. I'll do so with additional atomic commits that can be squashed in the end.

[kerel-fs]

All comments except two have been addressed.

The remaining topics are:

1. The new callsign threshold handling has some quirk, I'd like your feedback if this is ok / if you see an alternative.
2. The crc checks from python/crcs.py aren't pure Pyhton methods, but GNU Radio blocks. Using them as replacement of check_eseo_crc.crc16_ccitt_zero as suggested would require the rewrite of this otherwise finished & tested code.

For details and further discussion see the respective threads.

Major changes:

• Added QA tests for callsign, control encoding/decoding & the low-level FEC methods
• Renamed mobitex_deframer to mobitex_to_datablocks
• Added the callsign as deframer option, which reduced the required variants down to only three (BEESAT-1 with a different header, BEESAT-9 with hard-coded number of blocks and the default variant)
• New default of callsign_treshold=12 when callsign is known

[daniestevez]

This can be written as Optional[tuple[list[int], int]]. I've checked the docs and apparently Optional is older. The X | None syntax appeared in Python 3.10. So here Optional would be preferred to support older Python versions, although I don't know what happens with type annotations that are unsupported by an older Python. Maybe they are ignored.

This X | None -> Optional[X] can be applied to a couple other places in the code.

[daniestevez]

Sorry for the delay in reviewing this. There are a couple of outstanding things (and one of them is on me to figure out the scrambler). All of them are in unresolved comments. Besides these the PR looks quite good now.