Skip to content

Fix CBOR Encode for cbor v9: use streaming Encoder with pre-sorted Maps#2423

Open
J8k3 wants to merge 1 commit into
gchq:masterfrom
J8k3:fix/cbor-v9-encode
Open

Fix CBOR Encode for cbor v9: use streaming Encoder with pre-sorted Maps#2423
J8k3 wants to merge 1 commit into
gchq:masterfrom
J8k3:fix/cbor-v9-encode

Conversation

@J8k3
Copy link
Copy Markdown

@J8k3 J8k3 commented May 20, 2026

Problem

cbor v9 changed Encoder.encode() and encodeCanonical() from synchronous to async stream-based, but the static sync wrappers were kept and silently return only the first chunk (~1 byte) of output. The existing CBOREncode operation called Cbor.encodeCanonical(input) synchronously, so all multi-byte values were truncated.

Affected test cases (currently failing):

  • CBOR Encode: Can encode text — expected 64 54 65 78 74, received 64
  • CBOR Encode: Can encode map — expected a3 61 61 01 ..., received a3
  • CBOR Encode: Can encode list — expected 83 00 01 02, received 83
  • CBOR Decode: Can encode decimal — input truncated before decode

Single-byte values (0f for integer 15, f4/f5 for booleans) still passed by coincidence.

Fix

Drive a streaming Encoder directly:

  1. Pre-sort plain object keys into Maps using RFC 7049 canonical order (shortest encoded key first, then lexicographic byte comparison) before encoding — so insertion order produces the canonical result.
  2. Register a custom Map semantic type that bypasses the encoder's own canonical sort (which has the same first-byte-only bug in v9 due to an internal NoFilter stream).
  3. Collect all output chunks via the data event, then return a correctly sliced ArrayBuffer (avoiding Node.js Buffer pool aliasing via byteOffset/byteLength).

run() becomes async; the CyberChef framework already awaits run().

Test plan

  • CBOR Encode: Can encode integer0f
  • CBOR Decode: Can encode decimalf9 3e 00
  • CBOR Encode: Can encode text64 54 65 78 74
  • CBOR Encode: Can encode boolean truef5
  • CBOR Encode: Can encode boolean falsef4
  • CBOR Encode: Can encode mapa3 61 61 01 61 62 02 61 63 03
  • CBOR Encode: Can encode list83 00 01 02

🤖 Generated with Claude Code

cbor v9 changed Encoder.encode() and encodeCanonical() from sync to async
stream-based, but the static sync wrappers still exist and silently return
only the first chunk (~1 byte) of the output.

Fix: drive a streaming Encoder directly —
  - Pre-sort object keys into Maps using RFC 7049 canonical order
    (length-first, then lexicographic byte comparison) before encoding,
    so insertion order gives the canonical result.
  - Register a custom Map semantic type that bypasses the encoder's own
    canonical sort (which has the same single-byte bug in v9).
  - Buffer all output chunks, then return the correctly sliced ArrayBuffer
    (avoiding Node.js Buffer pool aliasing via byteOffset/byteLength).

The run() method becomes async; the framework already awaits run().

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant