Skip to content

Reject non-canonical Base32 input (non-zero trailing bits and malformed padding) #13

Description

@darianmiller

TBase32.Decode accepts non-canonical input: it discards leftover trailing bits
without checking they are zero, and ignores padding (=) entirely. RFC 4648
section 3.5 says decoders SHOULD reject encodings where the pad/trailing bits are
non-zero, because they are not produced by a conforming encoder.

Consequences:

  • Two different input strings can map to the same decoded key (the trailing bits
    are silently thrown away), which masks copy/paste corruption of a secret.
  • A stray extra character on the end of an otherwise-valid secret is accepted.

TestLongerInput_LoremIpsum (Source/Tests/DUnit/radRTL.Base32Encoding.Tests.pas:83)
already documents the defect -- it asserts that appending a stray L:

// note: an extra unused byte may be discarded depending on padding..
CheckEquals(LOREM_TEXT, TBase32.Decode(LOREM_BASE32+'L'));

still decodes to the original text (line 85 shows that two extra chars LL
change the result, so the tolerance is exactly one non-canonical trailing char).

Add opt-in validation that rejects non-canonical input. Like the strict-decode
work, this is off by default so current lenient behavior is preserved.

Implementation notes:

  • Coordinate with the invalid-character strict-decode ticket
    (Add opt-in strict-decode mode to TBase32.Decode ...). Preferably fold both
    under a single strict/canonical switch (the same pStrict parameter and the
    same EBase32DecodeError exception) so callers get one "be strict" flag rather
    than two overlapping ones. Decide this when the first of the two is implemented.
  • Trailing-bit check: after the decode loop, the number of unconsumed bits is
    vBitsInBuffer. For a canonical encoding those residual bits must all be zero.
    In strict mode, raise if ExtractLastBits(vBuffer, vBitsInBuffer) <> 0.
  • Padding check (strict mode): validate that the = count and position are legal
    for the data length -- padding may only appear as a contiguous run at the very
    end, the run length must match pDataLength mod 8 (0/1/3/4/6 pad chars for the
    valid quantum sizes), and no alphabet character may follow a pad char. Reject
    otherwise. If padding validation proves large, split it into its own ticket and
    land the trailing-bit check first (it is the higher-value RFC-3.5 fix).
  • Default (pStrict = False) path must remain byte-for-byte unchanged; the
    existing TestLongerInput_LoremIpsum assertions stay valid for lenient mode.

Acceptance criteria:

  • With strict validation off (default), decode behavior is unchanged and all existing Base32 tests pass, including the LOREM_BASE32+'L' tolerance case.
  • With strict validation on, decoding input whose trailing bits are non-zero raises EBase32DecodeError.
  • With strict validation on, LOREM_BASE32+'L' (a non-canonical trailing char) is rejected.
  • With strict validation on, a correctly-encoded, correctly-padded string still decodes successfully to the same bytes as lenient mode.
  • New DUnit tests demonstrate two distinct inputs that decode identically in lenient mode, where the non-canonical one is rejected in strict mode.
  • Padding position/count validation is either implemented and tested, or explicitly deferred to a new open ticket referenced here.

Activity

  1. darianmiller commented on Jul 21, 2026

    @darianmiller
    ContributorAuthor

    Implementations MUST include appropriate pad characters at the end of encoded data unless the specification referring to this document explicitly states otherwise.

  2. darianmiller commented on Jul 21, 2026

    @darianmiller
    ContributorAuthor

    For TOTP, the governing spec is the otpauth Key URI Format (Google Authenticator's de-facto standard), and that convention stores/accepts the base32 secret unpadded. That's why real secrets in the wild are unpadded — not cause 4648 permits it freely, but because the referring spec omits it. So "unpadded is fine for TOTP" is correct, via 3.2's delegation, not in general.

    Since the applicable spec (otpauth) omits padding, the right call for this library is option (a) accept unpadded, but if any = is present, require the correct count/position. Requiring full canonical padding (option b) would reject legitimate otpauth secrets even in strict mode. (Let's pick option a)

    §3.3 also validates #12. The very next paragraph says implementations MUST reject non-alphabet characters "unless the specification referring to this document explicitly states otherwise." So the old lenient skip-everything behavior actually violated 4648's default — which is precisely what the #12 strict mode restores. (Lenient stays as an opt-out for the MIME-style "be liberal" behavior §3.3 also mentions.)

  3. darianmiller commented on Jul 21, 2026

    @darianmiller
    ContributorAuthor

    Extended strict mode (issue #12's pStrict / EBase32DecodeError) to reject non-canonical input: (1) non-zero trailing bits per RFC 4648 section 3.5, caught via ExtractLastBits(vBuffer, vBitsInBuffer) after the decode loop -- e.g. 'MZ======' and 'MY======' both decode to 'f' leniently, but strict rejects the non-canonical 'MZ======'; and (2) a data character appearing after a pad character. Full padding count/length validation and the padded-vs-unpadded policy were deferred to new open ticket #19, because real TOTP secrets are frequently stored unpadded and requiring canonical padding needs a deliberate policy decision. Resolved in v1.0.39

  4. darianmiller commented on Jul 21, 2026

    @darianmiller
    ContributorAuthor

    Completed in v1.0.39. Padding count/length validation and the padded-vs-unpadded policy are deferred to #19.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions