Skip to content

fix(protobuf): reject malformed varints across values and framing#2258

Open
samchon wants to merge 3 commits into
masterfrom
fix/protobuf-varint-bounds
Open

fix(protobuf): reject malformed varints across values and framing#2258
samchon wants to merge 3 commits into
masterfrom
fix/protobuf-varint-bounds

Conversation

@samchon

@samchon samchon commented Jul 19, 2026

Copy link
Copy Markdown
Owner

Closes #2256.

A malformed Protocol Buffer varint could become success-shaped decoded output. All value and framing consumers share the tenth wire byte, so the fix enforces its single legal payload bit in one reader boundary in _ProtobufReader.ts, and corrects the #2139 reader regression that had frozen ten-continuation acceptance.

Verification

Independently verified by an agent that did not write the implementation and built its own oracle rather than reusing the batch's test material.

Measure Result
Differential vs official protowire v1.34.2 — 885 byte strings x 8 reader entry points 7080/7080 cells agree, 0 disagreements
Same differential on the base commit 3971 disagreeing cells over 530 inputs, every one accepts-illegal
Regressions / loosened cells introduced 0 / 0
Oracle-accepted inputs decoding to the exact value at the exact consumed length 208 / 208
Exhaustive tenth-byte partition, both prefix families accepts exactly {0x00, 0x01}, bit-identical to the oracle; base accepted all 256
Cross-reader consistency among the 8 consumers 0 disagreements, against 269 on base
Public wrapper matrix (direct/factory x raw/assert/is/validate) 152/152 threw a prefixed Error — never null, never success: false

Mutation proof: reverting only _ProtobufReader.ts fails exactly the three coupled regressions while all fifteen adjacent protobuf controls pass. Gates at the head: pnpm build, 18 protobuf cases, 178-case test-typia-schema, feature identity, go vet in both trees, and tagged native ../native/... (28 packages, 0 failures).

The changed #2139 expectation is a correction, not a laundered regression

A previously merged test asserted that typia must accept 0x80 x10, 0x01 as 0. protowire rejects that input with n = -3. The old expectation was wrong; the new one matches both the wire specification and the oracle. Adjudicated independently against the specification rather than against this pull request's narrative.

Second commit: the oracle test could not fail

Independent verification found the first commit's permanent TestProtobufVarintBoundsOracle passed with the product fix fully reverted — it exercised protowire alone, never called typia, and its agreement with the TypeScript regressions was hand transcription. A permanent test that cannot fail when the work regresses is a false green, so the batch was returned for a test-asset remediation.

Every byte string and verdict now lives in one packages/typia/test/protobuf_varint_corpus.json that the Go test and both TypeScript regressions read. The Go test is renamed to state that it pins the official parser's verdicts only. Verified by mutating the corpus seven ways — value, consumed length, fault relabelling, emptied entries, unknown class, a fabricated contradictory row, and an accepted/rejected flip — with both halves reacting to every mutation.

The same commit splits one error message that had described two different faults: a continuing tenth byte keeps varint exceeds 10 bytes., while a terminating tenth byte carrying payload above bit 0 now reports varint exceeds 64 bits. Acceptance is byte-identical — proven by re-running the full 7080-cell differential across both heads: 0 acceptance differences, 0 value differences, 2088 message-only differences. No website page, documentation, example, or tracked test outside this batch pinned the old string.

Out of scope, recorded for the next discovery round

Two verified, user-visible defects present identically on the base commit and on this fix, outside this issue's published invariant: length prefixes and field tags are silently truncated to 32 bits, so a field declaring 4,294,967,301 bytes reads five, and field 536870913 aliases onto field 1. A third consumer of the same truncation, bool(), decodes from the low 32 bits only. None is locked in by the new tests, so the next round can decide the contract rather than inherit a snapshot.

Campaign constraints

No package version, tag, release, publication, or packages/interface/src change. pnpm format was not run.

samchon added 3 commits July 19, 2026 17:31
Claim issue #2256 before implementation so one batch owns value, tag, length-prefix, wrapper, and independent-oracle consequences of the shared reader boundary.

Constraint: Campaign pull requests cannot change versions, tags, releases, or publication state.

Rejected: Limit the fix to declared integer fields | uint32 also owns tags and every length prefix.

Confidence: high

Scope-risk: moderate

Directive: Enforce the wire invariant without prescribing reader control flow and retain every valid boundary control.

Tested: Independent value/tag/length reproductions and official Go oracle critic review passed on ce88a88.

Not-tested: Product implementation and integration gates are intentionally pending.

Related: #2256 #2135 #2139
All value and framing consumers share the tenth wire byte, so enforce its
single legal payload bit in one reader boundary. Permanent runtime, generated
wrapper, and official protowire regressions keep malformed input from becoming
success-shaped decoded output.

Constraint: Campaign implementation freezes package versions, tags, and publication state
Rejected: Mask or discard the tenth byte | preserves the silent truncation defect
Rejected: Guard generated decoders only | direct readers, tags, lengths, and unknown fields share the runtime owner
Confidence: high
Scope-risk: moderate
Reversibility: clean
Directive: Preserve legal ten-byte maxima and non-canonical in-range encodings when changing varint parsing
Tested: Focused reader/decode regressions, official Go oracle, mutation proof, full schema workspace, feature identity, pnpm build, pnpm test, tagged native Go tests, native/test go vet
Not-tested: Prospective-merge and exact-merge gates; independent verifier owns those campaign stages
Related: #2256
The Go oracle test exercised protowire alone, so it stayed green with the
product fix fully reverted while its name promised typia's own bounds. Every
byte string and verdict now lives in one corpus both languages read: the Go
test proves the corpus still matches the official parser, and the TypeScript
regressions read the same rows through typia's reader and generated decoders,
so neither side can transcribe a verdict the other has not seen.

The tenth byte also carried one message for two faults. A continuing tenth
byte would occupy an eleventh, but an excess payload bit terminates inside ten
bytes and exceeds no length, so it now reports the 64-bit domain it left.
Acceptance is unchanged: the same two tenth-byte values pass as before.

Constraint: Campaign implementation freezes package versions, tags, and publication state
Rejected: Run typia's reader from Go | the one Go-to-node harness stubs every typia/lib/internal import away, and a built lib is no prerequisite of the Go suites
Rejected: Keep the corpus in each test | hand transcription is what let the oracle and the regressions drift
Rejected: Keep one message for both tenth-byte faults | a ten-byte payload terminates within ten bytes and exceeds no length, so one message could only name one of the two faults truthfully
Confidence: high
Scope-risk: low
Reversibility: clean
Directive: Read the shared corpus instead of writing a varint byte string into a new test
Tested: Isolated ttsx cache; corpus oracle test with two corpus-drift controls, focused reader and decode regressions, all 18 protobuf cases, 178-case schema workspace, feature identity, go vet over the test module, and a mutation proof with an executed-code probe
Not-tested: Prospective-merge and exact-merge gates; the lead owns those campaign stages
Related: #2256
@samchon
samchon marked this pull request as ready for review July 19, 2026 15:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(protobuf): reject malformed varints across values and framing

1 participant