security(reviewer): fail closed on invalid UTF-8 changed-file evidence

[seonghobae]

Fresh boundary

Protected main is c85d710804139c0697d7ef8fa47d02b1389e6d84. reviewer/noema_reviewer/github_io.py currently decodes GitHub contents bytes with base64.b64decode(...).decode("utf-8", errors="replace") in _fetch_changed_file().

Fresh issue/PR searches found no open lane claiming this changed-file decode boundary. This issue owns a direct-main repair unless writer state changes before any write.

RCA

First failing boundary: exact current-head GitHub blob bytes -> bounded ChangedFile.content supplied to the Noema reviewer.

Immediate cause: malformed UTF-8 bytes are silently replaced with U+FFFD.

Root cause: the evidence collector treats lossy text normalization as equivalent to exact source evidence instead of distinguishing valid UTF-8 text from binary/malformed content.

Impact: reviewer/model/human evidence can describe bytes different from the current-head blob while the manifest still appears complete. For security-sensitive diffs this is an integrity blind spot, not a harmless rendering choice.

Distinct remedies / feasibility

1. Strict UTF-8 decode and explicit evidence failure for non-text bytes — selected smallest safe remedy. Do not invent replacement text.
2. Base64-retain raw bytes in the manifest — preserves identity but expands the model/public schema and token budget.
3. Heuristic binary detection plus replacement decode — rejected because malformed text can evade the heuristic and remains lossy.

Test-first acceptance

• Deterministic current-head contents fixture with invalid UTF-8 must not produce replacement-character source evidence.
• _fetch_changed_file must distinguish unavailable/deleted content from malformed UTF-8; malformed bytes fail closed visibly.
• fetch_manifest must surface the changed-file evidence failure so a review cannot become APPROVE on incomplete/lossy source context.
• Preserve bounded context, deleted-file handling and normal UTF-8 behavior.
• Focused RED -> narrow GREEN -> reviewer/full exact-head validation.

Do not treat model output, replacement text, or a synthetically reconstructed file as exact current-head evidence.