Skip to content

Harden email extraction boundary handling and UTF-16 regressions #620

Description

@simsong

Follow up on the unresolved Copilot review findings in PR #593.

Investigate and add focused regression coverage for all four concerns together:

  1. Preserve extraction of ASCII emails before sentence punctuation without accepting a partial email/TLD match.
  2. Treat a preceding non-ASCII UTF-16LE code unit as a left boundary even when its low byte is ASCII alphanumeric.
  3. Define and test the UTF-16 trailing-context behavior for unaligned following bytes and sentence punctuation; compare it with the corresponding UTF-16 URL rule before changing the lexer rule.
  4. Strengthen accepted UTF-16 length-limit tests to assert the exact emitted email.txt feature and offset, not merely non-empty output.

Document the boundary cases chosen and verify that the change does not increase partial-email false positives.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions