Skip to content

test262 harness: un-skip stale categories (eval/intl402/property-escapes) so the parity number is honest #5300

Description

@proggeramlug

Summary

scripts/test262_subset.py hides categories Perry actually handles well, via three skip layers. Three _PATH_SKIP entries (and the features-applicable.txt allowlist) are stale:

skipped category Perry actual skip reason (stale)
eval (eval-code + built-ins/eval) 94% "AOT — can't eval" (eval works; PERRY_ALLOW_EVAL=1 for dynamic)
intl402 78% "out of scope (no ICU)"
RegExp/property-escapes 87% "Rust regex crate gap" (\\p{} works)

(RegExp/lookbehind skip is vestigial — that subdir has 0 files. staging stays skipped — genuinely partial.)

The headline ~95.2% is therefore parity over a curated subset, not all of ECMAScript. True parity over everything runnable ≈ 91.8% (~34,305 / 37,384).

Proposed change

  1. Remove eval, intl402, RegExp/(?:lookbehind|property-escapes) from _PATH_SKIP (keep staging, module/agent flag-skips).
  2. Add intl402 to DEFAULT_DIRS.
  3. Add to features-applicable.txt: regexp-unicode-property-escapes + the Intl.* feature tags.
  4. Set PERRY_ALLOW_EVAL=1 in the perry compile env so eval-tagged cases run instead of compile-failing.
  5. Update the stale comments.

This makes the official number reflect reality (drops the headline to ~92%, honestly). Gaps tracked separately: #5297 (annexB/language).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions