Skip to content

Test infra: continuation (chaos topics, transport chaos, fuzz corpus) #48

Description

@EdmondDantes

Continuation of the test-infra epic. Shipped so far: Windows/macOS CI (#39/#40), conformance nightly (#41/#42 — h2spec, testssl, h2load), chaos suite (#43/#44 — Http11Probe, scheduler-chaos, rapid-reset + CONTINUATION-flood), and the committed Http11Probe report (#46). Real RFC validation gaps are tracked in #47.

This issue tracks the remaining testing work (no server-behaviour fixes — those go to #47):

Chaos / attack suite

  • Triage scheduler-chaos findings → flip to gating. The TRUE_ASYNC_SCHED=random:N matrix is informational; 5 phpt fail under it. Confirm whether the worker-pool worker thread exited cleanly line under fuzz is a real early-worker-exit (the line is absent in the normal run — a worker genuinely returns from start() under the randomised/debug build; needs an --enable-async-fuzz local repro to isolate fuzz vs debug) or benign; make the 5 tests scheduler-order-independent; then gate.
  • HPACK bomb + PING flood (CVE-2019-9512) chaos tests via H2TestClient.
  • CVE-2019-9511..9518 nghttp2 family (data dribble, 0-length headers leak, settings flood).
  • Topics h2_framing (random HEADERS/DATA/CONTINUATION/RST/SETTINGS ordering) and h1_pipeline (N requests, random sizes / keep-alive / mid-stream close).

Transport chaos

  • EvilPeer — pure-PHP declarative toxics (payload slicing, drip delay, mid-stream close), seeded by CHAOS_GEN_SEED; runs in PR CI.
  • Toxiproxy nightly job (bandwidth throttle, latency+jitter, segment slicing, truncation), opt-in via --SKIPIF-- probe.

Fuzz corpus

  • Fold the HTTP Garden request-smuggling corpus (arxiv 2405.17737) into the http1 libFuzzer corpus (differential against llhttp + H1↔H2 normalization).

Conformance follow-ups

  • tlsfuzzer (deferred from Test infra: external conformance suites + chaos/fuzz harness #41) — TLS 1.0–1.3 RFC conformance + fuzz, curated vector scripts; informational first.
  • Improve tests/bench/probe_server.php — echo request body + use the static handler for conditional/304 — removes the harness-limitation "failures"/warnings from the Http11Probe report (chunked/POST echo, caching-304, cookie echo) so the score reflects real behaviour.

Supersedes the follow-up list in #43 (closed).

Activity

  1. EdmondDantes commented on May 29, 2026

    @EdmondDantes
    ContributorAuthor

    Flaky test: tests/phpt/server/core/018-log-off-no-overhead.phpt fails intermittently on macOS DEBUG CI (timing-based — measures log-disabled hot-path overhead vs debug; flakes on loaded runners). Hit PR #44 and #45, passes on rerun. Make it non-timing-dependent (relax threshold / use a deterministic gate) or mark a known macOS-debug skip.

  2. added this to the Next2 milestone on May 29, 2026
  3. EdmondDantes commented on May 29, 2026

    @EdmondDantes
    ContributorAuthor

    Update: 018-log-off-no-overhead flakes on macOS RELEASE too (not just debug) — PR #46 hit it on release. It's a recurring CI nuisance across both macOS configs; should be made deterministic or skipped on macOS.

  4. EdmondDantes commented on May 29, 2026

    @EdmondDantes
    ContributorAuthor

    Another flaky test: core/030-response-json on Linux DEBUG — sub-requests intermittently return status=0 (client gets no response; connection-setup race under CI load, one sub-request still succeeds). Seen on main after the #45/#46 CI-only merges (not caused by them). Make 030 + 018 deterministic / retry-tolerant.

  5. self-assigned this
    on May 29, 2026
  6. added theissue type on May 29, 2026
  7. EdmondDantes commented on May 29, 2026

    @EdmondDantes
    ContributorAuthor

    h3spec / HTTP/3 conformance — dropped. HTTP/3 isn't production-ready in the server, so H3 conformance probing is out of scope here. (HTTP/2 stays covered by h2spec.)

  8. added 2 commits that reference this issue on May 29, 2026
  9. added
    enhancementNew feature or request
    and removed
    bugSomething isn't working
    on May 29, 2026
  10. changed the issue type fromtoon May 29, 2026
  11. removed this from the Next2 milestone on Aug 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions