Repository navigation
No end-to-end test has ever run a pcapng, a gzipped capture, or four common input shapes #94
Description
Found in the final /project:verify pass. Every behaviour below was run and is correct today — the gap is coverage, not defects.
Every end-to-end test in tests/test_cli.py and tests/integration/ feeds the plain tests/fixtures/benign.pcap. Measured through cli.main:
| input | result | covered? |
|---|---|---|
plain.pcapng |
format: pcapng, normalization: ["convert: editcap -F pcap"], 2 flows, 2 labels |
only at the normalize() unit level |
plain.pcap.gz |
format: pcap.gz, normalization: ["decompress: gzip"], 2 flows, 2 labels |
same |
| ARP-only | 0 flows, complete run directory, 0 labels | no |
| one-packet capture | correct | no |
| VLAN-tagged (802.1Q) | 2 flows, 2 labels, correlates | no |
| IPv6-only TCP | 1 flow, 1 label | no — and this is the only exercise anywhere of spec §8's "Suricata expands, Zeek compresses" IPv6 canonicalisation over a real IPv6 capture |
| ICMPv6 echo pair | correlates through the 128↔129 counterpart relaxation | no |
Two loss conditions also have a real fault available for free, where today they are asserted against hand-built structs:
detection_uncorrelatable— the ESP fixture from the protocol issue is a natural real fault, and would have caught that defect on day one.- Tool non-zero exit — the zero-packet capture is a real Suricata failure through the full pipeline, replacing a monkeypatched synthetic
ToolError.
Multi-datalink discard is likewise reachable end to end and was verified by hand: exit 0, input_status: partial, discarded_link_types: ["LINUX_SLL"], discarded_packets: 4, multi_datalink_discard: true, and it still labels the kept flow. Behaviour correct, test missing.
Ambiguous flow match is genuinely hard with real tools — an alert's ts always falls inside its own flow's window, except under #60's rounding. Worth recording in the spec as deliberately unit-only rather than leaving it looking like an oversight.
Related: Goal 2 reproducibility is only ever measured over benign.pcap — never a pcapng (where editcap enters the path), a gzipped capture, or any corpus capture. The gate itself is strong; its input set is one file.
Activity
docs: Phase 1 sign-off (closes #5) (#102)