Repository navigation
JA4T/JA4L empty for tunneled connections #324
Description
When running against GENEVE-encapsulated traffic, ja4t and ja4ts in conn.log are empty for the inner connections. JA4 (TLS) fingerprints work fine for the same traffic.
This was reported by a user in the Zeek forum here. Below reproducer was generated with Claude Opus 4.6.
Reproducer
gen_pcap.py creates a PCAP with a plain and a GENEVE-wrapped TCP
handshake carrying identical TCP options:
uv run gen_pcap.py
zeek -r ja4t-geneve.pcap Zeek::Geneve
zeek-cut id.orig_h id.resp_h id.resp_p ja4t ja4ts < conn.log10.0.0.1 10.0.0.2 443 65535_2-3-4-1-1_1460_7 65535_2-3-4-1-1_1460_7
172.16.0.1 172.16.0.2 443 (empty) (empty)
Expected: both connections produce the same ja4t/ja4ts values.
Likely cause
parse_tcp_options in zeek/src/ja4t.cc walks raw packet bytes from the outer Ethernet header, but uses ip_hl/tcp_hl from the inner (decapsulated) headers, so the computed offset is wrong when any tunnel encapsulation beyond VLAN is present. The LINK_ETHERNET check in scripts/fingerprints/ja4t/main.zeek:49 may also reject some tunnel types early.
Activity
john-althouse commentedon Sep 25, 2026
@mmguero What do you think?
mmguero commentedon Sep 29, 2026
I will try to take a look in the next day or two!
ricardomvalverde commentedon Oct 9, 2026
Hi. Any news?
mmguero commentedon Oct 9, 2026
Sorry, I've been crazy busy. I'll take a look.
Fix JA4T/JA4TS/JA4L for tunneled TCP connections (FoxIO-LLC#324)
update zeek tests for FoxIO-LLC#324
update zeek tests for FoxIO-LLC#324
mmguero commentedon Oct 9, 2026
Thanks @bbannier for the reproducer and @ricardomvalverde for the report and the detail in the forum thread. Sorry this took a while for me to look at.
I think the fields go empty one step earlier than the issue suggests. The JA4T and JA4L scripts read TCP header fields with
get_current_packet_header()and timestamps withget_current_packet(). Both of those describe the outermost packet Zeek read off the wire. For a connection inside GENEVE, that's the tunnel's Ethernet/IP/UDP header, sorph?$tcpis false.In JA4T,
new_connectionchecks!rph?$tcp || rph$tcp$flags != TH_SYNand returns right there. The SYN never gets recorded,syn_window_sizestays 0, andconnection_state_removewrites nothing.parse_tcp_optionsinja4t.ccnever runs for these connections. If the scripts had been seeing inner headers, you'd get a window size followed by junk options, not an empty field.JA4L fails the same way.
new_connectiontakes the non-TCP path and records the SYN with the tunnel's TTL. When the SYN-ACK threshold fires,!rp?$tcpreturns early, soja4l_snever gets written and the threshold for the client's ACK never gets armed. That's where the all-emptyja4l/ja4lscolumns in the forum post come from.ssl_server_hellohas the same!rp?$tcpcheck, which is there to skip QUIC, and it also drops every TLS connection inside a UDP tunnel.A fix in the BiF won't work. Zeek builds the inner packet in the tunnel analyzer, processes it, and frees it before the event queue drains. By the time any script handler or BiF runs, the inner packet is already gone. The data has to come from events that Zeek fills in from the inner headers while it still has them.
The change
JA4T/JA4TS. Window size comes from
connection_SYN_packet. Option kinds, MSS, and window scale come fromtcp_options. Zeek raises those two as separate events for the same packet, so whichever arrives first parks its data and the other claims it by matching packet timestamps. That keeps the code independent of event order. SYN-ACK retransmit delays come from consecutive responder SYN events. The-Rsuffix comes fromconnection_resetwhen history shows a responder RST. The client's second packet still ends tracking through ConnThreshold. The old rules carry over: only the connection's opening SYN counts, and the SYN-ACK has to be the responder's first packet.JA4L. The SYN and SYN-ACK timestamps and both TTLs come from
connection_SYN_packet, so tunneled connections get the inner TTL.ssl_server_hellochecks the connection's own transport protocol, not the outer header.Dead code. With JA4T now in pure script, the
JA4::parse_tcp_optionsBiF,src/ja4t.cc, andsrc/ja4t.hare removed. CMakeLists and the README are updated to match.Behavior changes for non-tunneled traffic
rph$l2$encap != LINK_ETHERNETcheck returned empty options for anything else. For example, the existingipv6.pcapngtrace used in some of the baseline tests is NULL/Loopback, and its baseline recorded65535_00_00_00for both directions. It now records65535_2-1-3-1-1-8-4_1346_6and65535_2-1-1-4-1-3_1346_10, which match the option kinds, MSS, and window scale tshark reports for the SYN and SYN-ACK. I updated the baseline.TH_SYN.ja4l_cwithout a SYN-ACK. Before, a client's second packet with no SYN-ACK seen (a SYN retransmit, say) gave a duration measured from zero.Tests
New test
ja4-conn-genevewith a trace built by a scapy script (gen_pcap-v2.py). The trace has two scenarios, each in the clear and inside GENEVE:Timestamps are fixed so the JA4L values stay stable. Inner TTLs are 50 for the client and 60 for the server, and the outer tunnel TTL is 200, so any fingerprint that picks up the tunnel header shows it right away. The baseline has the plain and tunneled rows identical for each scenario:
62500_50/125000_60and65535_2-3-4-1-1_1460_7for the handshake, and..._1-2-R4in JA4TS for the scanner. The outer GENEVE UDP connection has no fingerprints, and TTL 200 never shows up. As far as I can tell, this is also the first test coverage the JA4TS delay and RST suffix path has had.Known gaps, not addressed here
tcp_packetornew_packet, so tunneled connections skip F andja4l_deltastays empty for them. Logging a timestamp from a packet that might carry no data seemed worse than leaving it unset.Before I open a PR: performance
There's one cost I want to measure first. A
tcp_optionshandler tells Zeek to build aTCP::OptionListvector of records and queue a script event for every TCP packet that carries options. Linux puts timestamps on nearly every segment, so on typical traffic that's most TCP packets. The handler returns after a couple of field checks once the handshake is settled, but the allocation and dispatch on the C++ side happen before the handler runs. The old BiF ran twice per connection. Dropping JA4TS's per-packet responder threshold rearming saves a little, but probably not enough to offset this.The plan is to benchmark this branch against current
mainon large real captures, with both mostly plain traffic and tunnel-heavy traffic, looking at wall-clock time and Zeek's script profiler.If the regression is noticeable, the fallback is to split the paths again. Non-tunneled connections would go back to raw-header parsing. The
tcp_optionshandler would move into an event group that starts disabled and turns on the first time a connection withc?$tunnelshows up. A sensor that never sees tunnels would pay nothing, and one that does would pay the per-packet cost only from then on. The downside is that the raw parser comes back, along with its link-type limits for plain traffic, unless it gets fixed too. If the numbers come out close, I'd rather keep the single event-based path.@ricardomvalverde, once the branch is up, it'd help a lot if you could run it against your NSX traffic. Real GENEVE at 25 Gbps would tell us more about both correctness and cost than anything I can put together here.
mmguero commentedon Oct 9, 2026
It might be next week before I can do the performance regression testing though
ricardomvalverde commentedon Oct 9, 2026
Hi @mmguero !
Thank you for the progress in diagnosing the issue! Regarding the capture of the inner packets within the tunnel, although we don't fully understand the code, we requested AI assistance to generate two scripts: one called myrtt to calculate the client's RTT, the delta, and the approximate hop distance between them; and another called myrtt-s to calculate the server and application network response times (starting from the client hello/server hello).
To verify if they were generating the correct values, we ran them alongside JA4L on non-tunneled packets, and the times and other values matched perfectly.
We applied Geneve traffic and noticed that both scripts continued to calculate all the times on the tunneled packets.
Since we don't have the "official" script for analyzing inner tunneled packets, we can only infer that they are still being calculated correctly.
We also have no idea whether the script is prioritizing performance or if the analysis is incurring a high computational cost.
Therefore, if possible, I would like to know if you could review both codes I'm sharing with you to see if anything can be leveraged for the issues you encountered during your tests.
We also remain available to perform any necessary tests.
https://pastes.io/8KXQAiHU
myrtt
https://pastes.io/NWyNzIeu
MyRTTs