[Windows] Proxy rules stop being applied after hours of uptime: stale port-decision cache leaks traffic direct

[rsyuzyov]

Describe the bug

On Windows, connections of processes covered by a PROXY rule gradually start going out
directly, bypassing the proxy, after ProxyBridge has been running for a few hours.
Restarting ProxyBridge restores correct behaviour for another few hours.

It starts as occasional single connections ("flapping": the same process is proxied in
one request and direct in the next) and degrades over uptime until a browser session is
effectively permanently direct.

The cause is the per-port decision fast-path cache in packet_processor
(port_decided_bitmap / port_direct_bitmap): a DIRECT verdict cached for a source
port outlives the connection that produced it and is then inherited by an unrelated
process that reuses the same ephemeral port.

Environment

• OS: Windows 11 Pro 24H2 (26200)
• ProxyBridge: dev @ d773d3d (4.0.13-Beta)
• Platform: Windows
• Rules: ~20 enabled per-process rules → PROXY (SOCKS5), plus a LAN range → DIRECT;
  Localhost via Proxy disabled

To reproduce

1. Add a rule curl.exe → PROXY (any SOCKS5/HTTP proxy with a distinguishable exit IP).
2. Leave a chatty process that is not proxied running — anything that opens many
   short-lived connections (a browser with a disabled rule, local dev tooling, an
   IPC-heavy editor). Each of its connections caches a DIRECT verdict on its ephemeral
   port.
3. Poll the exit address in a loop, e.g. every 5 minutes: curl.exe https://ifconfig.me
4. Within a few hours some replies return the host WAN address instead of the proxy
   exit. The share of leaked connections grows with uptime.

The leak is easiest to see with an instrumented build that logs (a) the rule verdict for
every SYN and (b) every fast-path hit on a cached DIRECT.

What actually happens

Eviction of the decision cache is incomplete. port_clear() is only reached from the
outbound FIN/RST branches:

if (port_is_decided(sp))
{
    if (tcp_header->Fin || tcp_header->Rst)
        port_clear(sp);
    if (port_is_direct(sp))
    {
        WinDivertSend(windivert_handle, packet, packet_len, NULL, &addr);
        continue;
    }
}

so an entry survives whenever the close does not pass through that branch:

• the server closes first — the FIN arrives on the inbound branch, which never
  touches the bitmaps;
• loopback IPC and abortive teardowns are not seen by the filter at all.

The entry then stays set until the process restarts. When Windows reuses that ephemeral
port for a different process, the first SYN hits the fast path, is sent unmodified and
check_process_rule() never runs — a PROXY rule is silently skipped.

Measured on an instrumented build over 63 h: 232 056 ports were freed by Windows
while their cached verdict was still set and no FIN/RST had reached the filter. Cache
fill grew from ~2 400 to ~57 000 of 65 536 ports.

Note the SYN path already anticipates port recycling — it evicts the PID cache
there:

if (tcp_header->Syn && !tcp_header->Ack)
    remove_cached_pid(ip_header->SrcAddr, sp, FALSE);

but leaves the decision cache stale, so the wrong-app problem is only half closed.

Expected behaviour

A new connection is matched against the rules that apply to its process, regardless of
which process previously used the ephemeral port.

Additional context

• The IPv6 branch shares the same bitmaps and has no eviction, so a verdict cached for an
  IPv4 connection can be inherited by an IPv6 one reusing the port number.
• Evicting the decision cache on a fresh SYN (next to the existing remove_cached_pid)
  fixes it: 63 h uptime, ~1.4 M connection events, zero leaks, exit IP stable across 757
  probes. Happy to send a PR against dev.

[rsyuzyov]

A ready fix for this, in case it's useful — branch rsyuzyov:pr/cache-fix (compare): evict the stale port-decision cache entry on a fresh SYN, right where the PID cache is already evicted, in both IP stacks. Ran it for ~63h — no leaks, exit IP stable across 757 probes. Glad to iterate if you'd like to take it.

[dentatli]

@rsyuzyov
I previously investigated and fixed a separate Windows UDP relay issue involving stale fd_set state after SOCKS5 UDP ASSOCIATE sockets were recreated. After fixing that issue, I initially expected that the connection interruptions I had observed would no longer occur.

However, after reading your findings about stale cache state, source-port reuse, and the existing PID-cache invalidation, I decided to inspect this behaviour more closely. I found a couple of additional details related specifically to the port-decision cache lifecycle.

First, evicting the cached decision on every fresh SYN, before checking the fast path, appears to be the correct minimal fix. A fresh SYN represents a new connection and potentially a different process reusing the same ephemeral source port.

Applying this to both IPv4 and IPv6 is also necessary because the two protocol families currently share the same port-decision bitmaps. The proposed rsyuzyov:pr/cache-fix branch therefore appears to address the primary cause of #206 correctly.

FIN/RST may clear and then recreate the DIRECT entry

There is an additional lifecycle detail in the existing code:

if (port_is_decided(sp))
{
    if (tcp_header->Fin || tcp_header->Rst)
        port_clear(sp);

    if (port_is_direct(sp))
    {
        WinDivertSend(windivert_handle, packet, packet_len, NULL, &addr);
        continue;
    }
}

For a previously DIRECT connection, port_clear(sp) makes the following port_is_direct(sp) check return false. Processing can then continue to check_process_rule(), receive DIRECT again, and call port_set_direct(sp) later during the same packet-processing iteration.

As a result, an outbound FIN/RST does not necessarily leave the decision entry cleared. This may help explain why cached decisions remain populated even when some outbound teardown packets are observed.

The proposed fresh-SYN invalidation still prevents this entry from being inherited by the next connection, so this does not appear to block the current fix. It may nevertheless be worth simplifying the teardown path so that a previously DIRECT FIN/RST packet is sent unmodified after clearing the entry, without performing another PID/rule lookup and caching DIRECT again.

Minor clarification regarding loopback traffic

The current WinDivert filter explicitly includes loopback TCP and UDP traffic. Therefore, the statement that loopback IPC is not seen by the filter at all may be too broad unless it refers to a narrower teardown scenario confirmed by instrumentation.

This does not affect the main diagnosis. FIN/RST-based cache eviction is not reliable enough to define the lifetime of a source-port decision. Invalidating the decision when a fresh SYN is observed is still the correct ownership boundary.

[steveepreston]

I faced 2 issues:

1. ProxyBridge randomly ignoring rules. regardless of time spending.
2. ProxyBridge sounds has a background crash after some hours and no more connections handled. my first idea was it's connection logs getting huger and huger.

due these critical issues, sadly i stopped using ProxyBridge for now.

[rsyuzyov]

Thanks @dentatli — that is a useful correction, and it lines up with what the instrumentation showed.

You are right about the teardown path. After port_clear(sp) on FIN/RST, port_is_direct(sp) is false, so the same iteration falls through to check_process_rule() and re-caches DIRECT through port_set_direct(sp). An outbound FIN/RST therefore does not actually leave the entry cleared, which explains why I kept seeing populated decisions in the logs long after teardown. My wording about loopback was too broad as well — loopback TCP/UDP is explicitly in the WinDivert filter, so that part of my reasoning was wrong even though the conclusion holds: FIN/RST is not a reliable lifetime boundary for a source-port decision, and a fresh SYN is.

Simplifying the teardown path the way you describe — send the previously DIRECT FIN/RST unmodified after clearing, without a second PID/rule lookup — looks right to me too.

Your #209 reads like the same underlying defect one layer down. The port-decision bitmaps are keyed on the numeric source port, and connection_hash_table is keyed on src_port alone, so in both cases flow identity is incomplete and an unrelated later flow inherits state. The SYN eviction in pr/cache-fix only bounds how long a decision can live; it does not make the identity itself correct, so it should be complementary to your change rather than overlapping with it.

[hewzhew]

I independently reproduced this on Windows 11 with ProxyBridge v4.0.0 after
about 13 hours of uptime. The active rule included node_repl.exe -> PROXY,
but repeated fresh TCP connections from node_repl.exe to port 443 remained in
SYN_SENT on the direct destination. A browser navigation plus DOM snapshot
took about 43.3 seconds because the direct telemetry requests timed out.

Fully restarting ProxyBridge cleared the stale state: the same end-to-end check
dropped to 0.93 seconds (908 ms navigation, 24 ms DOM snapshot), and no direct
node_repl.exe TCP/443 connection was observed.

I also reviewed @rsyuzyov's pr/cache-fix commit 20e46022 and rebased it
unchanged onto the current dev commit 9c9dd1d. The stable patch ID is
identical, git diff --check passes, and the Windows x64 core DLL builds
successfully with MSVC 2022 and WinDivert 2.2.2-A.

@rsyuzyov, would you like to open the PR from your branch? That would be my
preferred path. If not, with your permission and maintainer approval I can
submit the rebased commit while preserving your authorship and Signed-off-by.
I would scope it strictly to #206 and list #209 only as related work.

[rsyuzyov]

@hewzhew Thanks for reproducing it and for rebasing the commit — I've opened the PR from pr/cache-fix against dev myself: #215. It is scoped strictly to #206, with #209 listed as related work only.

One thing worth setting expectations on: I don't think a PR is the fast path here. My earlier PR #165 has been open since June without maintainer feedback, and as far as I can tell no external PR has ever been merged into this repo — every merged PR is the maintainer's own. Issues do get answered (#207 and #208 were closed within two days), so this isn't inattention; upstream simply doesn't seem to take outside code.

That's why I ended up building for myself rather than waiting: https://github.com/rsyuzyov/ProxyBridge/releases/tag/v4.0.13-portcachefix — running here since 22 July, no leaks. @steveepreston, that may be relevant to you, since symptom (1) in your comment is exactly this bug.

If the PR stalls, you have my permission upfront to submit the rebased commit yourself — attribution is genuinely not a concern here. I didn't hand-write a line of that patch: Claude Opus did, under my direction, instrumentation and 63 hours of soak testing. CONTRIBUTING already welcomes AI-assisted code, so if any byline is deserved, that's the honest one. The one thing I'd keep is the Signed-off-by — that's a DCO provenance statement, not a credit line.