How to filter NetFlow with nfreplay, including NAT Events?

[n-lyakhovoy]

Hello, i'm trying to capture NetFlow v9 (NAT translation from proprietary hardware). I need to filter some subnets and send to another NetFlow colletror.

When i run:
nfcapd -T nsel,nel -l /var/log/nat/netflow/ -p 5556 -t 2 -x "nfreplay -r /var/log/nat/netflow/%f -f /srv/scripts/nfdump.filter -v 9"

i got NetFlow without right template and fields:

Flow Record:
Flags = 0x06 NETFLOW v9, Unsampled
label =
export sysid = 1
size = 56
first = 1602186349 [2020-10-08 22:45:49]
last = 1602186349 [2020-10-08 22:45:49]
msec_first = 0
msec_last = 0
src addr = 10.4.218.117
dst addr = 31.13.72.8
src port = 44460
dst port = 443
fwd status = 0
tcp flags = 0x00 ........
proto = 6 TCP
(src)tos = 0
(in)packets = 0
(in)bytes = 0

Original data is like that:

Flow Record:
Flags = 0x46 EVENT, Unsampled
label =
export sysid = 10
size = 100
first = 1602186433 [2020-10-08 22:47:13]
last = 1602186433 [2020-10-08 22:47:13]
msec_first = 0
msec_last = 0
src addr = 10.4.80.208
dst addr = 64.233.161.157
src port = 46963
dst port = 443
fwd status = 0
tcp flags = 0x00 ........
proto = 6 TCP
(src)tos = 0
(in)packets = 0
(in)bytes = 0
connect ID = 0
fw event = 1: CREATE
fw ext event = 0: Ignore
secgroup tag = 0
Event time = 1602186433000 [2020-10-08 22:47:13.000]
src xlt port = 5271
dst xlt port = 443
src xlt ip = 31.185.7.194
dst xlt ip = 64.233.161.157
nat event = 1: CREATE
ingress VRF = 0
egress VRF = 0

How to resend filtered netflow with NEL/NSEL data using nfreplay or may be other open source projects?

[phaag]

Currently nfreplay does not support replaying NSEL/NAT event records. It only supports plain v9 without those event specific fields.

[phaag]

nfdump-1.7.x may be able to forward more special NAT/NSEL elements

[n-lyakhovoy]

Hello!
It's a goot news, but can't build 1.7 unicorn :)

https://pastebin.com/LW24At9F

make all-recursive
make[1]: Entering directory '/usr/src/nfdump-unicorn'
Making all in .
make[2]: Entering directory '/usr/src/nfdump-unicorn'
make[2]: Leaving directory '/usr/src/nfdump-unicorn'
Making all in bin
make[2]: Entering directory '/usr/src/nfdump-unicorn/bin'
make all-am
make[3]: Entering directory '/usr/src/nfdump-unicorn/bin'
/bin/bash ../libtool --tag=CC --mode=link gcc -g -O3 -std=gnu11 -Wall -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wmissing-noreturn -fno-strict-aliasing -DNSEL -o nfcapd nfcapd-nfcapd.o nfcapd-nfstatfile.o nfcapd-launch.o nfcapd-nfnet.o nfcapd-collector.o nfcapd-netflow_v1.o nfcapd-netflow_v5_v7.o nfcapd-netflow_v9.o nfcapd-ipfix.o nfcapd-bookkeeper.o nfcapd-expire.o -lnfdump -lresolv -lbz2
libtool: link: gcc -g -O3 -std=gnu11 -Wall -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wmissing-noreturn -fno-strict-aliasing -DNSEL -o .libs/nfcapd nfcapd-nfcapd.o nfcapd-nfstatfile.o nfcapd-launch.o nfcapd-nfnet.o nfcapd-collector.o nfcapd-netflow_v1.o nfcapd-netflow_v5_v7.o nfcapd-netflow_v9.o nfcapd-ipfix.o nfcapd-bookkeeper.o nfcapd-expire.o /usr/src/nfdump-unicorn/bin/.libs/libnfdump.so -lresolv -lbz2
/usr/src/nfdump-unicorn/bin/.libs/libnfdump.so: undefined reference to pthread_detach' /usr/src/nfdump-unicorn/bin/.libs/libnfdump.so: undefined reference to __atomic_compare_exchange_16'
/usr/src/nfdump-unicorn/bin/.libs/libnfdump.so: undefined reference to __atomic_load_16' /usr/src/nfdump-unicorn/bin/.libs/libnfdump.so: undefined reference to pthread_join'
/usr/src/nfdump-unicorn/bin/.libs/libnfdump.so: undefined reference to __atomic_store_16' /usr/src/nfdump-unicorn/bin/.libs/libnfdump.so: undefined reference to pthread_create'
collect2: error: ld returned 1 exit status
Makefile:946: recipe for target 'nfcapd' failed
make[3]: *** [nfcapd] Error 1
make[3]: Leaving directory '/usr/src/nfdump-unicorn/bin'
Makefile:804: recipe for target 'all' failed
make[2]: *** [all] Error 2
make[2]: Leaving directory '/usr/src/nfdump-unicorn/bin'
Makefile:413: recipe for target 'all-recursive' failed
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory '/usr/src/nfdump-unicorn'
Makefile:345: recipe for target 'all' failed

[n-lyakhovoy]

Seems to be missing pthread and atomic libs in Makefile-s
LIBS = -lresolv -lbz2 -lpthread -latomic

I have success to build binaries,

nfcapd -V

nfcapd: Version: 1.7.0

nfreplay seems not support NEL/NSEL

Sample data in dump
low Record:
Flags = 0x00 FLOW, Unsampled
Elements = 6: 1 2 13 21 23 26
size = 132
engine type = 0
engine ID = 10
export sysid = 6
first = 0 [.000]
last = 0 [.000]
received at = 1603802049510 [2020-10-27 15:34:09.510]
proto = 6 TCP
tcp flags = 0x00 ........
src port = 48796
dst port = 443
src tos = 0
in packets = 0
in bytes = 0
src addr = 10.4.221.187
dst addr = 94.100.185.164
ip exporter = 10.4.1.15
src xlt ip = 31.185.7.202
dst xlt ip = 94.100.185.164
src xlt port = 5542
dst xlt port = 443
nat event = 1: ADD
Event time = 1603802049000 [2020-10-27 15:34:09.000]
ingress VRF = 0
egress VRF = 0

data after nfreplay looks like

Flow Record:
Flags = 0x00 FLOW, Unsampled
Elements = 3: 1 2 13
size = 84
engine type = 91
engine ID = 85
export sysid = 1
first = 0 [.000]
last = 0 [.000]
received at = 1603803946931 [2020-10-27 16:05:46.931]
proto = 6 TCP
tcp flags = 0x00 ........
src port = 48788
dst port = 443
src tos = 0
in packets = 0
in bytes = 0
src addr = 10.4.221.187
dst addr = 94.100.185.164
ip exporter = 127.0.0.1

May be i'm doing something wrong?

[phaag]

No - you are doing all right. But these extension for NSEL/NAT forwarding are not yet in.

[phaag]

With the code changes in that latest master repo, you can forward records natively with nfdump protocol 250. Use
nfreplay -v 250 .... and make sure the receiving end is an up to date nfcapd.

[phaag]

As this is now integrated, I close the ticket.