New setup IPFIX isse unexpected EOF

[dietybright]

I am setting up golow2 using docker compose kcg,

Cisco router IOS-XE send ipfix to golow2.

Prometheus query no data, there is a error message on goflow2 console log, I am using ipfix not netflow, appreciated for any help

level=ERROR msg=error scheme=netflow hostname="" port=2055 count=1 workers=2 blocking=false queue_size=1000000 error="receiver: message from [::ffff:172.16.x.3]:51206 unexpected EOF

[lspgn]

I would need a packet capture to help more.
It seems it's not an ipfix packet

[dietybright]

Hi @lspgn ,

This is the data template pcap

Frame 517: 166 bytes on wire (1328 bits), 166 bytes captured (1328 bits)
Ethernet II, Src: Cisco_1b:7c:f4 (c0:14:fe:1b:7c:f4), Dst: VMware_89:45:9d (00:50:56:89:45:9d)
Internet Protocol Version 4, Src: 172.16.28.2, Dst: 172.16.28.12
User Datagram Protocol, Src Port: 63266, Dst Port: 2055
Cisco NetFlow/IPFIX
    Version: 10
    Length: 124
    Timestamp: Feb  7, 2025 15:09:11.000000000 Malay Peninsula Standard Time
        ExportTime: 1738912151
    FlowSequence: 317317
    Observation Domain Id: 512
    Set 1 [id=2] (Data Template): 260
        FlowSet Id: Data Template (V10 [IPFIX]) (2)
        FlowSet Length: 108
        Template (Id = 260, Count = 21)
            Template Id: 260
            Field Count: 21
            Field (1/21): IP_SRC_ADDR
                0... .... .... .... = Pen provided: No
                .000 0000 0000 1000 = Type: IP_SRC_ADDR (8)
                Length: 4
            Field (2/21): IP_DST_ADDR
                0... .... .... .... = Pen provided: No
                .000 0000 0000 1100 = Type: IP_DST_ADDR (12)
                Length: 4
            Field (3/21): INPUT_SNMP
                0... .... .... .... = Pen provided: No
                .000 0000 0000 1010 = Type: INPUT_SNMP (10)
                Length: 4
            Field (4/21): IP_DSCP
                0... .... .... .... = Pen provided: No
                .000 0000 1100 0011 = Type: IP_DSCP (195)
                Length: 1
            Field (5/21): PROTOCOL
                0... .... .... .... = Pen provided: No
                .000 0000 0000 0100 = Type: PROTOCOL (4)
                Length: 1
            Field (6/21): L4_SRC_PORT
                0... .... .... .... = Pen provided: No
                .000 0000 0000 0111 = Type: L4_SRC_PORT (7)
                Length: 2
            Field (7/21): L4_DST_PORT
                0... .... .... .... = Pen provided: No
                .000 0000 0000 1011 = Type: L4_DST_PORT (11)
                Length: 2
            Field (8/21): TCP_FLAGS
                0... .... .... .... = Pen provided: No
                .000 0000 0000 0110 = Type: TCP_FLAGS (6)
                Length: 1
            Field (9/21): flowEndReason
                0... .... .... .... = Pen provided: No
                .000 0000 1000 1000 = Type: flowEndReason (136)
                Length: 1
            Field (10/21): biflowDirection
                0... .... .... .... = Pen provided: No
                .000 0000 1110 1111 = Type: biflowDirection (239)
                Length: 1
            Field (11/21): Unknown(12432)
                1... .... .... .... = Pen provided: Yes
                .011 0000 1001 0000 = Type: Unknown (12432)
                Length: 4
                PEN: ciscoSystems (9)
            Field (12/21): Unknown(12434)
                1... .... .... .... = Pen provided: Yes
                .011 0000 1001 0010 = Type: Unknown (12434)
                Length: 4
                PEN: ciscoSystems (9)
            Field (13/21): Unknown(12441)
                1... .... .... .... = Pen provided: Yes
                .011 0000 1001 1001 = Type: Unknown (12441)
                Length: 8
                PEN: ciscoSystems (9)
            Field (14/21): APPLICATION_ID
                0... .... .... .... = Pen provided: No
                .000 0000 0101 1111 = Type: APPLICATION_ID (95)
                Length: 4
            Field (15/21): OUTPUT_SNMP
                0... .... .... .... = Pen provided: No
                .000 0000 0000 1110 = Type: OUTPUT_SNMP (14)
                Length: 4
            Field (16/21): FLOW_SAMPLER_ID
                0... .... .... .... = Pen provided: No
                .000 0000 0011 0000 = Type: FLOW_SAMPLER_ID (48)
                Length: 1
            Field (17/21): Unknown(12433)
                1... .... .... .... = Pen provided: Yes
                .011 0000 1001 0001 = Type: Unknown (12433)
                Length: 4
                PEN: ciscoSystems (9)
            Field (18/21): BYTES
                0... .... .... .... = Pen provided: No
                .000 0000 0000 0001 = Type: BYTES (1)
                Length: 8
            Field (19/21): PKTS
                0... .... .... .... = Pen provided: No
                .000 0000 0000 0010 = Type: PKTS (2)
                Length: 8
            Field (20/21): flowStartMilliseconds
                0... .... .... .... = Pen provided: No
                .000 0000 1001 1000 = Type: flowStartMilliseconds (152)
                Length: 8
            Field (21/21): flowEndMilliseconds
                0... .... .... .... = Pen provided: No
                .000 0000 1001 1001 = Type: flowEndMilliseconds (153)
                Length: 8

This is the data flow pcap

Cisco NetFlow/IPFIX
    Version: 10
    Length: 348
    Timestamp: Feb  7, 2025 15:09:13.000000000 Malay Peninsula Standard Time
        ExportTime: 1738912153
    FlowSequence: 317317
    Observation Domain Id: 512
    Set 1 [id=260] (4 flows)
        FlowSet Id: (Data) (260)
        FlowSet Length: 332
        [Template Frame: 517]
        Flow 1
            SrcAddr: 172.16.22.1
            DstAddr: 192.168.99.10
            InputInt: 26
            DSCP: 0
            Protocol: UDP (17)
            SrcPort: 3383 (3383)
            DstPort: 161 (161)
            TCP Flags: 0x00
                00.. .... = Reserved: 0x0
                ..0. .... = URG: Not used
                ...0 .... = ACK: Not used
                .... 0... = PSH: Not used
                .... .0.. = RST: Not used
                .... ..0. = SYN: Not used
                .... ...0 = FIN: Not used
            Flow End Reason: Idle timeout (1)
            Biflow Direction: Initiator (1)
            Enterprise Private entry: (ciscoSystems) Type 12432: Value (hex bytes): 00 00 00 00
            Enterprise Private entry: (ciscoSystems) Type 12434: Value (hex bytes): 00 00 00 14
            Enterprise Private entry: (ciscoSystems) Type 12441: Value (hex bytes): ec cb 57 a0 00 08 db 61
            Classification Engine ID: IANA-L4 (3)
            Selector ID: 0000a1
            OutputInt: 1
            SamplerID: 0
            Enterprise Private entry: (ciscoSystems) Type 12433: Value (hex bytes): 00 00 00 2c
            Octets: 661
            Packets: 1
            [Duration: 0.000000000 seconds (milliseconds)]
                StartTime: Feb  7, 2025 15:09:02.512000000 Malay Peninsula Standard Time
                EndTime: Feb  7, 2025 15:09:02.512000000 Malay Peninsula Standard Time
        Flow 2
            SrcAddr: 192.168.99.10
            DstAddr: 172.16.22.1
            InputInt: 1
            DSCP: 0
            Protocol: UDP (17)
            SrcPort: 161 (161)
            DstPort: 3383 (3383)
            TCP Flags: 0x00
                00.. .... = Reserved: 0x0
                ..0. .... = URG: Not used
                ...0 .... = ACK: Not used
                .... 0... = PSH: Not used
                .... .0.. = RST: Not used
                .... ..0. = SYN: Not used
                .... ...0 = FIN: Not used
            Flow End Reason: Idle timeout (1)
            Biflow Direction: ReverseInitiator (2)
            Enterprise Private entry: (ciscoSystems) Type 12432: Value (hex bytes): 00 00 00 2c
            Enterprise Private entry: (ciscoSystems) Type 12434: Value (hex bytes): 00 00 00 14
            Enterprise Private entry: (ciscoSystems) Type 12441: Value (hex bytes): ec cb 57 a0 00 08 db 61
            Classification Engine ID: IANA-L4 (3)
            Selector ID: 0000a1
            OutputInt: 26
            SamplerID: 0
            Enterprise Private entry: (ciscoSystems) Type 12433: Value (hex bytes): 00 00 00 00
            Octets: 769
            Packets: 1
            [Duration: 0.000000000 seconds (milliseconds)]
                StartTime: Feb  7, 2025 15:09:02.520000000 Malay Peninsula Standard Time
                EndTime: Feb  7, 2025 15:09:02.520000000 Malay Peninsula Standard Time
        Flow 3
            SrcAddr: 192.168.97.128
            DstAddr: 172.16.19.12
            InputInt: 1
            DSCP: 24
            Protocol: TCP (6)
            SrcPort: 50192 (50192)
            DstPort: 5060 (5060)
            TCP Flags: 0x18, ACK, PSH
                00.. .... = Reserved: 0x0
                ..0. .... = URG: Not used
                ...1 .... = ACK: Used
                .... 1... = PSH: Used
                .... .0.. = RST: Not used
                .... ..0. = SYN: Not used
                .... ...0 = FIN: Not used
            Flow End Reason: Idle timeout (1)
            Biflow Direction: Initiator (1)
            Enterprise Private entry: (ciscoSystems) Type 12432: Value (hex bytes): 00 00 00 2c
            Enterprise Private entry: (ciscoSystems) Type 12434: Value (hex bytes): 00 00 00 14
            Enterprise Private entry: (ciscoSystems) Type 12441: Value (hex bytes): ed 07 13 b0 00 08 27 9f
            Classification Engine ID: PANA-L7 (13)
            Selector ID: 0006e3
            OutputInt: 26
            SamplerID: 0
            Enterprise Private entry: (ciscoSystems) Type 12433: Value (hex bytes): 00 00 00 00
            Octets: 1170
            Packets: 3
            [Duration: 0.009000000 seconds (milliseconds)]
                StartTime: Feb  7, 2025 15:09:02.690000000 Malay Peninsula Standard Time
                EndTime: Feb  7, 2025 15:09:02.699000000 Malay Peninsula Standard Time
        Flow 4
            SrcAddr: 172.16.19.12
            DstAddr: 192.168.97.128
            InputInt: 26
            DSCP: 24
            Protocol: TCP (6)
            SrcPort: 5060 (5060)
            DstPort: 50192 (50192)
            TCP Flags: 0x18, ACK, PSH
                00.. .... = Reserved: 0x0
                ..0. .... = URG: Not used
                ...1 .... = ACK: Used
                .... 1... = PSH: Used
                .... .0.. = RST: Not used
                .... ..0. = SYN: Not used
                .... ...0 = FIN: Not used
            Flow End Reason: Idle timeout (1)
            Biflow Direction: ReverseInitiator (2)
            Enterprise Private entry: (ciscoSystems) Type 12432: Value (hex bytes): 00 00 00 00
            Enterprise Private entry: (ciscoSystems) Type 12434: Value (hex bytes): 00 00 00 14
            Enterprise Private entry: (ciscoSystems) Type 12441: Value (hex bytes): ed 07 13 b0 00 08 27 9f
            Classification Engine ID: PANA-L7 (13)
            Selector ID: 0006e3
            OutputInt: 1
            SamplerID: 0
            Enterprise Private entry: (ciscoSystems) Type 12433: Value (hex bytes): 00 00 00 2c
            Octets: 1166
            Packets: 2
            [Duration: 0.002000000 seconds (milliseconds)]
                StartTime: Feb  7, 2025 15:09:02.694000000 Malay Peninsula Standard Time
                EndTime: Feb  7, 2025 15:09:02.696000000 Malay Peninsula Standard Time

[lspgn]

Could you send it as a .pcap, otherwise I won't be able to replay it in an attempt to reproduce it.

[dietybright]

Hi @lspgn ,

Kindly check attached

pcap2.zip

[lspgn]

@dietybright
Thank you,
I tried replaying the two packets and the four samples correctly show up.

I am guessing there are some healthchecks or bad packets that cannot be decoded and are logged. But the samples are fine.
Tried this version: f0ea9c3

[dietybright]

Hi @lspgn , sorry for late, it still same error message

[lspgn]

Hi @dietybright,
Apologies for the delay
I think you will need to provide me with a longer packet capture. I am not able to reproduce.

[dietybright]

Hi @lspgn ,

I just ran the tcpdump on goflow2, kindly check attached.

pcap3.zip

[lspgn]

@dietybright thank you
I tried again with the same setup but I cannot reproduce.
Have you tried git pull followed by docker-compose build --no-cache just in case?

How often does the error message happen?

[dietybright]

Hi @lspgn ,

Yes did git pul ....

This is new setup for 1 Cisco SDWAN router, all flows received having this same error

[lspgn]

Which OS are you using?

[dietybright]

Hi @lspgn ,

It is Cisco IOS XE Software, Version 17.12.04a

[lspgn]

And on the collector side? Is it Linux?

[dietybright]

Yes, it is Ubuntu 22.04.2 LTS

[eobgolts]

I have a similar problem: error messages repeat every 30 seconds, consistent with sending an Option Template (configured to repeat every 30s on device side).

Netflow v9, Cisco IOS XE 17.09.05e, goflow2 collector - container inside Kubernetes

06T06:07:55.881Z level=ERROR msg=error scheme=netflow hostname="" port=2055 count=1 workers=2 blocking=false queue_size=1000000 error="receiver: message from [::ffff:172.16.14.240]:52795 unexpected EOF"
time=2026-01-06T06:08:25.879Z level=ERROR msg=error scheme=netflow hostname="" port=2055 count=1 workers=2 blocking=false queue_size=1000000 error="receiver: message from [::ffff:172.16.14.240]:52795 unexpected EOF"
time=2026-01-06T06:08:55.877Z level=ERROR msg=error scheme=netflow hostname="" port=2055 count=1 workers=2 blocking=false queue_size=1000000 error="receiver: message from [::ffff:172.16.14.240]:52795 unexpected EOF"
time=2026-01-06T06:09:25.878Z level=ERROR msg=error scheme=netflow hostname="" port=2055 count=1 workers=2 blocking=false queue_size=1000000 error="receiver: message from [::ffff:172.16.14.240]:52795 unexpected EOF"

nflow.zip