Unstable ping RTT with ethernet ipv4 networking

[vakulgarg]

The ping response time suddenly degrades with ipv4/ethernet on nxp-frdm-k64f board.
To reproduce the issue, run samples/net/echo_client with CONF_FILE=prj_frdm_k64f.conf.
Run 'ping 192.0.2.1' from the machine configured with PEER_IPV4_ADDR=192.0.2.2.

For first 60-70 ping requests, the response time is about .2ms.
Then suddenly, ping response time degrades to more than 200ms and at times goes as high as 1s.

(The problem does not reproduce on qemu.)

[jukkar]

I could not reproduce this. Both IPv4 and IPv6 ping work consistently in my frdm-k64f.

--- 2001:db8::3 ping statistics ---
310 packets transmitted, 310 received, 0% packet loss, time 316435ms
rtt min/avg/max/mdev = 0.205/0.241/0.543/0.033 ms

--- 192.0.2.3 ping statistics ---
322 packets transmitted, 322 received, 0% packet loss, time 328720ms
rtt min/avg/max/mdev = 0.152/0.193/0.747/0.040 ms

[vakulgarg]

Jukaar, did you connect the k64f board directly to Linux using cross cable? In my setup, board is on local lan and could receive many ARP broadcasts. But it always show spike in RTT after a couple of minutes.

[vakulgarg]

The problem is easily reproducible at my end when the ethernet hub between frdm-k64f and ping linux machine is also connected to local LAN. Once ping RTT goes up, then even when I disconnected the hub from local LAN, it does not recover.

[b16394@lti openssl_ktls]$ ping 192.0.2.1
PING 192.0.2.1 (192.0.2.1) 56(84) bytes of data.
64 bytes from 192.0.2.1: icmp_seq=1 ttl=64 time=0.196 ms
64 bytes from 192.0.2.1: icmp_seq=2 ttl=64 time=0.181 ms
64 bytes from 192.0.2.1: icmp_seq=3 ttl=64 time=0.207 ms
64 bytes from 192.0.2.1: icmp_seq=4 ttl=64 time=0.181 ms
64 bytes from 192.0.2.1: icmp_seq=5 ttl=64 time=0.199 ms
64 bytes from 192.0.2.1: icmp_seq=6 ttl=64 time=0.184 ms
64 bytes from 192.0.2.1: icmp_seq=7 ttl=64 time=0.160 ms
64 bytes from 192.0.2.1: icmp_seq=8 ttl=64 time=0.192 ms
64 bytes from 192.0.2.1: icmp_seq=9 ttl=64 time=0.193 ms
64 bytes from 192.0.2.1: icmp_seq=10 ttl=64 time=0.210 ms
64 bytes from 192.0.2.1: icmp_seq=11 ttl=64 time=0.169 ms
64 bytes from 192.0.2.1: icmp_seq=12 ttl=64 time=0.205 ms
64 bytes from 192.0.2.1: icmp_seq=13 ttl=64 time=0.188 ms
64 bytes from 192.0.2.1: icmp_seq=14 ttl=64 time=0.186 ms
64 bytes from 192.0.2.1: icmp_seq=15 ttl=64 time=0.143 ms
64 bytes from 192.0.2.1: icmp_seq=16 ttl=64 time=0.168 ms
64 bytes from 192.0.2.1: icmp_seq=17 ttl=64 time=0.169 ms
64 bytes from 192.0.2.1: icmp_seq=18 ttl=64 time=0.172 ms
64 bytes from 192.0.2.1: icmp_seq=19 ttl=64 time=0.186 ms
64 bytes from 192.0.2.1: icmp_seq=20 ttl=64 time=0.176 ms
64 bytes from 192.0.2.1: icmp_seq=21 ttl=64 time=0.187 ms
64 bytes from 192.0.2.1: icmp_seq=22 ttl=64 time=0.168 ms
64 bytes from 192.0.2.1: icmp_seq=23 ttl=64 time=0.150 ms
64 bytes from 192.0.2.1: icmp_seq=24 ttl=64 time=0.166 ms
64 bytes from 192.0.2.1: icmp_seq=25 ttl=64 time=0.167 ms
64 bytes from 192.0.2.1: icmp_seq=26 ttl=64 time=0.173 ms
64 bytes from 192.0.2.1: icmp_seq=27 ttl=64 time=0.177 ms
64 bytes from 192.0.2.1: icmp_seq=28 ttl=64 time=0.187 ms
64 bytes from 192.0.2.1: icmp_seq=29 ttl=64 time=0.166 ms
64 bytes from 192.0.2.1: icmp_seq=30 ttl=64 time=0.197 ms
64 bytes from 192.0.2.1: icmp_seq=31 ttl=64 time=0.180 ms
64 bytes from 192.0.2.1: icmp_seq=32 ttl=64 time=0.174 ms
64 bytes from 192.0.2.1: icmp_seq=33 ttl=64 time=521 ms
64 bytes from 192.0.2.1: icmp_seq=34 ttl=64 time=270 ms
64 bytes from 192.0.2.1: icmp_seq=35 ttl=64 time=19.1 ms
64 bytes from 192.0.2.1: icmp_seq=36 ttl=64 time=19.1 ms
64 bytes from 192.0.2.1: icmp_seq=37 ttl=64 time=334 ms
64 bytes from 192.0.2.1: icmp_seq=38 ttl=64 time=83.3 ms
64 bytes from 192.0.2.1: icmp_seq=39 ttl=64 time=81.8 ms
64 bytes from 192.0.2.1: icmp_seq=40 ttl=64 time=60.2 ms
64 bytes from 192.0.2.1: icmp_seq=41 ttl=64 time=12.3 ms
64 bytes from 192.0.2.1: icmp_seq=42 ttl=64 time=360 ms
64 bytes from 192.0.2.1: icmp_seq=43 ttl=64 time=326 ms
^C
--- 192.0.2.1 ping statistics ---
44 packets transmitted, 43 received, 2% packet loss, time 43811ms
rtt min/avg/max/mdev = 0.143/48.757/521.860/119.156 ms

Once, I also got below error prints.

shell> [dev/eth_mcux] [ERR] eth_rx: Failed to get fragment buf
[dev/eth_mcux] [ERR] eth_rx: Failed to get fragment buf
[dev/eth_mcux] [ERR] eth_rx: Failed to get fragment buf
[dev/eth_mcux] [ERR] eth_rx: Failed to get fragment buf
[dev/eth_mcux] [ERR] eth_rx: Failed to get fragment buf
[dev/eth_mcux] [ERR] eth_rx: Failed to get fragment buf
[dev/eth_mcux] [ERR] eth_rx: Failed to get fragment buf
[dev/eth_mcux] [ERR] eth_rx: Failed to get fragment buf
[dev/eth_mcux] [ERR] eth_rx: Failed to get fragment buf
[dev/eth_mcux] [ERR] eth_rx: Failed to get fragment buf
[dev/eth_mcux] [ERR] eth_rx: Failed to get fragment buf
[dev/eth_mcux] [ERR] eth_rx: Failed to get fragment buf
[dev/eth_mcux] [ERR] eth_rx: ENET_GetRxFrameSize return: 4000

[jukkar]

Jukaar, did you connect the k64f board directly to Linux using cross cable? In my setup, board is on local lan and could receive many ARP broadcasts. But it always show spike in RTT after a couple of minutes.

No, I have a switch between frdm and linux host.

[jukkar]

the ethernet hub

Do you really mean a hub (these are quite rare nowadays) or a switch, I was only testing with a switch.

[vakulgarg]

It is a switch.

[jukkar]

This could be related to possible network traffic in your system. Anyway, one option is to increase the number of buffers as you seem to run out of them. Please try #7775 if it helps the situation.

[vakulgarg]

Jukkar, I can understand that sometimes buffer may get exhausted. But once RTT has increased, it never recovers back to normal (even if I isolate the switch from local lan).

[pfalcon]

For first 60-70 ping requests, the response time is about .2ms.
Then suddenly, ping response time degrades to more than 200ms and at times goes as high as 1s.

This is #3132, right on.