VXLAN: Loss of Connectivity Between Two VMs After Idle Time on KVM Hosts – CloudStack 4.20

[lexmora]

Description:

Hello,

I hope you're doing well, I've recently returned to using CloudStack after a few years, and I'm currently facing a VXLAN connectivity issue between two virtual machines deployed on different KVM hosts.

Issue Summary:

• Two VM's are deployed on separate KVM Hosts.
• VXLAN bridges and interfaces are correctly created by CLoudStack.
• Initial connectivity work fine -- VM's can ping each other and the gateway (Virtual Router).
• After stopping traffic between them and waiting appoximately 5 minutes, the VMs can no loger communicate with each other.
• The issue affects both directions (bi'directional connectivity loss).
• The bridge fdb show command no longer shows the remote MAC addresses associated with the VXLAN interface.
• This leads to traffic being dropped, and ping fails to resume.

Temporary Workaround:
The only way to restore traffic is by manually bringing down and up the physical interface on the bond carrying the VXLAN traffic.

Environment Details:

• 3 KVM Hosts (Kernel 6.2.0)

• CloudStack version 4.20.0.0

• Advaced Zone

• Traffic types configured:

  • Management: VLAN
  • Storage: VLAN
  • Guest: VXLAN
  • public VLAN

• VXLAN is configured using multicast
  

• Native Linux bridge is in use (not OVS)

• IGMP Snooping is disable on the physical switches

Concers and Question:

Why does VXLAN stop learning or maintaining remote VM MAC addresses after idle time?
is this a know issue with VXLAN multicast setup?
Are there kernel or bridge parameters that should be adjusted (e.g., aging time)?

Thanks you in advance for your help and support