It seems that the af_packet code from gopacket never puts the interface in promiscuous mode, which results in Packetbeat not seeing any packets when using mirror ports. The workaround is simple, make sure to run this command before starting packetbeat:

 ip link set enp5s0f1 promisc on

We should try to do the same programmatically from the af_packet code, so this is no longer required.