Use NXAST_CONTROLLER2 and NXT_PACKET_IN2 in PacketInHandler

[wenyingd]

Describe what you are trying to solve

In the existing antrea code, a feature needs to use action NXAST_CONTROLLER in a flow and register a PacketInHandler with a desired reason if it is expected to leverage OpenFlow packet_in mechanism to perform special actions, e.g., TraceFlow, ANP with FQDN, etc.

There exists limitations in the available reasons in OVS packets:

1. OVS has limited vlues in packet_in.reason field, the stand values include [0, 5], although the field type is uint8. After testing with OVS 2.17.0, only value 0/1/3/4/5 could work with packet_in message, and value 2 may cause OVS fails to send the packet to Antrea Agent via the UDS or Windows named pipe.
2. Only reason 0 and 1 are used in antrea, because the previous OVS versions have issues with other values, e.g., the packetIn message fails to receive when using other values.

To levearge the packet_in mechanism, antrea uses reason 0 and 1, and use reg0 to extend for different mechanism:

1. Reason==1 is used by Traceflow feature
2. Reason==0 is shared by all other features, ANP(logging/reject/FQDN)/Multicast/Service
3. reg0[13..17] is used to indicate the reasons of sending packet to the controller

The weaknesses of the existing solution include,

1. A mix usage of packetIn.reason and OVS regs: packetIn.reason is used to register PacketInHandler and dispatch the packetIn messages to handlers.
2. PacketInHandler performance. For a handler using the shared reasons, it may receive messages that it is not focusing on, and a second filter is needed based on the OVS regs
3. The extendability of OVS regs. Each time a new PacketInHandler is introduced, a new bit in reg0 needs to be reserved

Describe the solution you have in mind

A new design for the packetIn usage is proposed in this doc by introducing NXAST_CONTROLLER2 and NXT_PACKET_IN2 in the desired Flow action and packetIn format.

NXAST_CONTROLLER2 action adds a new field userdata whose type is []byte comparing to NXAST_CONTROLLER, which has little limitation on the usage. Hence, antrea can leverage it in the flow action "controller" to substitute the previous customization on reg0. eg.,

cookie=0x5, duration=95.734s, table=0, priority=100,tcp,nw_src=1.1.1.2,nw_dst=2.2.2.1 actions=set_field:0x1234/0xffff->reg0,set_field:0xaaaa00/0xffff00->reg1,controller(max_len=128,id=13342,userdata=01.02.03)

The userdata value could be taken by a NXT_PACKET_IN2 message. So antrea can use it as the key to dispatch the messages to handler. In this way, we could implement a flat organization for multiple packetIn handlers.

Describe how your solution impacts user flows

PacketInHandlers don't need to use reg0 to customize reasons any more (including set value in OpenFlow actions and parse values from packetIn message).

Describe the main design/architecture of your solution

To use NXAST_CONTROLLER2 and NXT_PACKET_IN2 in PacketInHandler, following steps are needed:

1. Set PacketInFormat with value openflow15.OFPUTIL_PACKET_IN_NXT2 on br-int each time antrea-agent connected to OVS
2. Use OpenFlow message NXAST_CONTROLLER2 in the flow action SendToController when expecting to send the packet to antrea-agent. In the meanwhile, supporting to set userdata in the parameter.
3. Use OpenFlow message NXT_PACKET_IN2 for ofctrl.PacketIn
4. Use the userdata as a key to register PacketInHandler or dispatch packetIn mesasge
5. Remove stale code in the existing handlers, including the logics to set or parseOVS regs which are used to represent the customized reason.

Test plan

1. Update the existing unit test and integration code as the corresponding flow actions and the parsers in packetIn messages are changed.
2. The existing e2e test is supposed to work for the new design.

Additional context

[github-actions]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment, or this will be closed in 90 days

[tnqn]

@wenyingd I suppose this issue can be closed?

[wenyingd]

Yes.