Introduction
Broadcom’s current generation of XGS switches supports a capability called “Flow Tracker” which monitors statistics for IP Flows through the switch and exports information about these flows in IPFIX format (IP Flow Information for Export, see RFCs 3917, 7011 – 7015, and 5103). Broadcom’s DNX switches do not support this capability and there are several aspects that make offloading challenging, including memory limitations and processing and CPU limitations. Recognizing IP Flows, monitoring them, collecting statistics on them, and generating IPFIX records are challenging—and these need to be done at line rate in these terabit switches.
DNX customers, nonetheless, require these capabilities, and Northforge has developed a solution to meet this requirement.
Design Approach
The solution uses the port mirroring capability on a DNX switch such as the Qumran or Jericho. An egress port can be mirrored to another port and this capability is frequently used for troubleshooting network problems. But in this case, we use this capability to feed an external component, a network processor (NPU) or an FPGA, that performs the IP flow management. The NPU/FPGA detects, monitors, recognizes IP flows and builds statistics tables for them. Finally, it exports IPFIX records at regular intervals that can be collected and post-processed. The process looks like this.
Frames enter a DNX switch on one or more ingress ports. The switching function directs some of the packets to a monitored egress port (for a layer 2 switch this would be based on MAC-layer addressing and for a layer 3 router it would be based on IP address). The monitored egress port has its traffic mirrored to another port that has the NPU/FPGA attached.
The frames that enter the NPU/FPGA are inspected to ensure that they are IP frames (and discarded if not) and then a flow is identified based on an IP 5-tuple (source IP address, destination IP address, source IP port, destination IP port, layer 4 protocol). The NPU/FPGA maintains a flow table and statistics are collected for each flow. When a flow terminates or times out an IPFIX record is exported for storage and subsequent post-processing.
Summary
At IP Infusion we have developed and implemented this innovative solution with Qumran AX (BCM88470) using SDK 6.5.14. We used both a Broadcom’s BCM5871x NPU and an FPGA for flow processing and an open source IPFIX library.