Control Plane Protection¶

Overview¶

This guide describes how to enable and configure the Control Plane Protection mechanism. Enabling this feature reduces the risk of dropping control packets when the target is under high load, or when the transmission link is overloaded.

In a network architecture, control packets are critical, since losing some of them has stronger consequences than losing data packets. For instance:

losing ARP packets can make a gateway unreachable
losing OSPF/BGP/… packets can make a network unreachable
losing IKE packets can prevent the setup of IPsec security associations
losing LACP packets can bring a link down

Control Plane Protection is a software mechanism that reduces the risk of dropping these control packets. It has an impact on performance, which can be tuned depending on the required throughput and criticity of losing control packets.

This guide describes how to enable and configure this Control Plane Protection mechanism.

Recognized packet types¶

The software parser recognizes ARP, ICMP, ICMPv6, OSPF, VRRP, IKE, DHCP, DHCPv6, BGP, LACP, SSH, OpenFlow, JSON RPC (TCP port 7406), Stats Collector (TCP port 39090), BFD, DPVI packets. All can be encapsulated in VLAN, QinQ or FPTUN.

Supported platforms¶

This feature is only available on products based on DPDK. It requires a specific PMD API (called RX/TX descriptor status) which is available for the following PMDs: i40e, ixgbe, mlx5, e1000.

Usage¶

Control Plane Protection is disabled by default. If control plane packets are lost on a given port, enable Control Plane Protection using the following command:

root@host# fp-cli dpdk-cp-filter-set eth0 enable
rx cp filter is enabled: rxd_thres=256, rxd_count=512
tx cp filter is enabled: txd_thres=256, txd_count=512

To enable Control Plane Protection automatically on start, you can specify FPN-SDK option in fast-path.env:

root@host# vi /etc/fast-path.env
  [...]
  FPNSDK_OPTIONS="--rx-cp-filter=enable --tx-cp-filter=enable"
  [...]

Control Plane Protection works according to a maximum CPU budget. If control plane packets are still dropped after enabling Control Plane Protection, it means that this budget has to be increased using the following command:

root@host# fp-cli dpdk-cp-filter-budget-set 15
cpu budget is 15%

Design¶

Packet loss can occur at several places.

RX: when the router is overloaded, the software may not able to dequeue the incoming packets fast enough. In that case, the hardware RX ring becomes full and the NIC starts to drop packets.
TX: if the router tries to send more packets than what the network link support, the hardware TX ring becomes full and the software starts to drop packets.
Exceptions (packets sent to Linux): when the exception rate is too high, the software ring becomes full and the software starts to drop packets.

Control Plane Protection is disabled by default in RX and TX, and always enabled for exceptions.

When enabled, the Control Plane Protection mechanism checks the filling level of the rings (RX, TX, or Exception) before enqueuing or dequeuing.

If the filling level is below a threshold, nothing else is done and the packets are processed normally. Therefore, the additional CPU cost is very low in this situation.
Else, if the filling level is above the threshold, the retrieved/transmitted packets are filtered: the data plane packets are dropped and the control plane are kept. Since most packets are data plane, the ring is emptied quickly and its filling level falls back below the threshold.

The filtering of packets is done by a software parser. This gives more flexibility, but it implies a cost in terms of CPU cycles. To avoid spending all the CPU time to filter and drop without doing any effective processing:

It is advised to have at least one core per 10Gb link: this ensures that even in the worst case, most CPU power is available for the processing application.
A maximum CPU budget can be assigned to the Control Plane Protection to limit the impact on throughput performance. If the limit is exceeded, only a fraction of the packets will be filtered and the Control Plane Protection will be disabled for the other part. Therefore, control plane packets can be dropped in this situation.

Statistics¶

Control Plane Protection provides statistics to monitor the number of filtered packets.

RX and TX Control Plane Protection statistics¶

The RX and TX Control Plane Protection statistics are available through the ethtool command.

root@host# ethtool -S eth0
  {...}
  fpn.rx_cp_passthrough: 0
  fpn.rx_cp_kept: 0
  fpn.rx_dp_drop: 0
  fpn.rx_cp_overrun: 0
  fpn.tx_cp_passthrough: 0
  fpn.tx_cp_kept: 0
  fpn.tx_dp_drop: 0
  fpn.tx_cp_overrun: 0
  {...}

When RX Control Plane Protection is enabled, fpn.rx_cp_passthrough is increased for each received packet when machine is not overloaded. These packets are processed normally without being analyzed.

If the machine is loaded (RX ring length exceeds the threshold) and the CPU budget is not reached, fpn.rx_cp_kept and fpn.rx_dp_drop will increase respectively for each control plane packet (kept) and for each data plane packet (drop).

If the CPU budget is exceeded, fpn.rx_cp_overrun is increased for each received packet. These packets are processed normally without being analyzed.

The same applies for TX.

Exception Control Plane Protection statistics¶

For exceptions Control Plane Protection, the statistics are available through fp-shmem-dpvi.

root@host# fp-shmem-dpvi
rx-ring[00] enq=00000025 deq=00000025 enq_err=00000000 enq_cp_kept=00000000 enq_dp_drop=00000000 deq_err=00000000 deq_copyerr=00000000
tx-ring[00] enq=00000039 deq=00000039 enq_err=00000000 enq_cp_kept=00000000 enq_dp_drop=00000000 deq_err=00000000 deq_copyerr=00000000

The Control Plane Protection on exceptions is always enabled, without CPU limit:

enq is increased for each packet enqueued in the exception ring
when the ring is more than half-full, enq_dp_drop is increased for each data plane packet dropped
when the ring is more than half-full, enq_cp_kept is increased for each control plane packet enqueued in the ring