3.2.30. fast-path

Note

requires a Turbo Router Network License.

Fast path configuration.

vrouter running config# system fast-path

enabled

Enable or disable the fast path.

vrouter running config# system fast-path
vrouter running fast-path# enabled true|false
Default value
true

port

A physical network port managed by the fast path.

vrouter running config# system fast-path
vrouter running fast-path# port PORT

PORT values

Description

<pci-port>

PCI port name.

<device-tree-port>

Device tree port name.

<device-tree-port>

Hyper-V port name.

core-mask

Dedicate cores to fast path or exception path.

vrouter running config# system fast-path core-mask

fast-path

List of cores dedicated to fast path.

vrouter running config# system fast-path core-mask
vrouter running core-mask# fast-path FAST-PATH

FAST-PATH values

Description

max

Dedicate the maximum number of cores to the fast path.

half

Dedicate half of the cores to the fast path.

min

Dedicate the minimum number of cores to the fast path.

<cores-list>

A comma-separated list of cores or core ranges. Example: ‘1,4-7,10-12’.

exception

Control plane cores allocated to exception packets processing. If unset, use the first non fast path core.

vrouter running config# system fast-path core-mask
vrouter running core-mask# exception EXCEPTION

EXCEPTION

A comma-separated list of cores or core ranges. Example: ‘1,4-7,10-12’.

linux-to-fp

Fast path cores that can receive packets from Linux. It must be included in fast path mask. If unset, all fast path cores can receive packets from Linux.

vrouter running config# system fast-path core-mask
vrouter running core-mask# linux-to-fp LINUX-TO-FP

LINUX-TO-FP

A comma-separated list of cores or core ranges. Example: ‘1,4-7,10-12’.

qos

Fast path cores dedicated for qos schedulers. These cores do not received any packets from the NIC or Linux.

vrouter running config# system fast-path core-mask
vrouter running core-mask# qos QOS

QOS

A comma-separated list of cores or core ranges. Example: ‘1,4-7,10-12’.

port

Map fast path cores with network ports, specifying which logical cores poll which ports. Example: ‘c1=0:1/c2=2/c3=0:1:2’ means the logical core 1 polls the port 0 and 1, the core 2 polls the port 2, and the core 3 polls the ports 0, 1, and 2. If unset, each port is polled by all the logical cores of the same socket.

vrouter running config# system fast-path core-mask
vrouter running core-mask# port <core-port-map>

cp-protection

Control plane protection configuration.

vrouter running config# system fast-path cp-protection

budget

Maximum CPU usage allowed for Control Plane Protection in percent.

vrouter running config# system fast-path cp-protection
vrouter running cp-protection# budget <int16>
Default value
10

crypto

Fast path crypto configuration.

vrouter running config# system fast-path crypto

driver

Crypto driver. If unset, select automatically.

vrouter running config# system fast-path crypto
vrouter running crypto# driver DRIVER

DRIVER values

Description

multibuffer

Intel multibuffer library.

quickassist

Intel quickassist.

dpdk-pmd

DPDK crypto PMD.

octeontxcpt

Marvell Octeon TX.

octeontx2cpt

Marvell Octeon TX2.

offload-core-mask

Fast path cores that can do crypto operations for other fast path cores. It must be included in fast path mask. The crypto offloading is always done on cores in the same NUMA node.

vrouter running config# system fast-path crypto
vrouter running crypto# offload-core-mask OFFLOAD-CORE-MASK

OFFLOAD-CORE-MASK values

Description

<cores-list>

A comma-separated list of cores or core ranges. Example: ‘1,4-7,10-12’.

none

Disable crypto offload.

nb-session

Maximum number of cryptographic sessions.

vrouter running config# system fast-path crypto
vrouter running crypto# nb-session <uint32>

nb-buffer

Maximum number of cryptographic buffers, representing the maximum number of in-flight operations, either being processed by the asynchronous crypto engine, or waiting in crypto device queues.

vrouter running config# system fast-path crypto
vrouter running crypto# nb-buffer <uint32>

advanced

Advanced configuration for fast path.

vrouter running config# system fast-path advanced

nb-mbuf

Number of mbufs (network packet descriptors). The value can be an integer representing the total number of mbufs, an integer prefixed with ‘+’ representing the number of mbufs to add to the automatic value. In case of NUMA, the value can be a per-socket list. If unset, nb-mbuf is determined automatically.

vrouter running config# system fast-path advanced
vrouter running advanced# nb-mbuf <nb-mbuf>

machine-memory

Set the memory that will be used by the fast path (hugepages, shm, mallocs…) so it can run on a machine with this amount of physical memory.

vrouter running config# system fast-path advanced
vrouter running advanced# machine-memory <uint32>

mainloop-sleep-delay

If set, add a sleep time after each idle mainloop turn. This will drastically decrease performance.

vrouter running config# system fast-path advanced
vrouter running advanced# mainloop-sleep-delay <uint16>

offload

Enable or disabled advanced offload features such as TSO, L4 checksum offloading, or offload information forwarding from a guest to the NIC through a virtual interface. If unset, use default product configuration.

vrouter running config# system fast-path advanced
vrouter running advanced# offload true|false

vlan-strip

Strip the VLAN header from incoming frames if supported by the hardware. By default, vlan stripping feature is disabled.

vrouter running config# system fast-path advanced
vrouter running advanced# vlan-strip true|false

intercore-ring-size

Set the size of the intercore rings, used by dataplane cores to send messages to another dataplane core. The default size depends on the product.

vrouter running config# system fast-path advanced
vrouter running advanced# intercore-ring-size <uint16>

software-txq

Set the default size of Tx software queue. This field must be a power of 2. Default is 0 (no software queue).

vrouter running config# system fast-path advanced
vrouter running advanced# software-txq <uint16>

nb-rxd

Set the default number of Rx hardware descriptors for Ethernet ports. The value must be accepted by all devices on the system. If unset, an automatic value is used.

vrouter running config# system fast-path advanced
vrouter running advanced# nb-rxd <uint16>

nb-txd

Set the default number of Tx hardware descriptors for Ethernet ports. The value must be accepted by all devices on the system. If unset, an automatic value is used.

vrouter running config# system fast-path advanced
vrouter running advanced# nb-txd <uint16>

reserve-hugepages

Enable or disable the automatic huge pages allocation by the fast path. When disabled, the user is responsible for providing enough huge pages for the fast path to start.

vrouter running config# system fast-path advanced
vrouter running advanced# reserve-hugepages true|false

ipv4-netfilter-cache

Enable or disable the IPv4 netfilter cache.

vrouter running config# system fast-path advanced
vrouter running advanced# ipv4-netfilter-cache true|false
Default value
true

ipv6-netfilter-cache

Enable or disable the IPv6 netfilter cache.

vrouter running config# system fast-path advanced
vrouter running advanced# ipv6-netfilter-cache true|false
Default value
true

ipv4-pre-ipsec-fragmentation

Configure IPv4 pre IPsec fragmentation. When enabled, this behavior helps releasing pressure on the decrypting device, as the reassembly will be done on the destination host of the inner packet instead of the decrypting device. It applies only in tunnel mode.

vrouter running config# system fast-path advanced
vrouter running advanced# ipv4-pre-ipsec-fragmentation IPV4-PRE-IPSEC-FRAGMENTATION

IPV4-PRE-IPSEC-FRAGMENTATION values

Description

always

Pre IPsec fragmentation is always performed.

check-df-bit

Pre IPsec fragmentation is performed only if the don’t fragment bit is not set on the inner packet. Applies only to IPv4 inner packets.

off

Post IPsec fragmentation is performed.

Default value
off

ipv6-pre-ipsec-fragmentation

Configure IPv6 pre IPsec fragmentation. When enabled, this behavior helps releasing pressure on the decrypting device, as the reassembly will be done on the destination host of the inner packet instead of the decrypting device. It applies only in tunnel mode.

vrouter running config# system fast-path advanced
vrouter running advanced# ipv6-pre-ipsec-fragmentation IPV6-PRE-IPSEC-FRAGMENTATION

IPV6-PRE-IPSEC-FRAGMENTATION values

Description

always

Pre IPsec fragmentation is always performed.

check-df-bit

Pre IPsec fragmentation is performed only if the don’t fragment bit is not set on the inner packet. Applies only to IPv4 inner packets.

off

Post IPsec fragmentation is performed.

Default value
off

hardware-queue-map

Hardware queue map used to change the destination queue according the hash computed on the packet from the RSS function.

vrouter running config# system fast-path advanced
vrouter running advanced# hardware-queue-map <port> <uint16> <uint16>

<port>

PCI port name.

<uint16>

Hardware queue map table index.

<uint16>

Destination Rx queue.

limits

Global runtime limits for fast path.

vrouter running config# system fast-path limits

fp-max-if

Maximum number of interfaces. It includes physical ports and virtual interfaces like gre, vlan, …

vrouter running config# system fast-path limits
vrouter running limits# fp-max-if <uint32>

fp-max-vrf

Maximum number of VRFs.

vrouter running config# system fast-path limits
vrouter running limits# fp-max-vrf <uint32>

ip4-max-addr

Maximum number of IPv4 addresses.

vrouter running config# system fast-path limits
vrouter running limits# ip4-max-addr <uint32>

ip4-max-route

Maximum number of IPv4 routes.

vrouter running config# system fast-path limits
vrouter running limits# ip4-max-route <uint32>

ip4-max-neigh

Maximum number of IPv4 neighbors.

vrouter running config# system fast-path limits
vrouter running limits# ip4-max-neigh <uint32>

ip6-max-addr

Maximum number of IPv6 addresses.

vrouter running config# system fast-path limits
vrouter running limits# ip6-max-addr <uint32>

ip6-max-route

Maximum number of IPv6 routes.

vrouter running config# system fast-path limits
vrouter running limits# ip6-max-route <uint32>

ip6-max-neigh

Maximum number of IPv6 neighbors.

vrouter running config# system fast-path limits
vrouter running limits# ip6-max-neigh <uint32>

pbr-max-rule

Maximum number of PBR rules.

vrouter running config# system fast-path limits
vrouter running limits# pbr-max-rule <uint32>

filter4-max-rule

Maximum number of IPv4 Netfilter rules.

vrouter running config# system fast-path limits
vrouter running limits# filter4-max-rule <uint32>

filter6-max-rule

Maximum number of IPv6 Netfilter rules.

vrouter running config# system fast-path limits
vrouter running limits# filter6-max-rule <uint32>

filter4-max-ct

Maximum number of IPv4 Netfilter conntracks.

vrouter running config# system fast-path limits
vrouter running limits# filter4-max-ct <uint32>

filter6-max-ct

Maximum number of IPv6 Netfilter conntracks.

vrouter running config# system fast-path limits
vrouter running limits# filter6-max-ct <uint32>

filter-max-ipset

Maximum number of ipsets per VRF.

vrouter running config# system fast-path limits
vrouter running limits# filter-max-ipset <uint32>

filter-max-ipset-entry

Maximum number of entries per ipset.

vrouter running config# system fast-path limits
vrouter running limits# filter-max-ipset-entry <uint32>

filter-bridge-max-rule

Maximum number of bridge filter rules.

vrouter running config# system fast-path limits
vrouter running limits# filter-bridge-max-rule <uint32>

vxlan-max-port

Maximum number of (VXLAN destination port, VRF) pairs.

vrouter running config# system fast-path limits
vrouter running limits# vxlan-max-port <uint32>

vxlan-max-if

Maximum number of VXLAN interfaces.

vrouter running config# system fast-path limits
vrouter running limits# vxlan-max-if <uint32>

vxlan-max-fdb

Maximum number of VXLAN forwarding database entries.

vrouter running config# system fast-path limits
vrouter running limits# vxlan-max-fdb <uint32>

reass4-max-queue

Maximum number of simultaneous reassembly procedures for IPv4.

vrouter running config# system fast-path limits
vrouter running limits# reass4-max-queue <uint32>

reass6-max-queue

Maximum number of simultaneous reassembly procedures for IPv6.

vrouter running config# system fast-path limits
vrouter running limits# reass6-max-queue <uint32>

ipsec-max-sp

Maximum number of IPv4 and IPv6 IPsec SPs.

vrouter running config# system fast-path limits
vrouter running limits# ipsec-max-sp <uint32>

ipsec-max-sa

Maximum number of IPv4 and IPv6 IPsec SAs.

vrouter running config# system fast-path limits
vrouter running limits# ipsec-max-sa <uint32>

ip-max-8-table (deprecated)

Attention

Deprecated since: 2021-04-12
Obsolete in release: 21q3
Description: This option is not relevant with new LPM algorithm.
Replacement: / system fast-path limits ip-max-lpm-memory

Maximum number of IPv4 and IPv6 /8 table entries.

vrouter running config# system fast-path limits
vrouter running limits# ip-max-8-table <uint32>

ip-max-lpm-table

Maximum number of IPv4 and IPv6 tables.

vrouter running config# system fast-path limits
vrouter running limits# ip-max-lpm-table <uint32>

ip-max-lpm-memory

Amount of memory reserved for IPv4 and IPv6 LPM tree.

vrouter running config# system fast-path limits
vrouter running limits# ip-max-lpm-memory <uint32>

filter-max-cache

Maximum number of IPv4 flows stored in filter cache.

vrouter running config# system fast-path limits
vrouter running limits# filter-max-cache <uint32>

filter6-max-cache

Maximum number of IPv6 flows stored in filter cache.

vrouter running config# system fast-path limits
vrouter running limits# filter6-max-cache <uint32>

vlan-max-if

Maximum number of VLAN interfaces.

vrouter running config# system fast-path limits
vrouter running limits# vlan-max-if <uint32>

macvlan-max-if

Maximum number of MACVLAN (VRRP) interfaces.

vrouter running config# system fast-path limits
vrouter running limits# macvlan-max-if <uint32>

gre-max-if

Maximum number of GRE interfaces.

vrouter running config# system fast-path limits
vrouter running limits# gre-max-if <uint32>

svti-max-if

Maximum number of SVTI interfaces.

vrouter running config# system fast-path limits
vrouter running limits# svti-max-if <uint32>

fp-cur-if (state only)

Current number of interfaces. It includes physical ports and virtual interfaces like gre, vlan, …

vrouter> show state system fast-path limits fp-cur-if

fp-cur-vrf (state only)

Current number of VRFs.

vrouter> show state system fast-path limits fp-cur-vrf

ip4-cur-addr (state only)

Current number of IPv4 addresses.

vrouter> show state system fast-path limits ip4-cur-addr

ip4-cur-route (state only)

Current number of IPv4 routes.

vrouter> show state system fast-path limits ip4-cur-route

ip4-cur-neigh (state only)

Current number of IPv4 neighbors.

vrouter> show state system fast-path limits ip4-cur-neigh

ip6-cur-addr (state only)

Current number of IPv6 addresses.

vrouter> show state system fast-path limits ip6-cur-addr

ip6-cur-route (state only)

Current number of IPv6 routes.

vrouter> show state system fast-path limits ip6-cur-route

ip6-cur-neigh (state only)

Current number of IPv6 neighbors.

vrouter> show state system fast-path limits ip6-cur-neigh

pbr-cur-rule (state only)

Current number of PBR rules.

vrouter> show state system fast-path limits pbr-cur-rule

filter4-cur-rule (state only)

Current number of IPv4 Netfilter rules.

vrouter> show state system fast-path limits filter4-cur-rule

filter6-cur-rule (state only)

Current number of IPv6 Netfilter rules.

vrouter> show state system fast-path limits filter6-cur-rule

filter4-cur-ct (state only)

Current number of IPv4 Netfilter conntracks.

vrouter> show state system fast-path limits filter4-cur-ct

filter6-cur-ct (state only)

Current number of IPv6 Netfilter conntracks.

vrouter> show state system fast-path limits filter6-cur-ct

filter-cur-ipset (state only)

Current number of ipsets per VRF.

vrouter> show state system fast-path limits filter-cur-ipset

vxlan-cur-port (state only)

Current number of (VXLAN destination port, VRF) pairs.

vrouter> show state system fast-path limits vxlan-cur-port

vxlan-cur-if (state only)

Current number of VXLAN interfaces.

vrouter> show state system fast-path limits vxlan-cur-if

vxlan-cur-fdb (state only)

Current number of VXLAN forwarding database entries.

vrouter> show state system fast-path limits vxlan-cur-fdb

ipsec-cur-sp (state only)

Current number of IPv4 and IPv6 IPsec SPs.

vrouter> show state system fast-path limits ipsec-cur-sp

ipsec-cur-sa (state only)

Current number of IPv4 and IPv6 IPsec SAs.

vrouter> show state system fast-path limits ipsec-cur-sa

ip-cur-8-table (deprecated) (state only)

Attention

Deprecated since: 2021-04-12
Obsolete in release: 21q3
Description: This option is not relevant with new LPM algorithm. It is replaced by ip-max-lpm-memory, which can be set to (ip-max-8-table / 512).
Replacement: state system fast-path limits ip-cur-lpm-memory

Current number of IPv4 and IPv6 /8 table entries.

vrouter> show state system fast-path limits ip-cur-8-table

ip-cur-lpm-table (state only)

Current number of IPv4 and IPv6 tables.

vrouter> show state system fast-path limits ip-cur-lpm-table

ip-cur-lpm-memory (state only)

Current amount of memory reserved for IPv4 and IPv6 LPM tree.

vrouter> show state system fast-path limits ip-cur-lpm-memory

vlan-cur-if (state only)

Current number of VLAN interfaces.

vrouter> show state system fast-path limits vlan-cur-if

macvlan-cur-if (state only)

Current number of MACVLAN (VRRP) interfaces.

vrouter> show state system fast-path limits macvlan-cur-if

gre-cur-if (state only)

Current number of GRE interfaces.

vrouter> show state system fast-path limits gre-cur-if

svti-cur-if (state only)

Current number of SVTI interfaces.

vrouter> show state system fast-path limits svti-cur-if

cg-nat

Fast path cg-nat configuration.

vrouter running config# system fast-path limits cg-nat

max-conntracks

Maximum number of tracked connections.

vrouter running config# system fast-path limits cg-nat
vrouter running cg-nat# max-conntracks <uint32>

max-nat-entries

Maximum number of NAT translations.

vrouter running config# system fast-path limits cg-nat
vrouter running cg-nat# max-nat-entries <uint32>

max-users

Maximum number of users.

vrouter running config# system fast-path limits cg-nat
vrouter running cg-nat# max-users <uint32>

max-blocks

Maximum number of blocks.

vrouter running config# system fast-path limits cg-nat
vrouter running cg-nat# max-blocks <uint32>

max-block-size

Maximum number of ports per block.

vrouter running config# system fast-path limits cg-nat
vrouter running cg-nat# max-block-size <uint32>

linux-sync

Advanced tuning for fast path / Linux synchronization.

vrouter running config# system fast-path linux-sync

fpm-socket-size

Buffer size of the socket used to communicate between the cache manager and the fast path manager.

vrouter running config# system fast-path linux-sync
vrouter running linux-sync# fpm-socket-size <uint32>
Default value
2097152

nl-socket-size

Buffer size of the cache manager netlink socket.

vrouter running config# system fast-path linux-sync
vrouter running linux-sync# nl-socket-size <uint32>
Default value
67108864

ipset-dump-delay

Delay period for polling the ipset content.

vrouter running config# system fast-path linux-sync
vrouter running linux-sync# ipset-dump-delay <uint32>
Default value
1

disable

Disable synchronization for specific modules.

vrouter running config# system fast-path linux-sync
vrouter running linux-sync# disable DISABLE

DISABLE values

Description

bpf

Disable BPF synchronization (used by traffic capture).

bridge

Disable bridge interface synchronization.

conntrack

Disable connection tracking synchronization.

firewall

Disable firewall synchronization.

gre

Disable GRE interface synchronization.

ipip

Disable IP in IP interface synchronization.

ipsec

Disable IPsec synchronization.

ipset4

Disable IPv4 ipset synchronization (used by firewall IPv4 address/network groups).

ipset6

Disable IPv6 ipset synchronization (used by firewall IPv6 address/network groups).

ipv6

Disable IPv6 synchronization.

lag

Disable LAG interface synchronization.

macvlan

Disable MACVLAN interface synchronization (used by VRRP).

mpls

Disable MPLS synchronization.

nat

Disable NAT synchronization.

svti

Disable SVTI interface synchronization.

vlan

Disable VLAN interface synchronization.

vxlan

Disable VXLAN interface synchronization.

cpu-usage (state only)

The list of busy percentage per CPU.

busy (state only)

The busy percentage.

vrouter> show state system fast-path cpu-usage <string> busy