4.2.3. Networking Issues

Ports synchronization problems

Symptoms

  • No ports are displayed when calling fp-cli iface.

Hints

  • If you are dealing with physical NIC: Check that your NIC is detected by Linux, using lspci. See lspci section for details.

  • Check the output from fast-path.sh config --display and make sure your NIC is among the selected ethernet cards.

No packets are forwarded

Symptoms

  • No packets are forwarded.

  • fp-cli stats non-zero shows no (or low) IpForwDatagrams stats.

  • ethtool -S <port> shows no (or low) rx/tx packets stats.

  • ip -s link show <interface> shows no (or low) rx/tx packets stats.

  • kill -USR1 $(pidof fp-rte) (Intel and Arm only) shows no (or low) rx/tx packets stats.

Hints

  • Check whether configurations between Linux and the fast path are consistent:

    • Check IP addresses and routes configured in the kernel, using ip address show and ip route show. Check whether the interfaces and bridges are up and running using ip link show and brctl show <bridge_name>.

  • Check IP addresses and routes known to the fast path, using fp-cli route4 type all.

  • If you are using bridges, check whether your bridges have correct states, using fp-cli bridge.

  • Check that fp_dropped fast path statistics are not too high using fp-cli stats percore non-zero. A high fp_dropped stat suggests that packets are somehow not acceptable for the fast path. The ideal case is when forwarding stats are evenly spread throughout cores, that is when each core more or less forwards as many packets as the others. See Fast Path statistics section for an example of stats.

  • Check that exception stats fast path statistics are not too high. Basic exceptions indicate how many packets could not be processed by the fast path, and have thus been injected in the linux stack for slow path processing. If the value is high, it is a good indicator that IP addresses/routes/tunnels in the fast path are badly configured. See Fast Path statistics section for an example of stats.

  • Check whether it works correctly when the fast path is turned off. See Turn Fast Path off section for details.

Netfilter synchronization problems

Symptoms

  • Packets are not filtered according to your iptables rules.

Hints

  • Check whether filtering rules between Linux and fast path are consistent:

    • Check filtering rules in the kernel, using ip[6]tables -S. Refer to the ip[6]tables manpage for details on this command.

    • Check filtering rules known to the fast path, using fp-cli nf[4|6]-table <filter|mangle|nat> [all|nonzero]. Check also whether the filtering module is enabled, using fp-cli nf[4|6]. Some targets and rules are not supported in the fast path: check that you are using only documented supported options.

Connectivity problems

Symptoms

  • I can no longer connect (via the network) to my machine.

  • The VM was configured to redirect connections to the guest (using something like -netdev user,id=user.0,hostfwd=tcp::35047-:22).

Hints

  • When starting Turbo CG-NAT, NIC kernel drivers have been unloaded and thus all IP configuration lost.

Network configuration lost after restart

Symptoms

  • My network configuration no longer works after reboot or Turbo CG-NAT restart. For example:

    1. My linux bridge is empty after stopping (or restarting) the fast path:

      # brctl show
bridge name     bridge id               STP enabled     interfaces
Hints

  • The fast path may replace, change, delete and create netdevices. Any tool (brctl, iproute2, etc.) that use “old” references to netdevices must have its configuration refreshed when the fast path is stopped.

    • For linux bridge, recreate the bridge and re-add the ports if need be. e.g.:

      # brctl addbr br0
# brctl addif br0 eth1
# brctl addif br0 tap0

DKMS takes too long

Symptoms

  • Modules recompilation/removal with DKMS takes too long.

Hints

  • Edit the DKMS configuration in /etc/dkms/framework.conf, to prevent it from running some long operations:

    # mkdir -p /etc/dkms
# echo 'no_initrd="y"' >> /etc/dkms/framework.conf
# echo 'no_depmod="y"' >> /etc/dkms/framework.conf

  • Disable weak-modules:

    # chmod a-x /sbin/weak-modules