2.2.6. Install as a VM using VMware

VMware basic deployment

Turbo Router is provided in the form of an OVA file. It is supported on:

  • ESX/ESXi 5.5 and later
  • vCenter Server 5.5 and later
  • Fusion 6.x
  • Workstation 10.x
  • Player 6.x

See also

Refer to this link and that one for compatibility. Turbo Router’s hardware version is 10.

The image is configured to run with:

  • 4 cores
  • 6GB RAM
  • 1 vmxnet3 NIC

If you wish to add other NICs, make sure they have the vmxnet3 virtualDev attribute, or Turbo Router will not be able to use them.

In order to boot your Turbo Router VM, import the OVA file in your VMware product.

See also

Refer to VMware documentation for details on how to deploy VM images. For instance Deploying using vSphere 6.5, ESXi 6.5 or vCenter Server 6.5

VMware performance tuning

All ESXi version

Optimizations must be done in the hypervisor to achieve the best performance.

In the Virtual Hardware tab of the VM settings, set:

  • VM CPU Reservation field to its maximal value
  • VM CPU Limit field to Unlimited

In the VM Options tab, Advanced part of the VM settings, set:

  • sched.cpu.latencySensitivity to ‘High’: used to ensure pinning and exclusive affinity of all CPUs of a VNF

ESXi 6.5 and newer versions

Since ESXi 6.5, new tuning options are available to improve hypervisor’s performance. Before going further, all the settings described in the previous section must be applied.

In the VM Options tab, Advanced part of the VM settings, press the Configuration Parameters button to set:

  • ethernetX.ctxPerDev to 1 (where ethernetX is the NIC which will be handled by the Turbo Router): each NIC configured with ctxPerDev will receive a TX thread in the hypervisor. It can be checked in the esxtop output. The ctxPerDev recommendation must be enabled for NICs that are expected to process an high packet load.
  • sched.cpu.latencySensitivity.sysContexts to numerical value: system threads (TX and RX) are assigned exclusive physical CPU cores. The numerical value assigned to sched.cpu.latencySensitivity.sysContexts must equal the number of active threads for the VNF. For example, if one receive thread exists and three TX threads have been set using the ctxPerDev command, the value set must be 4. In this example, 4 physical CPU cores must be available and unreserved.

More details are available in VMware document regarding high performance setups.

esxtop reading

First, run esxtop command in the hypervisor’s console.

Here is the default esxtop screen (also accessible by hitting ‘c’):

 4:53:33pm up 12 days  8:06, 654 worlds, 2 VMs, 5 vCPUs; CPU load average: 0.24, 0.05, 0.02
PCPU USED(%): 0.0 0.4 0.0 0.2 2.9 0.1 0.1 1.6 0.1 0.0 118 0.0 0.0 0.0 0.1 0.0 0.0 0.2 112 0.0 0.1 1.7 0.0 0.2 AVG: 9.9
PCPU UTIL(%): 0.1 100 0.1 0.3 2.5 0.1 0.2 1.5 0.1 0.1 100 0.1 0.1 0.1 0.1 0.1 0.1 0.2 100 0.1 0.2 1.6 0.1 0.3 AVG:  12
CORE UTIL(%): 100     0.3     2.6     1.6     0.2     100     0.2     0.2     0.3     100     1.7     0.2     AVG:  25

      ID      GID NAME                                 NWLD   %USED    %RUN    %SYS   %WAIT %VMWAIT    %RDY   %IDLE  %OVRLP   %CSTP  %MLMTD  %SWPWT
  685528   685528 6WIND-TI                               11  237.16  301.35    0.00  803.45    0.00    0.01    0.00    0.02    0.00    0.00    0.00
   21609    21609 VMware vCenter Server Appliance        13    3.59    3.08    0.02 1300.00    0.00    0.02  198.30    0.01    0.00    0.00    0.00
  685520   685520 esxtop.228984                           1    2.87    2.46    0.00   97.97       -    0.00    0.00    0.00    0.00    0.00    0.00
       1        1 system                                270    0.42 2103.44    0.00 24709.04       -  307.76    0.00    0.28    0.00    0.00   40.78
   10304    10304 vpxa.67910                             24    0.17    0.15    0.00 2400.00       -    0.00    0.00    0.00    0.00    0.00    0.00
    5662     5662 hostd.67290                            24    0.12    0.09    0.04 2400.00       -    0.00    0.00    0.02    0.00    0.00    0.00
       8        8 helper                                142    0.02    0.03    0.00 14200.00       -    0.01    0.00    0.00    0.00    0.00    0.00
    4241     4241 ioFilterVPServer.67102                  2    0.02    0.02    0.00  200.00       -    0.00    0.00    0.00    0.00    0.00    0.00
  685432   685432 sshd.228973                             1    0.02    0.02    0.00  100.00       -    0.00    0.00    0.00    0.00    0.00    0.00
      10       10 ft                                      4    0.01    0.01    0.00  400.00       -    0.00    0.00    0.00    0.00    0.00    0.00

Threads (including ctxPerDev) threads can be displayed by hitting ‘e’, with the GID number of the process. You can check here the number of threads created for the VM, and their current load:

 4:55:29pm up 12 days  8:08, 654 worlds, 2 VMs, 5 vCPUs; CPU load average: 0.26, 0.15, 0.05
PCPU USED(%): 0.0 0.4 0.0 0.0 2.3 0.0 0.1 0.2 0.2 0.0 113 0.0 0.0 2.2 0.0 0.0 0.0 2.7 118 0.0 0.0 0.0 0.0 0.1 AVG:  10
PCPU UTIL(%): 0.1 100 0.1 0.1 2.2 0.1 0.1 0.3 0.2 0.1 100 0.1 0.1 2.0 0.1 0.1 0.1 2.4 100 0.1 0.1 0.1 0.1 0.1 AVG:  12
CORE UTIL(%): 100     0.3     2.3     0.4     0.4     100     2.1     0.1     2.5     100     0.3     0.3     AVG:  25

      ID      GID NAME                                 NWLD   %USED    %RUN    %SYS   %WAIT %VMWAIT    %RDY   %IDLE  %OVRLP   %CSTP  %MLMTD  %SWPWT
  228985   685528 vmx                                     1    0.01    0.00    0.00  100.00       -    0.00    0.00    0.00    0.00    0.00    0.00
  228987   685528 NetWorld-VM-228986                      1    0.00    0.00    0.00  100.00       -    0.00    0.00    0.00    0.00    0.00    0.00
  228988   685528 vmast.228986                            1    0.00    0.00    0.00  100.00       -    0.00    0.00    0.00    0.00    0.00    0.00
  228991   685528 vmx-vthread-7                           1    0.00    0.00    0.00  100.00       -    0.00    0.00    0.00    0.00    0.00    0.00
  228993   685528 vmx-mks:6WIND-TI                        1    0.01    0.01    0.00  100.00       -    0.00    0.00    0.00    0.00    0.00    0.00
  228994   685528 vmx-svga:6WIND-TI                       1    0.02    0.02    0.00  100.00       -    0.01    0.00    0.00    0.00    0.00    0.00
  228998   685528 vmx-vcpu-0:6WIND-TI                     1    0.41  100.17    0.00    0.00    0.00    0.00    0.00    0.01    0.00    0.00    0.00
  228999   685528 vmx-vcpu-1:6WIND-TI                     1  113.65  100.17    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00
  229000   685528 vmx-vcpu-2:6WIND-TI                     1  118.87  100.17    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00
  229170   685528 NetWorld-Dev-67108888-Tx                1    0.00    0.00    0.00  100.00       -    0.00    0.00    0.00    0.00    0.00    0.00
  229171   685528 NetWorld-Dev-50331672-Tx                1    0.00    0.00    0.00  100.00       -    0.00    0.00    0.00    0.00    0.00    0.00
   21609    21609 VMware vCenter Server Appliance        13    4.66    4.01    0.02 1298.06    0.00    0.08  196.53    0.01    0.00    0.00    0.00

The network screen (accessible by hitting ‘n’) is really useful to check if the hypervisor is dropping packets:

5:00:32pm up 12 days  8:13, 649 worlds, 2 VMs, 5 vCPUs; CPU load average: 0.26, 0.26, 0.14

  PORT-ID USED-BY                         TEAM-PNIC DNAME              PKTTX/s  MbTX/s   PSZTX    PKTRX/s  MbRX/s   PSZRX %DRPTX %DRPRX
 33554433 Management                            n/a vSwitch0              0.00    0.00    0.00       0.00    0.00    0.00   0.00   0.00
 33554434 vmnic0                                  - vSwitch0              6.65    0.01  229.00       6.46    0.01  145.00   0.00   0.00
 33554435 Shadow of vmnic0                      n/a vSwitch0              0.00    0.00    0.00       0.00    0.00    0.00   0.00   0.00
 33554436 vmk0                               vmnic0 vSwitch0              6.65    0.02  335.00       6.06    0.01  131.00   0.00   0.00
 33554438 69973:VMware vCenter Server Ap     vmnic0 vSwitch0              4.70    0.01  189.00       4.89    0.01  355.00   0.00   0.00
 33554463 228986:6WIND-VA-1.6.2-1            vmnic0 vSwitch0              0.00    0.00    0.00       1.96    0.00  117.00   0.00   0.00
 50331649 Management                            n/a DvsPortset-0          0.00    0.00    0.00       0.00    0.00    0.00   0.00   0.00
 50331650 LACP_MgmtPort                         n/a DvsPortset-0          0.00    0.00    0.00       0.00    0.00    0.00   0.00   0.00
 50331651 lag1                                  n/a DvsPortset-0          0.00    0.00    0.00       0.00    0.00    0.00   0.00   0.00
 50331652 vmnic7                                  - DvsPortset-0          0.00    0.00    0.00       0.00    0.00    0.00   0.00   0.00
 50331653 Shadow of vmnic7                      n/a DvsPortset-0          0.00    0.00    0.00       0.00    0.00    0.00   0.00   0.00
 50331654 vmnic6                                  - DvsPortset-0          0.20    0.00  124.00       0.00    0.00    0.00   0.00   0.00
 50331655 Shadow of vmnic6                      n/a DvsPortset-0          0.00    0.00    0.00       0.00    0.00    0.00   0.00   0.00
 50331656 vmnic5                                  - DvsPortset-0          0.20    0.00  124.00       0.00    0.00    0.00   0.00   0.00
 50331657 Shadow of vmnic5                      n/a DvsPortset-0          0.00    0.00    0.00       0.00    0.00    0.00   0.00   0.00
 50331658 vmnic4                                  - DvsPortset-0          0.20    0.00  124.00       0.00    0.00    0.00   0.00   0.00
 50331659 Shadow of vmnic4                      n/a DvsPortset-0          0.00    0.00    0.00       0.00    0.00    0.00   0.00   0.00
 50331672 228986:6WIND-TI.eth2                lag1* DvsPortset-0          0.00    0.00    0.00       0.00    0.00    0.00   0.00   0.00
 67108865 Management                            n/a DvsPortset-1          0.00    0.00    0.00       0.00    0.00    0.00   0.00   0.00
 67108888 228986:6WIND-TI.eth1                 void DvsPortset-1          0.00    0.00    0.00       0.00    0.00    0.00   0.00   0.00
 83886081 Management                            n/a DvsPortset-2          0.00    0.00    0.00       0.00    0.00    0.00   0.00   0.00
 83886087 228986:6WIND-TI.eth3                 void DvsPortset-2          0.00    0.00    0.00       0.00    0.00    0.00   0.00   0.00

The column details can be checked in the esxtop statistics reading guide.