3.2.4. Virtual Environment

This section gathers issues that happen with VMs started on top of Virtual Accelerator.

libvirt and hugepages

Symptoms

• Starting a VM (with hugepages support) with libvirt fails, yielding:

  error: internal error: Unable to find any usable hugetlbfs mount for 2048 KiB
  

  Your XML libvirt domain contains something like this:

  <memory unit='KiB'>2097152</memory>
  <memoryBacking>
    <hugepages>
      <page size="2048" unit="KiB" nodeset="0"/>
    </hugepages>
  </memoryBacking>
  

• Starting a VM with libvirt fails, yielding:

  error: operation failed: domain is no longer running
  

  Or:

  error: unsupported configuration: Shared memory mapping is supported only with hugepages
  

• The logs in /var/log/libvirt/libvirtd.log show:

  Cannot set up guest memory 'pc.ram': Cannot allocate memory
  error : qemuDomainObjExitMonitor:1603 : operation failed: domain is no longer running
  

  Or:

  error : qemuBuildNumaArgStr:6852 : unsupported configuration: Shared memory mapping is supported only with hugepages
  

Hints

• The hugepages are allocated by Virtual Accelerator at startup and libvirt cannot detect them dynamically. libvirt must be restarted to take the hugepages into account. Use service libvirt-bin restart after having started the fast path. If you manually handle your hugepages, make sure there are enough (and also restart libvirt-bin afterwards).

• Check the output from /proc/meminfo, especially the MemFree and HugePage_Free fields. See meminfo section for details.

  HugePage_Free

  indicates how many huge pages are available for your VMs.

• Check the output from fast-path.sh config --dump --full, optionally with the --long option. VM_MEMORY might need adjustment to reserve more hugepages for VMs. Refer to the Fast Path Baseline for further details on the wizard (fast-path.sh config).

• Your XML libvirt domain needs an appropriate <hugepage> tag to leverage hugepages reserved by the fast path.

Cannot read disk image

Symptoms

• Starting a VM with libvirt fails, yielding:

  error: internal error: process exited while connecting to monitor:
  qemu-system-x86_64: -drive file=/root/ubuntu-14.04-template.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none: could not open disk image
  

Hints

• Your image is not readable (or in an inaccessible folder) by user libvirt, which runs qemu. Change your image/folder’s rights to allow access to libvirt. You may also move your image directly into /var/lib/libvirt/images.

Performance degradation and serial link

Symptoms

• My VM forwards packets slower when I use intensely the serial link to it. For instance, when refreshing top regularly in virsh console.

Hints

• The serial link triggers interruptions (IRQ) when there is something happening on it. You need to make sure dataplane cores are not among those that handle IRQ. Set appropriate affinities (with cores not used by the fast path) for IRQ in /proc/irq/*/smp_affinity. E.g.: if the fast path runs on core 1 (on a machine with 4 cores), use a 0xd mask:

  for aff in /proc/irq/*/smp_affinity; do echo d > $aff; done
  

Performance degradation and offloads

Symptoms

• VM packet processing is slower when packets are forwarded in the VM than when packets are terminated there.

Hints

• Consider disabling TSO/GRO offloading in case your VMs mostly forward traffic, to increase performance. You can dynamically try that by calling fp-shmem-ports -e all -K gro off and see if that improves performance. You can also configure that permanently in your fast path configuration file using the wizard (Advanced configuration section regarding offloads) or directly editing FP_OFFLOAD in fast-path.env.

  Refer to Configuration wizard for further details.