Scheduling

Scheduling allows to apply different types of processing to different egress queues configured on an interface. It assumes that the traffic is mapped to each queue, thanks to the concept of traffic class.

Scheduling provides two different queueing processings: Priority Queueing and PB-DWRR.

Each queue has several parameters:

  • The size of the queue defines how many packets can be stored in the queue. Longer queues mean longer delays.

  • The list of traffic classes that are submitted to the queue. Traffic classes are defined in the firewall section.

  • An input policer to rate limit incoming traffic.

  • An output shaper to rate limit outgoing traffic.

  • Specific parameters related to the selected queueing processing (Priority Queueing or PB-DWRR)

../../../_images/scheduler.svg

Scheduling is applied to egress traffic on physical interfaces.

Traffic classes

A class specifies a set of traffic flows that will be scheduled in the same queue.

Flow Matching

Classes are defined by the mark of the packet. Packet marking is done before QoS processing and must be configured at the IP packet filtering level.

QoS classification is usually based on the DSCP field of the IP header. In practice, this field is set for incoming packets by the border routers of a QoS network, allowing core routers to work with it without giving up too much processing power.

This example shows how to mark packets based on DSCP field:

vrouter running config# vrf main
vrouter running vrf main# firewall ipv4
vrouter running ipv4# mangle prerouting
vrouter running prerouting# rule 1000 dscp af41 inbound-interface eth1 action mark 0x23

This example shows how to mark packets based on DSCP field, and change the DSCP value. Only one action is allowed per rule, therefore this requires to either use a chain or repeat the same rule with different actions.

vrouter running config# vrf main
vrouter running vrf main# firewall ipv4
vrouter running ipv4# mangle prerouting
vrouter running prerouting# chain g3 rule 1 action mark 0x3
vrouter running prerouting# chain g3 rule 2 action dscp af41
vrouter running prerouting# chain g3 policy return
vrouter running prerouting# prerouting rule 3000 dscp af42 inbound-interface eth1 action chain g3

Here, the chain policy is return, which means that matching packets will be processed by the remaining rules after being marked. Use policy accept if marked packets should not be processed further by the firewall.

Note

Refer to the mark action of the rule command in the command reference.

Classification

Classes are created in the global qos context with the class command. They can then be referenced by any scheduler.

Enter the global qos context and create classes:

vrouter running config# qos
vrouter running qos#
vrouter running qos# class voip
vrouter running class voip#! mark 0x1
vrouter running class voip# ..
vrouter running qos# class mail
vrouter running class mail#! mark 0x2
vrouter running class mail# ..
vrouter running qos#

By default, all bits of the mark are used to specify classes. Therefore, up to 2**32 different classes are supported. It is possible to specify which bits are used for QoS in order to use the mark for different purposes. In this case, the number of available classes is 2**n where n is the number of bits reserved for the QoS in the mark.

To modify the mask used by the QoS enter the global qos context and edit the class-mask:

vrouter running config# qos
vrouter running qos# class-mask 0xff
vrouter running qos#

With this configuration, the first 8 bits of the mark are used to specify classes for QoS, so that 256 classes can be used.

A class can also be configured to match (or to not match) the output critical control plane traffic, i.e. the same traffic as the one protected by the Control Plane Protection feature, as described in the control plane protection.

The following class matches all critical control plane traffic:

vrouter running config# qos
vrouter running qos# class control
vrouter running qos# cp true
vrouter running qos#

The following class matches all critical control plane traffic with mark 0x30:

vrouter running config# / qos
vrouter running qos# class control30
vrouter running qos# cp true
vrouter running qos# mark 0x30
vrouter running qos#

The following class matches traffic with mark 0x30, except critical control plane traffic:

vrouter running config# / qos
vrouter running qos# class nocontrol30
vrouter running qos# cp false
vrouter running qos# mark 0x30
vrouter running qos#

Note

The result of AND operation between class-mask and mark of any specified class must be unique.

Note

A packet that does not belong to any class or whose class is not bound to any queue will be submitted to the last queue.

Scheduling algorithms

Priority Queueing

When the scheduling algorithm is Priority Queueing, N queues are defined. Each queue has a different priority. The first queue has the highest priority, the last one has the lowest. Queues are served by order of priority: the scheduler first takes packets from the highest priority queue and submits them to the network hardware. When the queue is empty, it starts processing the next queue and so on.

PB-DWRR

When the scheduling algorithm is PB-DWRR, N queues and two priority levels are defined: high and low.

Among the N queues, one has the high priority, and the N-1 others the low priority. Each low priority queue has a quantum that defines the share of the remaining bandwidth it will receive.

The high priority queue is served first. Once it is empty, other queues are served in a round robin fashion: the scheduler performs DWRR rounds between low priority queues. At each round, it checks each queue in sequence and enables it to send bytes up to its quantum. Then it serves the next queue, and so on.

When queue priorities are not set, all queues are served according to their quantum. This is the simple DWRR mode, which prevents starvation.

Scheduler templates

Scheduler templates are created in the global qos context with the scheduler command. They can then be referenced by a physical interface for egress.

Enter the global qos context and create a scheduler using Priority Queueing:

vrouter running config# qos
vrouter running qos#
vrouter running qos# scheduler sched1
vrouter running scheduler sched1#! pq
vrouter running scheduler pq# nb-queue 3
vrouter running scheduler pq# queue 1
vrouter running scheduler queue 1# class voip
vrouter running scheduler queue 1# shaper shaper1
vrouter running scheduler queue 1# ..
vrouter running scheduler pq# queue 2
vrouter running scheduler queue 2# class mail
vrouter running scheduler queue 2# .. .. ..
vrouter running qos#

Enter the global qos context and create a scheduler using PB-DWRR:

vrouter running config# qos
vrouter running qos#
vrouter running qos# scheduler sched2
vrouter running scheduler sched2#! core 2
vrouter running scheduler sched2#! pb-dwrr
vrouter running scheduler pb-dwrr# nb-queue 3
vrouter running scheduler pb-dwrr# queue 1
vrouter running scheduler queue 1# class voip
vrouter running scheduler queue 1# shaper shaper1
vrouter running scheduler queue 1# priority high
vrouter running scheduler queue 1# ..
vrouter running scheduler pb-dwrr# queue 2
vrouter running scheduler queue 2# class mail
vrouter running scheduler queue 2# quantum 3000
vrouter running scheduler queue 2# .. .. ..
vrouter running qos#

Note

A scheduler runs on a dedicated core which is chosen automatically if not set.

Review the QoS configuration:

vrouter running qos# show config
qos
    shaper shaper1
        bandwidth 1G
        burst 2K
        layer2-overhead 0
        ..
    scheduler sched1
        pq
            nb-queue 3
            queue 1
                size 256
                shaper shaper1
                class voip
                ..
            queue 2
                size 256
                class mail
                ..
            ..
        ..
    scheduler sched2
        pb-dwrr
            nb-queue 3
            queue 1
                size 256
                shaper shaper1
                class voip
                quantum 1500
                priority high
                ..
            queue 2
                size 256
                class mail
                quantum 1500
                priority low
                ..
            ..
        ..
    class-mask 0xff
    class voip
        mark 0x1
        ..
    class mail
        mark 0x2
        ..
    ..

The same settings can be applied using the following NETCONF XML configuration:

vrouter running config qos# show config xml absolute
<config xmlns="urn:6wind:vrouter">
  <qos xmlns="urn:6wind:vrouter/qos">
    <class-mask>0xff</class-mask>
    <shaper>
      <name>shaper1</name>
      <burst>2000</burst>
      <layer2-overhead>0</layer2-overhead>
      <bandwidth>1000000000</bandwidth>
    </shaper>
    <class>
      <name>voip</name>
      <mark>0x1</mark>
    </class>
    <scheduler>
      <name>sched1</name>
      <pq>
        <nb-queue>3</nb-queue>
        <queue>
          <id>1</id>
          <class>
            <name>voip</name>
          </class>
          <size>256</size>
          <shaper>shaper1</shaper>
        </queue>
        <queue>
          <id>2</id>
          <class>
            <name>mail</name>
          </class>
          <size>256</size>
        </queue>
      </pq>
    </scheduler>
    <class>
      <name>mail</name>
      <mark>0x2</mark>
    </class>
    <scheduler>
      <name>sched2</name>
      <core>2</core>
      <pb-dwrr>
        <nb-queue>3</nb-queue>
        <queue>
          <id>1</id>
          <class>
            <name>voip</name>
          </class>
          <size>256</size>
          <quantum>1500</quantum>
          <priority>high</priority>
          <shaper>shaper1</shaper>
        </queue>
        <queue>
          <id>2</id>
          <class>
            <name>mail</name>
          </class>
          <size>256</size>
          <quantum>1500</quantum>
          <priority>low</priority>
        </queue>
      </pb-dwrr>
    </scheduler>
  </qos>
</config>

Configuring a scheduler on an interface

Schedulers are configured in the qos context of physical interfaces.

Enter the qos context of the eth0 physical interface:

vrouter running config# vrf main
vrouter running vrf main# interface physical eth0
vrouter running physical eth0# qos

Configure sched1 as the scheduler for egress traffic:

vrouter running qos# egress scheduler sched1
vrouter running qos# egress rate-limit shaper shaper2
vrouter running qos# ..
vrouter running physical eth0#

Note

When a scheduler is configured on an interface, it is mandatory to also configure a rate limit shaper on the same interface.

Review eth0 configuration:

vrouter running physical eth0# show config nodefault
physical eth0
    (...)
    qos
        egress
            rate-limit
                shaper shaper2
                ..
            scheduler sched1
            ..
        ..
    ..

Commit the configuration:

vrouter running physical eth0# commit
Configuration committed.
vrouter running physical eth0# ..
vrouter running config#

Review the QoS state of the interface:

vrouter running config# show state vrf main interface physical eth0
qos
    egress
        rate-limit
            shaper
                bandwidth 10G
                burst 48K
                layer2-overhead 24
                stats
                    pass-packets 0
                    drop-packets 0
                    ..
                ..
            ..
        scheduler
            core 1
            pq
                nb-queue 3
                queue 1
                    size 256
                    shaper
                        bandwidth 1G
                        burst 2K
                        stats
                            pass-packets 0
                            drop-packets 0
                            ..
                        ..
                    class 0x00000001
                        stats
                            match-packets 0
                            ..
                        ..
                    stats
                        enqueue-packets 0
                        xmit-packets 0
                        drop-queue-full 0
                        ..
                    ..
                queue 2
                    size 256
                    class 0x00000002
                        stats
                            match-packets 0
                            ..
                        ..
                    stats
                        enqueue-packets 0
                        xmit-packets 0
                        drop-queue-full 0
                        ..
                    ..
                queue 3
                    size 256
                    stats
                        enqueue-packets 0
                        xmit-packets 0
                        drop-queue-full 0
                        ..
                    ..
                ..
            ..
        ..
    ..

The same settings can be applied using the following NETCONF XML configuration:

vrouter running config# show config xml absolute vrf main interface physical eth0
<config xmlns="urn:6wind:vrouter">
  <vrf>
    <name>main</name>
    <interface xmlns="urn:6wind:vrouter/interface">
      <physical>
        <name>eth0</name>
        (...)
        <qos>
          <egress>
            <scheduler>sched1</scheduler>
            <rate-limit>
              <shaper>shaper2</shaper>
            </rate-limit>
          </egress>
        </qos>
      </physical>
    </interface>
  </vrf>
</config>

Shaping the output while protecting critical control plane traffic

When configuring a simple shaper on the output of an interface that is constantly fed with traffic over the limit, a part of the traffic is necessarily dropped. There is a risk that critical control plane traffic be dropped.

A simple solution is to configure a Priority Queueing scheduler with 2 queues, one for the critical control plane traffic, the other for the rest of the traffic.

In this example, we configure an output shaper at 100Mbps, and the 2 queue Priority Queueing scheduler:

Configure the shaper wanshaper:

vrouter running config# / qos
vrouter running qos# shaper wanshaper
vrouter running shaper wanshaper#! bandwidth 100M
vrouter running shaper wanshaper# burst 125K
vrouter running shaper wanshaper# ..
vrouter running qos#

Configure the class for critical control plane traffic control:

vrouter running qos# class control
vrouter running class control#! cp true
vrouter running class control# ..
vrouter running qos#

Configure the scheduler wansched, and attach class control to the high priority queue:

vrouter running qos# scheduler wansched
vrouter running scheduler wansched#! pq
vrouter running pq#! nb-queue 2
vrouter running pq# queue 1
vrouter running queue 1# class control
vrouter running queue 1# ..
vrouter running pq# ..
vrouter running scheduler wansched# ..
vrouter running qos#

Attach the shaper and scheduler to the output interface:

vrouter running qos# / vrf main
vrouter running vrf main# interface physical eth0
vrouter running physical eth0# qos
vrouter running qos# egress
vrouter running egress# rate-limit
vrouter running rate-limit# shaper wanshaper
vrouter running rate-limit# ..
vrouter running egress# scheduler wansched
vrouter running egress# ..
vrouter running qos# ..
vrouter running physical eth0# ..

Review eth0 configuration:

vrouter running physical eth0# show config nodefault
physical eth0
    (...)
    qos
        egress
            rate-limit
                shaper wanshaper
                ..
            scheduler wansched
            ..
        ..
    ..

Commit the configuration:

vrouter running physical eth0# commit
Configuration committed.
vrouter running physical eth0# /
vrouter running config#

Review the QoS state of the interface:

vrouter running config# show state vrf main interface physical eth0
qos
    egress
        rate-limit
            shaper
                bandwidth 1M
                burst 1250
                layer1-overhead 0
                queue-size 256
                stats
                    pass-packets 311640
                    drop-packets 329125
                    ..
                ..
            ..
        scheduler
            core 1
            pq
                nb-queue 2
                queue 1
                    size 256
                    class cp
                        stats
                            match-packets 90
                            ..
                        ..
                    stats
                        enqueue-packets 90
                        xmit-packets 90
                        drop-queue-full 0
                        ..
                    ..
                queue 2
                    size 256
                    class default
                        stats
                            match-packets 640583
                            ..
                        ..
                    stats
                        enqueue-packets 311550
                        xmit-packets 311550
                        drop-queue-full 329125
                        ..
                    ..
                ..
            ..
        ..
    ..
..

The same settings can be applied using the following NETCONF XML configuration:

<config xmlns="urn:6wind:vrouter">
  (...)
  <vrf>
    <name>main</name>
    <interface xmlns="urn:6wind:vrouter/interface">
      <physical>
        <name>eth0</name>
        <port>pci-b0s5</port>
        <qos>
          <egress>
            <rate-limit>
              <shaper>wanshaper</shaper>
            </rate-limit>
            <scheduler>wansched</scheduler>
          </egress>
        </qos>
      </physical>
  </vrf>
  <qos xmlns="urn:6wind:vrouter/qos">
    <class>
      <name>control</name>
      <cp>true</cp>
    </class>
    <shaper>
      <name>wanshaper</name>
      <burst>125000</burst>
      <bandwidth>100000000</bandwidth>
    </shaper>
    <scheduler>
      <name>wansched</name>
      <pq>
        <nb-queue>2</nb-queue>
        <queue>
          <id>1</id>
          <class>
            <name>control</name>
          </class>
        </queue>
      </pq>
    </scheduler>
  </qos>
</config>