L3VRF configuration

There are a list of necessary elements to know when making an L3VRF configuration.

Basic elements for configuration

When making an L3VRF configuration, a unique routing table table-id must be assigned to the L3VRF.

The configuration below creates an L3VRF named l3vrf1 inside VRF main.

vsr running config# / vrf main
vsr running vrf main# l3vrf l3vrf1
vsr running l3vrf l3vrf1#! table-id 10
vsr running l3vrf l3vrf1# /
vsr running config# commit

Conversely, it is possible to suppress L3VRF configuration as below:

vsr running config# / vrf main
vsr running vrf main# del l3vrf l3vrf1
vsr running vrf main# /
vsr running config# commit

Note that the l3vrf name cannot be default, as this keyword is reserved in each VRF to reference the default L3VRF, which stands for the default routing table of the VRF (table main, table-id 254). Also, the L3VRF name must not conflict with another interface name in the same VRF and must be chosen wisely. Also, the chosen table-id value cannot be 254 as it conflicts with the main routing table.

To create a routing domain with L3VRF, just bind interfaces to a given L3VRF as below:

vsr> show config nodefault / vrf main
vrf main
    l3vrf l3vrf1
        table-id 10
        ..
        interface
            physical eth0
                port pci-b0s5
                ipv4
                    address 10.100.0.1/24
                    ..
                ipv6
                    address fd00:100::1/64
                    ..
                ..
            ..
        ..
    ..

A check can be done about routing table associated to L3VRF l3vrf1 by issuing one of the two proposed commands:

To display the routes in a specific L3VRF, use the following command:

vsr> show ipv4-routes l3vrf l3vrf1
Codes: K - kernel route, C - connected, S - static, R - RIP,
   O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
   T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
   F - PBR, f - OpenFabric,
   > - selected route, * - FIB route, q - queued route, r - rejected route

VRF l3vrf1:
K>* 0.0.0.0/0 [255/8192] unreachable (ICMP unreachable), 00:00:21
C>* 10.100.0.0/24 is directly connected, eth0, 00:00:21

Note that the keyword VRF in the output stands for the L3VRF name, not the VRF name (which is main).

As you can see, the connected route for the eth0 interface address is in the l3vrf1 routing table, instead of the main table.

You can also reference the L3VRF routing table table-id instead:

vsr> show ipv4-routes table 10
Codes: K - kernel route, C - connected, S - static, R - RIP,
   O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
   T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
   F - PBR, f - OpenFabric,
   > - selected route, * - FIB route, q - queued route, r - rejected route

VRF main table 10:
K>* 0.0.0.0/0 [255/8192] unreachable (ICMP unreachable), 00:02:32
C>* 10.100.0.0/24 is directly connected, eth0, 00:02:32

Note that when specifying the table-id, the route output does not care about L3VRFs, it considers the routing table as an ordinary routing table.

To dump the IPv6 routes, the following commands can be used:

vsr> show ipv6-routes l3vrf l3vrf1
Codes: K - kernel route, C - connected, S - static, R - RIPng,
       O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
       v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
       f - OpenFabric,
       > - selected route, * - FIB route, q - queued route, r - rejected route

VRF l3vrf1:
K>* ::/0 [255/8192] unreachable (ICMP unreachable), 00:02:50
C>* 10:100::/64 is directly connected, eth0, 00:02:50

vsr> show ipv6-routes table 10
Codes: K - kernel route, C - connected, S - static, R - RIPng,
       O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
       v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
       f - OpenFabric,
       > - selected route, * - FIB route, q - queued route, r - rejected route

VRF main table 10:
K>* ::/0 [255/8192] unreachable (ICMP unreachable), 00:03:10
C>* 10:100::/64 is directly connected, eth0, 00:03:10

As explained in the L3VRF overview.

  • routing tables are parsed according to the PBR rules,

  • creating L3VRFs automatically adds an intermediate rule of priority 1000, so that if an inbound interface is bound to an L3VRF, route lookups for packets coming from that interface are performed in a the L3VRF routing table (instead of the main table),

  • a default unreachable route is added at the end of each L3VRF routing table, in order to avoid resuming route lookup in the default L3VRF if no route was found.

If the user prefers that packets be silently dropped instead of triggering an ICMP unreachable message when no route is found in the L3VRF table, it is possible to override the default behaviour by adding blackhole routes in an L3VRF routing table, as follows:

vsr> show config / vrf main
vrf main
    l3vrf l3vrf1
        table-id 10
        routing
            static
                ipv4-route 0.0.0.0/0
                    next-hop blackhole distance 254
                    ..
                ipv6-route ::/0
                    next-hop blackhole distance 254
                    ..
                ..
            ..
        ..
    ..

To establish a forwarding scenario in a given L3VRF, a second interface is added in the L3VRF. Also, optional routing configuration can be done. Below configuration gives an example of static routing entry targeting an IP located behind a bridge interface.

../../../../_images/l3vrf-forwarding-with-bridge.svg

BGP l3vrf setup forwarding example between 2 routed interfaces: bridge and eth0

The whole configuration is depicted below:

rt1

rt1 running config# vrf main
rt1 running vrf main# interface
rt1 running interface# physical eth1
rt1 running physical eth1#! port pci-b0s5
rt1 running physical eth1# ..
rt1 running interface# physical eth2
rt1 running physical eth2#! port pci-b0s6
rt1 running physical eth2# ..
rt1 running interface# ..
rt1 running vrf main# l3vrf l3vrf1
rt1 running l3vrf l3vrf1#! table-id 10
rt1 running l3vrf l3vrf1# interface
rt1 running interface# physical eth0
rt1 running physical eth0#! port pci-b0s4
rt1 running physical eth0# ipv4 address 10.100.0.1/24
rt1 running physical eth0# ipv6 address 10:100::1/64
rt1 running physical eth0# ..
rt1 running interface# bridge bridge
rt1 running bridge bridge# ipv4
rt1 running ipv4# address 10.125.0.1/24
rt1 running ipv4# ..
rt1 running bridge bridge# ipv6
rt1 running ipv6# address 10:125::1/64
rt1 running ipv6# ..
rt1 running bridge bridge# link-interface eth1
rt1 running bridge bridge# link-interface eth2
rt1 running bridge bridge# ..
rt1 running interface# ..
rt1 running l3vrf l3vrf1# routing
rt1 running routing# static
rt1 running static# ipv4-route 10.200.0.0/24
rt1 running ipv4-route 10.200.0.0/24#! next-hop 10.125.0.2
rt1 running ipv4-route 10.200.0.0/24# ..
rt1 running static# ipv6-route 10:200::0/64
rt1 running ipv6-route 10:200::/64#! next-hop 10:125::2
rt1 running ipv6-route 10:200::/64# ..
rt1 running static# ..
rt1 running routing#

Below dump shows the routing entries an incoming packet will face when coming from either bridge or eth0 interface.

rt1> show ipv4-routes l3vrf l3vrf1

Codes: K - kernel route, C - connected, S - static, R - RIP,
   O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
   T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
   F - PBR, f - OpenFabric,
   > - selected route, * - FIB route, q - queued route, r - rejected route

VRF l3vrf1:
K>* 0.0.0.0/0 [255/8192] unreachable (ICMP unreachable), 00:16:56
C>* 10.100.0.0/24 is directly connected, eth0, 00:49:16
C>* 10.125.0.0/24 is directly connected, bridge, 00:00:04
S>* 10.200.0.0/24 [1/0] via 10.125.0.2, bridge, 00:00:04

rt1> show ipv6-routes l3vrf l3vrf1

Codes: K - kernel route, C - connected, S - static, R - RIPng,
       O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
       v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
       f - OpenFabric,
       > - selected route, * - FIB route, q - queued route, r - rejected route

VRF l3vrf1:
K>* ::/0 [255/8192] unreachable (ICMP unreachable), 00:17:30
C>* 10:100::/64 is directly connected, eth0, 00:51:30
C>* 10:125::/64 is directly connected, bridge, 00:00:24
S>* 10:200::/64 [1/0] via 10:125::2, bridge, 00:00:24

Note

L3VRF operates at layer 3 level. This implies that only interfaces with incoming traffic handled at routing level are eligible to be assigned in an L3VRF. The eth1 interface is attached to a bridge interface and incoming traffic coming from that interface is bridged, not routed. In other words, it is not possible for a given interface to be at the same time slave from a bridge interface, and be assigned an L3VRF. Similarly, the same constraint applies to interfaces belonging to a bonding interface. For instance, below configuration is not possible:

rt1 running config# vrf main
rt1 running vrf main# l3vrf l3vrf1
rt1 running l3vrf l3vrf1#! table-id 10
rt1 running l3vrf l3vrf1# interface vxlan vxl1
rt1 running vxlan vxl1#! vni 100
rt1 running vxlan vxl1# ipv4 address 10.126.0.1/24
rt1 running vxlan vxl1# ..
rt1 running interface# bridge bridge
rt1 running bridge bridge# link-interface vxl1
rt1 running bridge bridge#! validate
ERROR: / vrf main l3vrf l3vrf1 interface bridge bridge1 link-interface vxl1 slave vxl1:
  Duplicate value 'vxl1' found in /vrouter:config/vrf[name='main']/l3vrf[name='l3vrf1']/vrouter-interface:interface/vrouter-vxlan:vxlan[name='vxl1']/name.
ERROR: / vrf main l3vrf l3vrf1 interface bridge bridge1 link-interface vxl1 slave vxl1:
  An interface in a l3vrf can not be part of a bridge or a lag port.
Invalid configuration.
rt1 running bridge bridge#! ..
rt1 running interface#! del vxlan vxl1
rt1 running interface# ..
rt1 running l3vrf l3vrf1# ..
rt1 running vrf main# interface vxlan vxl1
rt1 running vxlan vxl1#! vni 100
rt1 running vxlan vxl1# ipv4 address 10.126.0.1/24
rt1 running vxlan vxl1#

This chapter introduced how to configure L3VRF and use basic operations. Further chapters will explain how to use specific services with L3VRF.

Routing services with L3VRF

The following routing services can be configured within a L3VRF domain, and are listed below:

See also

Impact on IP packet filtering and traffic control

There is a switching in the routing process before and after routing: at ingress, the interface information attached to the packet contains the L3VRF instance name, and not the original received interface. Similarly, at egress, the interface information attached contains the L3VRF instance name, and not the output one. The interface information is handled by services such as IP packet filtering. Those services are impacted.

The below figure explains the routing process differences for a given packet in an L3VRF domain or in the default L3VRF. The term chain mentioned in the drawing is used to mention the possible entry points for adding firewall service. The dotted line depicts the various chains that can be used on each L3VRF.

The first drawing depicts the processing of forwarded traffic.

../../../../_images/l3vrf_data_path_forwarding.svg

L3VRF forwarded data path traffic

Below figure explains the routing process differences for local traffic handling, in a given L3VRF and in the default L3VRF.

../../../../_images/l3vrf_data_path.svg

L3VRF local traffic

When configuring and defining an input or a forward rule with inbound-interface keyword, the behaviour slightly changes depending on the configured chain.

When a rule is created in prerouting chain, then the user can choose to select in inbound-interface keyword either the incoming original interface or the L3VRF instance name. This is also the case at egress for postrouting and output chains (for the outbound-interface keyword) where the user can use either the original interface name or the L3VRF instance name.

However, input and forward chain, if used to filter traffic across an L3VRF domain, will work only with the inbound-interface keyword set to the L3VRF instance name.

Below example illustrates that only rule 2 is able to match forwarded traffic across the given L3VRF domain. This is not the case if the user wants to filter traffic on default L3VRF: in that case, rule 1 would be sufficient assuming that eth0 sits on default L3VRF.

vsr running config# vrf main
vsr running vrf main# l3vrf l3vrf1
vsr running l3vrf l3vrf1#! table-id 10
vsr running l3vrf l3vrf1# interface
vsr running interface# physical eth0
vsr running physical eth0#! port pci-b0s4
vsr running physical eth0# ipv4
vsr running ipv4# address 10.100.0.1/24
vsr running ipv4# ..
vsr running physical eth0# ..
vsr running interface# physical eth1
vsr running physical eth1#! port pci-b0s5
vsr running physical eth1# ipv4
vsr running ipv4# address 10.125.0.1/24
vsr running ipv4# ..
vsr running physical eth1# ..
vsr running interface# ..
vsr running l3vrf l3vrf1# routing
vsr running routing# static
vsr running static# ipv4-route 10.200.0.0/24
vsr running ipv4-route 10.200.0.0/24#! next-hop 10.125.0.2
vsr running ipv4-route 10.200.0.0/24# ..
vsr running static# ..
vsr running routing# ..
vsr running l3vrf l3vrf1# ..
vsr running vrf main# firewall
vsr running firewall# ipv4
vsr running ipv4# filter
vsr running filter# forward
vsr running forward# rule 1 inbound-interface eth0 action accept
vsr running forward# rule 2 inbound-interface l3vrf1 action accept
vsr running forward# rule 5 action drop
vsr running forward#

Impact on IP PBR

Chapter PBR shows examples of how to select traffic and redirect this traffic to a specific routing table. Within a given L3VRF domain, it is possible to redirect traffic to an alternate routing table. The configuration below gives an example on how to redirect traffic with the inbound-interface keyword.

vsr running config# vrf main
vsr running vrf main# l3vrf l3vrf1
vsr running l3vrf l3vrf1#! table-id 10
vsr running l3vrf l3vrf1# interface
vsr running interface# physical eth0
vsr running physical eth0#! port pci-b0s5
vsr running physical eth0# ipv4 address 10.100.0.1/24
vsr running physical eth0# ..
vsr running interface# physical eth1
vsr running physical eth1#! port pci-b0s6
vsr running physical eth1# ipv4 address 10.125.0.1/24
vsr running physical eth1# ..
vsr running interface# ..
vsr running l3vrf l3vrf1# ..
vsr running vrf main# routing policy-based-routing
vsr running policy-based-routing# ipv4-rule 5 match inbound-interface l3vrf1 action lookup 40
vsr running policy-based-routing#

Note that it is not possible to add a PBR rule whose action is to lookup in a routing table owned by an L3VRF instance. Hence, table-id value in a l3vrf should not conflict with the lookup table used by policy based routing.