L3VRF Overview¶

Overview¶

Isolated routing domain¶

An L3VRF, is an object that performs traffic isolation at layer 3 level. It enables to create separate routing domains within the same VRF.

An L3VRF is a lighter traffic isolation system than a VRF: a VRF performs a hard isolation and enables to configure overlapping interface names, addresses, routing tables, control plane service instances, etc. among different VRFs, while an L3VRF is basically a method to bind a set of interfaces (of a same VRF) to a given routing table, without the need for defining multiple PBR rules.

L3VRFs also offer simple ways of providing connectivity between 2 L3VRF routing domains, called route leaking.

BGP l3vrf use case example¶

L3VRF instance¶

An L3VRF routing domain is assigned a unique name and a routing table (identified by its unique 32 bit table-id).

Network interfaces can be bound to an L3VRF, and hence become a member of its routing domain.

When a packet is received on an interface not bound to an L3VRF, route lookups are done in the default routing table (table-id 254, named main). They are said to be in the default L3VRF.

When a packet is received on an interface bound to an L3VRF, route lookups are done in the routing table assigned to the L3VRF, instead of the default routing table.

Creating an L3VRF instance¶

A routing domain is defined by creating an L3VRF instance, to which network interfaces may be bound.

The L3VRF is implemented as a special logical network interface. Its name must not conflict with the name of another network interface, and the name default is reserved to reference the default L3VRF.

Network interfaces member of an L3VRF are bound as a slave to the L3VRF logical interface, just like bridged interfaces are slaves of the bridge interface. Incidentally, it results that a bridge may be bound to an L3VRF, while a bridged interface cannot. This is not a problem, since bridged interfaces do not route packets at layer 3 level, packets are bridged at the layer 2 level.

The configuration below illustrates how an L3VRF instance named l3vrf1 is assigned a routing table with table-id 10. Two interfaces eth0 and eth1 are bound to that L3VRF instance, while interface eth2 remains in the default L3VRF:

vsr running config# vrf main
vsr running vrf main# interface
vsr running interface# physical eth2
vsr running physical eth2#! port pci-b0s3
vsr running physical eth2# ipv4
vsr running ipv4# address 10.150.0.1/24
vsr running ipv4# ..
vsr running physical eth2# ..
vsr running interface# ..
vsr running vrf main# l3vrf l3vrf1
vsr running l3vrf l3vrf1#! table-id 10
vsr running l3vrf l3vrf1# interface physical eth0
vsr running physical eth0#! port pci-b0s5
vsr running physical eth0# ipv4
vsr running ipv4# address 10.100.0.1/24
vsr running ipv4# ..
vsr running physical eth0# ..
vsr running interface# physical eth1
vsr running physical eth1#! port pci-b0s4
vsr running physical eth1# ipv4
vsr running ipv4# address 10.125.0.1/24
vsr running ipv4# ..
vsr running physical eth1# ..
vsr running interface# ..
vsr running l3vrf l3vrf1# ..
vsr running vrf main#

Traffic handling¶

Once this configuration is done, incoming traffic is received as usual on the incoming original interface; if the interface is slave of an L3VRF, then a reference to this L3VRF is added in the packet meta-data.

This meta-data is checked by the PBR framework. The default PBR rules are: look up in the local table (255), then the main table (254).

Creating L3VRFs automatically adds an intermediate rule that checks if the packet meta-data references an L3VRF, and if so, looks up for a route in the L3VRF routing table. Additionally, a default unreachable route is added at the end of each L3VRF routing table, so that if a packet is bound to an L3VRF and no route is found in the L3VRF table, an ICMP unreachable message will be triggered instead of resuming route lookup in the default L3VRF (i.e. in the main table).

To display the PBR rules, and see the L3VRF rule, below command can help:

vsr> show state vrf main routing policy-based-routing
policy-based-routing
   ipv4-rule 0 action lookup local
   ipv4-rule 1000 match l3vrf action lookup l3vrf-table
   ipv4-rule 32766 action lookup main
   ipv4-rule 32767 action lookup default
   ipv6-rule 0 action lookup local
   ipv6-rule 1000 match l3vrf action lookup l3vrf-table
   ipv6-rule 32766 action lookup main

Above highlighted lines are the IPv4 and IPv6 rules added with priority 1000, for redirecting traffic to the appropriate L3VRF. To understand more the impact of L3VRF with PBR, please go to following chapter: L3VRF PBR.

Route Configuration¶

Static routing¶

Configuring routes in L3VRFs is very similar to configuring routes in tables. To get more information on how to configure routes in tables, please refer to Policy-Based Routing routes.

As an L3VRF is associated to a routing table, from CLI’s perspective, routes specified in the L3VRF static routing context are added in the L3VRF table:

vsr running config# / vrf main l3vrf l3vrf1 routing static
vsr running static# ipv4-route 10.200.0.0/24 next-hop 10.150.0.2
vsr running static# commit

Below output shows routing table associated to l3vrf1 L3VRF:

vsr> show ipv4-routes l3vrf l3vrf1
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric,
       > - selected route, * - FIB route, q - queued route, r - rejected route

VRF l3vrf1:
K   0.0.0.0/0 [255/8192] unreachable (ICMP unreachable), 00:03:27
C>* 10.100.0.0/24 is directly connected, eth0, 00:03:27
C>* 10.125.0.0/24 is directly connected, eth1, 00:03:27
C>* 10.150.0.0/24 is directly connected, eth2, 00:00:43
S>* 10.200.0.0/24 [1/0] via 10.150.0.2, eth2, 00:00:08

Route leaking¶

Route leaking is specific to L3VRF. It stands for the ability to forward an incoming packet from a given L3VRF routing domain to the outgoing interface of a separate L3VRF routing domain. This feature enables to override the routing isolation when needed, as it enables a packet to leave the current routing domain and enter another routing domain handled by a separate routing table. This feature is made possible because the outgoing interfaces of a VRF are visible from all L3VRFs present in the same VRF.

With VRF, routes via cross-VRF interfaces must be used, and requires creating veth interfaces and standard routes to offer similar service, as per cross-vrf routes.

With L3VRF, route leaking is possible by using static routing, like below chapter explains. BGP also takes advantage of route leaking facility, as per BGP L3VPN.

Static route leak using L3VRF¶

Below example shows how to create static routes in both directions between 2 L3VRFs: l3vrf1 and l3vrf2.

vsr running config# vrf main
vsr running vrf main# l3vrf l3vrf1
vsr running l3vrf l3vrf1#! table-id 10
vsr running l3vrf l3vrf1# interface physical eth0
vsr running physical eth0#! port pci-b0s4
vsr running physical eth0# ipv4
vsr running ipv4# address 10.100.0.1/24
vsr running ipv4# ..
vsr running physical eth0# ..
vsr running interface# ..
vsr running l3vrf l3vrf1# ..
vsr running vrf main# l3vrf l3vrf2
vsr running l3vrf l3vrf2#! table-id 20
vsr running l3vrf l3vrf2# interface bridge br1
vsr running bridge br1# ipv4 address 10.125.0.1/24
vsr running bridge br1# link-interface eth1
vsr running bridge br1# ..
vsr running interface# ..
vsr running l3vrf l3vrf2# ..
vsr running vrf main# interface physical eth1
vsr running physical eth1#! port pci-b0s5

Below additional command is injected to the configuration, and propose a way to create route leaks from l3vrf1 to l3vrf2 routing domains. nexthop-l3vrf parameter adds a special constraint; it stands for the targeted L3VRF instance where the route next-hop should be in order to install the route. A route to nexthop 10.125.0.2 will have to be found on the l3vrf2 routing domain.

vsr running config# vrf main
vsr running vrf main# l3vrf l3vrf1 routing static
vsr running static# ipv4-route 10.200.0.0/24
vsr running ipv4-route 10.200.0.0/24#! next-hop 10.125.0.2 nexthop-l3vrf l3vrf2
vsr running ipv4-route 10.200.0.0/24# ..
vsr running static#

Conversely, an additional static route is necessary to route leak from l3vrf2 to l3vrf1 routing domains. nexthop-l3vrf parameter is also used, and forces the nexthop interface eth0 to be present in the l3vrf1 routing domain. If not, the route will not be installed.

vsr running config# vrf main
vsr running vrf main# l3vrf l3vrf2 routing static
vsr running static# ipv4-route 10.100.0.0/24
vsr running ipv4-route 10.100.0.0/24#! next-hop eth0 nexthop-l3vrf l3vrf1
vsr running ipv4-route 10.100.0.0/24# ..
vsr running l3vrf l3vrf2# ..
vsr running static#

Below output demonstrates that the two route leaks have been injected in the FIB of both L3VRF.

 vsr> show ipv4-routes l3vrf l3vrf2
 Codes: K - kernel route, C - connected, S - static, R - RIP,
    O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
    T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
    F - PBR, f - OpenFabric,
    > - selected route, * - FIB route, q - queued route, r - rejected route

 VRF l3vrf2:
 K>* 0.0.0.0/0 [255/8192] unreachable (ICMP unreachable), 00:11:19
 S>* 10.100.0.0/24 [1/0] is directly connected, eth0(vrf l3vrf1), 00:00:39
 C>* 10.125.0.0/24 is directly connected, br1, 00:11:19

 vsr> show ipv4-routes l3vrf l3vrf1
 Codes: K - kernel route, C - connected, S - static, R - RIP,
    O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
    T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
    F - PBR, f - OpenFabric,
    > - selected route, * - FIB route, q - queued route, r - rejected route

 VRF l3vrf1:
 K>* 0.0.0.0/0 [255/8192] unreachable (ICMP unreachable), 01:09:44
 C>* 10.100.0.0/24 is directly connected, eth0, 01:42:04
 S>* 10.200.0.0/24 [1/0] via 10.125.0.2, br1(vrf l3vrf2), 00:03:53

The previous route leak configuration can be replaced by the below two route leak entries. Actually, as each L3VRF is implemented as a special logical network interface, the route may use the L3VRF interface as nexthop.

next-hop and nexthop-l3vrf have same parameter name, but are necessary to be able to install the route entries. next-hop parameter is a nexthop interface and requires that the interface exists in the same VRF as the route. nexthop-l3vrf parameter requests that the previous interface is available in the given L3VRF domain; by default, the nexthop-l3vrf value stands for the same L3VRF where the route entry is executed. If l3vrf2 nexthop-l3vrf parameter had been omitted in the route entry from l3vrf1, then the route would not have been installed, since l3vrf2 interface is not available in l3vrf1 instance.

vsr running config# vrf main
vsr running vrf main# l3vrf l3vrf1 routing static
vsr running static# ipv4-route 10.200.0.0/24
vsr running ipv4-route 10.200.0.0/24#! next-hop l3vrf2 nexthop-l3vrf l3vrf2
vsr running ipv4-route 10.200.0.0/24# .. .. .. ..
vsr running vrf main# l3vrf l3vrf2 routing static
vsr running static# ipv4-route 10.100.0.0/24
vsr running ipv4-route 10.100.0.0/24#! next-hop l3vrf1 nexthop-l3vrf l3vrf1
vsr running ipv4-route 10.100.0.0/24#

Below output shows the associated routing dump in the FIB of both L3VRF.

 vsr> show ipv4-routes l3vrf l3vrf1
 Codes: K - kernel route, C - connected, S - static, R - RIP,
    O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
    T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
    F - PBR, f - OpenFabric,
    > - selected route, * - FIB route, q - queued route, r - rejected route

 VRF l3vrf1:
 K>* 0.0.0.0/0 [255/8192] unreachable (ICMP unreachable), 00:02:51
 C>* 10.100.0.0/24 is directly connected, eth0, 00:02:52
 S>* 10.200.0.0/24 [1/0] is directly connected, l3vrf2(vrf l3vrf2), 00:00:15

 vsr> show ipv4-routes l3vrf l3vrf2
 Codes: K - kernel route, C - connected, S - static, R - RIP,
    O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
    T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
    F - PBR, f - OpenFabric,
    > - selected route, * - FIB route, q - queued route, r - rejected route

 VRF l3vrf2:
 K>* 0.0.0.0/0 [255/8192] unreachable (ICMP unreachable), 00:02:52
 S>* 10.100.0.0/24 [1/0] is directly connected, l3vrf1(vrf l3vrf1), 00:00:16
 C>* 10.125.0.0/24 is directly connected, br1, 00:02:53

It is also possible to perform route leak between a given L3VRF and default L3VRF. This can be illustrated below. It is worth noting the next-hop syntax with 10.126.0.2%eth2: to be installed, a network to 10.126.0.2 has to be present on default L3VRF, and outgoing interface of that network has to be eth2 interface. If not, the route will not be installed.

vsr running vrf main# interface
vsr running interface# physical eth2
vsr running physical eth2#! port pci-b0s7
vsr running physical eth2# ipv4
vsr running ipv4# address 10.126.0.1/24
vsr running ipv4# ..
vsr running physical eth2# ..
vsr running interface# ..
vsr running vrf main# l3vrf l3vrf1 routing static
vsr running static# ipv4-route 10.201.0.0/24
vsr running ipv4-route 10.201.0.0/24#! next-hop 10.126.0.2%eth2 nexthop-l3vrf default
vsr running ipv4-route 10.201.0.0/24#

Below output can be seen:

 vsr> show ipv4-routes l3vrf l3vrf1
 Codes: K - kernel route, C - connected, S - static, R - RIP,
    O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
    T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
    F - PBR, f - OpenFabric,
    > - selected route, * - FIB route, q - queued route, r - rejected route

 VRF l3vrf1:
 K>* 0.0.0.0/0 [255/8192] unreachable (ICMP unreachable), 00:02:51
 C>* 10.100.0.0/24 is directly connected, eth0, 00:02:52
 S>* 10.200.0.0/24 [1/0] is directly connected, l3vrf2 (vrf l3vrf2), 00:11:47
 S>* 10.201.0.0/24 [1/0] via 10.126.0.2, eth2 (vrf default), 00:00:07

Advantages¶

L3VRF has some advantages compared with VRF.

The L3VRF implementation is light as it relies on creating a specific L3VRF instance attached to a routing table within a VRF, then to bind interfaces to this L3VRF.

Also, as mentioned in the previous chapter, route leaking is supported and is much easier to apply than cross-VRF routing.

Compared to L3VRF implementation, a VRF consumes more memory and is slower to initialize. This light implementation leverages the scalability, since the number of L3VRF to be populated for a given Virtual Service Router configuration is easier to increase than for VRF.

From a more general routing perspective, BGP and IGPs also benefit from L3VRF as OSPF can be hosted in a dedicated L3VRF to redistribute BGP routes, for instance.