BGP for L3VPN

L3VPN basic configuration

L3VPN is one of the VPN interconnection services that BGP offers. This chapter explains how to configure L3VPN service.

Configuring BGP L3VPN service

The below configuration shows how to configure L3VPN service on the main BGP instance, with a given neighbor:

vsr running config# vrf main
vsr running vrf main# routing bgp
vsr running bgp# as 65500
vsr running bgp# neighbor 2.2.2.2 remote-as 65500
vsr running bgp# neighbor 2.2.2.2 address-family ipv4-vpn enabled true

Configuring BGP route distribution

The below configuration shows how to configure a RD for all IP information from a given L3VRF. A separate BGP instance needs to be configured.

vsr running config# vrf main
vsr running vrf main# l3vrf customer1
vsr running l3vrf customer1#! table-id 10
vsr running l3vrf customer1# routing bgp
vsr running bgp# as 65500
vsr running bgp# address-family ipv4-unicast
vsr running ipv4-unicast# l3vpn export route-distinguisher 65500:55

The below configuration shows how to configure a RT for a given L3VRF:

vsr running config# vrf main
vsr running vrf main# l3vrf customer1
vsr running l3vrf customer1# routing bgp
vsr running bgp# address-family ipv4-unicast
vsr running ipv4-unicast# l3vpn export route-target 11:22
vsr running ipv4-unicast# l3vpn import route-target 11:22 route-target 22:44

L3VPN use case example

../../../../../_images/l3vpn-bgp-config.svg

The above drawing illustrates a setup made up of 2 symmetrical sites. Each site is separated with a PE device. In the case the site is a data center, the PE could be replaced with a DC-GW. To simplify, each site is made up of 2 distinct VPNs. L3VPN service will be gradually configured to do the following:

  • Interconnect traffic from different VPNs on the same site. This method is recommended only if the user has also some interest in using the same service for interconnecting sites. Otherwise, the user may use BGP peerings between VRFs.

  • Interconnect traffic between two VPNs located on remote sites.

Configure locally route leaking between VPNs

BGP configuration

The below configuration illustrates a setup with two VPNs. A BGP instance is set up in each of the two L3VRFs instances and on the default BGP instance. Also, each L3VRF BGP instance has its own RD. The RT export settings of each L3VRF is being matched by the RT import settings of the other L3VRF.

vsr running config# vrf main
vsr running vrf main# routing bgp
vsr running bgp#! as 65500
vsr running bgp# router-id 1.1.1.1
vsr running bgp# .. ..
vsr running vrf main# l3vrf customer1
vsr running l3vrf customer1#! table-id 10
vsr running l3vrf customer1# interface physical eth1
vsr running physical eth1#! port pci-b0s4
vsr running physical eth1# ipv4
vsr running ipv4# address 192.168.3.1/24
vsr running ipv4# ..
vsr running physical eth1# .. ..
vsr running l3vrf customer1# routing bgp
vsr running bgp# as 65500
vsr running bgp# router-id 1.1.1.1
vsr running bgp# address-family ipv4-unicast
vsr running ipv4-unicast# redistribute connected
vsr running ipv4-unicast# l3vpn export route-distinguisher 1:55
vsr running ipv4-unicast# l3vpn export route-target 11:22
vsr running ipv4-unicast# l3vpn export vpn true
vsr running ipv4-unicast# l3vpn import route-target 11:22 route-target 22:44
vsr running ipv4-unicast# l3vpn import vpn true
vsr running ipv4-unicast# .. ..
vsr running bgp# .. ..
vsr running l3vrf customer1# ..
vsr running vrf main# l3vrf customer2
vsr running l3vrf customer2#! table-id 20
vsr running l3vrf customer2# interface physical eth2
vsr running physical eth2#! port pci-b0s6
vsr running physical eth2# ipv4
vsr running ipv4# address 192.168.2.1/24
vsr running ipv4# ..
vsr running physical eth2# .. ..
vsr running l3vrf customer2# routing bgp
vsr running bgp# as 65500
vsr running bgp# router-id 1.1.1.1
vsr running bgp# address-family ipv4-unicast
vsr running ipv4-unicast# redistribute connected
vsr running ipv4-unicast# l3vpn export route-distinguisher 2:55
vsr running ipv4-unicast# l3vpn export route-target 22:44
vsr running ipv4-unicast# l3vpn export vpn true
vsr running ipv4-unicast# l3vpn import route-target 11:22 route-target 22:44
vsr running ipv4-unicast# l3vpn import vpn true
vsr running ipv4-unicast# .. ..
vsr running bgp#

Note

It is not recommended to set the same values for both RD and RT. RD stands for a given VPN identifier of the local ISP. RT stands for the exchanged information with the remote BGP peer, and is set accordingly by operators.

With the above configuration applied, L3VRF route leaking is possible. Subsequently, if BGP peering is done between a CE and the BGP instance of each L3VRF instance, then route importation and exportation occurs. The below output demonstrates that the routes from the customer2 L3VRF have been imported to the customer1 VRF. The L3VRF route leaks are visible with the @xx< indicating that the route entry is originated from the customer2 L3VRF.

vsr> show bgp l3vrf customer1 ipv4
BGP table version is 3, local router ID is 1.1.1.1, vrf id 11
Default local pref 100, local AS 65500
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath,
               i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes:  i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 192.168.2.0/24   0.0.0.0@12<              0         32768 ?
*> 192.168.3.0/24   0.0.0.0                  0         32768 ?

 Displayed  2 routes and 2 total paths

Note

The route leak entry contains a number just after @ character. This number stands for the identifier of the L3VRF where the prefix is originated from. To get the associated L3VRF, the below command gives the Id identifier associated with the L3VRF name.

vsr> show bgp l3vrfs
Type  Id     routerId          #PeersCfg  #PeersEstb  Name
             L3-VNI            RouterMAC              Interface
DFLT  0      1.1.1.1           0          0           default
             0                 00:00:00:00:00:00      unknown
 VRF  7      1.1.1.1           0          0           customer1
             0                 00:00:00:00:00:00      unknown
 VRF  12     1.1.1.1           0          0           customer2
             0                 00:00:00:00:00:00      unknown

Routing output

The following output displays the routing entries available in the customer1 L3VRF routing table. The local route leak is emphasized in the below output. Generated local route leaks are not using a label for encapsulation as the route is directly connected to the targeted L3VRF instance.

Note

Ensure that the entries are selected in the bgp RIB. For instance, the learned route may not be selected if the BGP instance from the customer2 L3VRF used the network command. By default, the network import-check command is turned on and may invalidate the entry.

vsr> show ipv4-routes l3vrf customer1
 Codes: K - kernel route, C - connected, S - static, R - RIP,
        O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
        T - Table, A - Babel, D - SHARP, F - PBR, f - OpenFabric,
        > - selected route, * - FIB route, q - queued, r - rejected, b - backup
        t - trapped, o - offload failure

 VRF customer1:
 B>* 192.168.2.0/24 [200/0] is directly connected, eth2(vrf customer2), 00:08:15
 C>* 192.168.3.0/24 is directly connected, eth1, 00:09:42

Note

When doing local route leaking, the ECMP routing is not supported. Ensure that there are no overlapping addresses across the VPNs to interconnect.

Interconnect two geographically separate VPNs

Connecting remote VPNs together is made with route leaks using MPLS labels as encapsulation. Conveying MPLS frames requires having an MPLS backbone. The MPLS backbone is generally based on an IP based network, interconnected by using an IGP protocol like OSPF or IS-IS. Then MPLS is turned on on this network. If the MPLS backbone is a non SR network, then LDP service needs to be configured. Otherwise, an IGP like IS-IS that leverages SR capability can be used.

See also

In order to create route leaks between each VPN, the ipv4-vpn address-family must be turned on on the default BGP instance. (respectively ipv6-vpn address-family for IPv6 VPN traffic.

vsr running config# vrf main
vsr running vrf main# interface physical eth0
vsr running physical eth0#! port pci-b0s5
vsr running physical eth0# ipv4 address 1.1.1.1/32
vsr running physical eth0# .. ..
vsr running vrf main# routing bgp
vsr running bgp# as 65500
vsr running bgp# neighbor 2.2.2.2 remote-as 65500
vsr running bgp# neighbor 2.2.2.2 update-source 1.1.1.1
vsr running bgp# neighbor 2.2.2.2 address-family ipv4-vpn enabled true
vsr running bgp#

Consequently, having an ipv4-vpn peering will trigger the importation of L3VPN entries. For instance, the presence of remote VPNs and associated prefixes from the 2.2.2.2 peer will trigger prefixes importation in the relevant L3VRFs. Those prefixes will be imported in the associated L3VRF, if the remote VPNs match the local VPNs. Route leaks will be injected to cross the border between the default L3VRF, and the given L3VRF.

In order for BGP to be able to export its own labels, BGP must be configured so as to rely on its own labels, either automatically, or by choosing its own. The below configuration illustrates the automatic label chosen by the customer1 L3VRF, while the customer2 L3VRF chooses to hard-set its exportation label to 300.

vsr running config# vrf main
vsr running vrf main# l3vrf customer1
vsr running l3vrf customer1# routing bgp
vsr running bgp# as 65500
vsr running bgp# address-family ipv4-unicast
vsr running ipv4-unicast# l3vpn export label auto
vsr running ipv4-unicast# .. ..
vsr running bgp# .. ..
vsr running l3vrf customer1# ..
vsr running vrf main# l3vrf customer2
vsr running l3vrf customer2# routing bgp
vsr running bgp# as 65500
vsr running bgp# address-family ipv4-unicast
vsr running ipv4-unicast# l3vpn export label 300
vsr running ipv4-unicast#

The below show bgp ipv4 vpn command dumps the remotely learned prefixes, while the show ipv4-routes l3vrf customer1 command dumps the imported prefixes that are installed on the local L3VRF.

vsr> show bgp ipv4 vpn
BGP table version is 5, local router ID is 1.1.1.1, vrf id 0
Default local pref 100, local AS 65500
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath,
               i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes:  i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 1:55
*> 192.168.3.0/24   0.0.0.0@7<         0         32768 ?
*>i192.168.4.0/24   2.2.2.2         0    100      0 ?
Route Distinguisher: 2:55
*> 192.168.2.0/24   0.0.0.0@8<         0         32768 ?
*>i192.168.5.0/24   2.2.2.2         0    100      0 ?

vsr> show ipv4-routes l3vrf customer1
 Codes: K - kernel route, C - connected, S - static, R - RIP,
        O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
        T - Table, A - Babel, D - SHARP, F - PBR, f - OpenFabric,
        > - selected route, * - FIB route, q - queued, r - rejected, b - backup
        t - trapped, o - offload failure

 VRF customer1:
 B>* 192.168.2.0/24 [200/0] is directly connected, eth2(vrf customer2), 00:05:26
 C>* 192.168.3.0/24 is directly connected, eth1, 00:19:06
 B>  192.168.4.0/24 [200/0] via 2.2.2.2(vrf main) (recursive), label 16, 00:05:26
   *                          via 2.2.2.2, eth0(vrf main) onlink, label 16, 00:05:26
 B>  192.168.5.0/24 [200/0] via 2.2.2.2(vrf main) (recursive), label 600, 00:01:14
   *                          via 2.2.2.2, eth0(vrf main) onlink, label 600, 00:01:14

The following show bgp ipv4 vpn prefix gives detailed information about a specific prefix and its associated attributes. The exported emphasized prefix has a hard set label set to 300.

vsr> show bgp ipv4 vpn prefix 192.168.2.0/24
 BGP routing table entry for 2:55:192.168.2.0/24
 not allocated
 Paths: (1 available, best #1)
   Advertised to non peer-group peers:
   2.2.2.2
   Local
     0.0.0.0 from 0.0.0.0 (1.1.1.1) vrf 8 announce-nh-self
       Origin incomplete, metric 0, weight 32768, valid, sourced, local, best (First path received)
       Extended Community: RT:22:44
       Originator: 1.1.1.1
       Remote label: 300
       Last update: Mon Aug 23 07:34:44 2021

Setup Example

../../../../../_images/bgp-l3vpn-configuration-leak.svg

The above diagram illustrates a topology made up of an MPLS-based backbone, with some LSR devices: rt3 and rt4, and LER devices. Each LER device has a BGP core instance that has ipv4-vpn address-family enabled. Next to each BGP instance, a BGP instance is created in each L3VRF. Each L3VRF stands for a private network, either A or B. The color semantic explains the relationship between the private networks on the various LER devices.

The displayed arrows indicate the interconnection between the VPNs defined on each site. The below configuration permits to do the following:

  • The green arrow connects all B networks together. B network from rt2 is interconnected with B network from rt1.

  • The orange arrow connects all A networks together. A network from rt2 is interconnected with A network from rt1 and A network from rt5.

rt1

rt1 running config# vrf main
rt1 running vrf main# interface
rt1 running interface# physical eth0_0
rt1 running physical eth0_0#! port pci-b0s6
rt1 running physical eth0_0# ipv4
rt1 running ipv4# address 6.6.6.1/24
rt1 running ipv4# ..
rt1 running physical eth0_0# ..
rt1 running interface# loopback loop1
rt1 running loopback loop1# ipv4
rt1 running ipv4# address 5.5.5.5/32
rt1 running ipv4# ..
rt1 running loopback loop1# ..
rt1 running interface# ..
rt1 running vrf main# routing
rt1 running routing# mpls
rt1 running mpls# ldp
rt1 running ldp# router-id 5.5.5.5
rt1 running ldp# dual-stack
rt1 running dual-stack# transport-preference ipv4
rt1 running dual-stack# ..
rt1 running ldp# address-family
rt1 running address-family# ipv4
rt1 running ipv4#! discovery
rt1 running discovery#! transport-address 5.5.5.5
rt1 running discovery# ..
rt1 running ipv4# interface eth0_0
rt1 running interface eth0_0# ..
rt1 running ipv4# ..
rt1 running address-family# ..
rt1 running ldp# ..
rt1 running mpls# ..
rt1 running routing# ospf
rt1 running ospf# network 6.6.6.0/24 area 0
rt1 running ospf# network 5.5.5.5/32 area 0
rt1 running ospf# redistribute connected
rt1 running ospf# ..
rt1 running routing# bgp
rt1 running bgp#! as 65500
rt1 running bgp# router-id 5.5.5.5
rt1 running bgp# neighbor 9.9.9.9
rt1 running neighbor 9.9.9.9#! remote-as 65500
rt1 running neighbor 9.9.9.9# update-source 5.5.5.5
rt1 running neighbor 9.9.9.9# address-family
rt1 running address-family# ipv4-vpn
rt1 running ipv4-vpn# ..
rt1 running address-family# ..
rt1 running neighbor 9.9.9.9# ..
rt1 running bgp# neighbor 15.15.15.15
rt1 running neighbor 15.15.15.15#! remote-as 65500
rt1 running neighbor 15.15.15.15# update-source 5.5.5.5
rt1 running neighbor 15.15.15.15# address-family
rt1 running address-family# ipv4-vpn
rt1 running ipv4-vpn# ..
rt1 running address-family# ..
rt1 running neighbor 15.15.15.15# ..
rt1 running bgp# ..
rt1 running routing# ..
rt1 running vrf main# l3vrf customer1
rt1 running l3vrf customer1#! table-id 10
rt1 running l3vrf customer1# # add l3vrf customer1 network configuration
rt1 running l3vrf customer1# routing bgp
rt1 running bgp# as 65500
rt1 running bgp# router-id 5.5.5.5
rt1 running bgp# # add neighbor information for customer1
rt1 running bgp# address-family ipv4-unicast
rt1 running ipv4-unicast# redistribute connected
rt1 running ipv4-unicast# l3vpn export route-distinguisher 65500:1
rt1 running ipv4-unicast# l3vpn export route-target 1:55
rt1 running ipv4-unicast# l3vpn export label auto
rt1 running ipv4-unicast# l3vpn export vpn true
rt1 running ipv4-unicast# l3vpn import route-target 1:55
rt1 running ipv4-unicast# l3vpn import vpn true
rt1 running ipv4-unicast# .. ..
rt1 running bgp# .. ..
rt1 running l3vrf customer1# ..
rt1 running vrf main# l3vrf customer2
rt1 running l3vrf customer2#! table-id 20
rt1 running l3vrf customer2# # add l3vrf customer2 network configuration
rt1 running l3vrf customer2# routing bgp
rt1 running bgp# as 65500
rt1 running bgp# router-id 5.5.5.5
rt1 running bgp# # add neighbor information for customer2
rt1 running bgp# address-family ipv4-unicast
rt1 running ipv4-unicast# redistribute connected
rt1 running ipv4-unicast# l3vpn export route-distinguisher 65500:2
rt1 running ipv4-unicast# l3vpn export route-target 2:55
rt1 running ipv4-unicast# l3vpn export label auto
rt1 running ipv4-unicast# l3vpn export vpn true
rt1 running ipv4-unicast# l3vpn import route-target 2:55
rt1 running ipv4-unicast# l3vpn import vpn true
rt1 running ipv4-unicast#

rt2

rt2 running config# vrf main
rt2 running vrf main# interface
rt2 running interface# physical eth0_0
rt2 running physical eth0_0#! port pci-b0s5
rt2 running physical eth0_0# ipv4
rt2 running ipv4# address 8.8.8.2/24
rt2 running ipv4# ..
rt2 running physical eth0_0# ..
rt2 running interface# loopback loop1
rt2 running loopback loop1# ipv4
rt2 running ipv4# address 9.9.9.9/32
rt2 running ipv4# ..
rt2 running loopback loop1# ..
rt2 running interface# ..
rt2 running vrf main# routing
rt2 running routing# mpls
rt2 running mpls# ldp
rt2 running ldp# router-id 9.9.9.9
rt2 running ldp# dual-stack
rt2 running dual-stack# transport-preference ipv4
rt2 running dual-stack# ..
rt2 running ldp# address-family
rt2 running address-family# ipv4
rt2 running ipv4#! discovery
rt2 running discovery#! transport-address 9.9.9.9
rt2 running discovery# ..
rt2 running ipv4# interface eth0_0
rt2 running interface eth0_0# ..
rt2 running ipv4# ..
rt2 running address-family# ..
rt2 running ldp# ..
rt2 running mpls# ..
rt2 running routing# ospf
rt2 running ospf# network 8.8.8.0/24 area 0
rt2 running ospf# network 9.9.9.9/32 area 0
rt2 running ospf# redistribute connected
rt2 running ospf# ..
rt2 running routing# bgp
rt2 running bgp#! as 65500
rt2 running bgp# router-id 9.9.9.9
rt2 running bgp# neighbor 5.5.5.5
rt2 running neighbor 5.5.5.5#! remote-as 65500
rt2 running neighbor 5.5.5.5# update-source 9.9.9.9
rt2 running neighbor 5.5.5.5# address-family
rt2 running address-family# ipv4-vpn
rt2 running ipv4-vpn# ..
rt2 running address-family# ..
rt2 running neighbor 5.5.5.5# ..
rt2 running bgp# neighbor 15.15.15.15
rt2 running neighbor 15.15.15.15#! remote-as 65500
rt2 running neighbor 15.15.15.15# update-source 9.9.9.9
rt2 running neighbor 15.15.15.15# address-family
rt2 running address-family# ipv4-vpn
rt2 running ipv4-vpn# ..
rt2 running address-family# ..
rt2 running neighbor 15.15.15.15# ..
rt2 running bgp# ..
rt2 running routing# ..
rt2 running vrf main# l3vrf customer1
rt2 running l3vrf customer1#! table-id 10
rt2 running l3vrf customer1# # add l3vrf customer1 network configuration
rt2 running l3vrf customer1# routing bgp
rt2 running bgp# as 65500
rt2 running bgp# router-id 9.9.9.9
rt2 running bgp# # add neighbor information for customer1
rt2 running bgp# address-family ipv4-unicast
rt2 running ipv4-unicast# redistribute connected
rt2 running ipv4-unicast# l3vpn export route-distinguisher 65500:3
rt2 running ipv4-unicast# l3vpn export route-target 1:55
rt2 running ipv4-unicast# l3vpn export label auto
rt2 running ipv4-unicast# l3vpn export vpn true
rt2 running ipv4-unicast# l3vpn import route-target 1:55
rt2 running ipv4-unicast# l3vpn import vpn true
rt2 running ipv4-unicast# .. ..
rt2 running bgp# .. ..
rt2 running l3vrf customer1# ..
rt2 running vrf main# l3vrf customer2
rt2 running l3vrf customer2#! table-id 20
rt2 running l3vrf customer2# # add l3vrf customer2 network configuration
rt2 running l3vrf customer2# routing bgp
rt2 running bgp# as 65500
rt2 running bgp# router-id 9.9.9.9
rt2 running bgp# # add neighbor information for customer2
rt2 running bgp# address-family ipv4-unicast
rt2 running ipv4-unicast# redistribute connected
rt2 running ipv4-unicast# l3vpn export route-distinguisher 65500:4
rt2 running ipv4-unicast# l3vpn export route-target 2:55
rt2 running ipv4-unicast# l3vpn export label auto
rt2 running ipv4-unicast# l3vpn export vpn true
rt2 running ipv4-unicast# l3vpn import route-target 2:55
rt2 running ipv4-unicast# l3vpn import vpn true
rt2 running ipv4-unicast#

rt5

rt5 running config# vrf main
rt5 running vrf main# interface
rt5 running interface# physical eth0_0
rt5 running physical eth0_0#! port pci-b0s5
rt5 running physical eth0_0# ipv4
rt5 running ipv4# address 12.12.12.0/24
rt5 running ipv4# ..
rt5 running physical eth0_0# ..
rt5 running interface# loopback loop1
rt5 running loopback loop1# ipv4
rt5 running ipv4# address 15.15.15.15/32
rt5 running ipv4# ..
rt5 running loopback loop1# ..
rt5 running interface# ..
rt5 running vrf main# routing
rt5 running routing# mpls
rt5 running mpls# ldp
rt5 running ldp# router-id 15.15.15.15
rt5 running ldp# dual-stack
rt5 running dual-stack# transport-preference ipv4
rt5 running dual-stack# ..
rt5 running ldp# address-family
rt5 running address-family# ipv4
rt5 running ipv4#! discovery
rt5 running discovery#! transport-address 15.15.15.15
rt5 running discovery# ..
rt5 running ipv4# interface eth0_0
rt5 running interface eth0_0# ..
rt5 running ipv4# ..
rt5 running address-family# ..
rt5 running ldp# ..
rt5 running mpls# ..
rt5 running routing# ospf
rt5 running ospf# network 12.12.12.0/24 area 0
rt5 running ospf# network 15.15.15.15/32 area 0
rt5 running ospf# redistribute connected
rt5 running ospf# ..
rt5 running routing# bgp
rt5 running bgp#! as 65500
rt5 running bgp# router-id 15.15.15.15
rt5 running bgp# neighbor 5.5.5.5
rt5 running neighbor 5.5.5.5#! remote-as 65500
rt5 running neighbor 5.5.5.5# update-source 15.15.15.15
rt5 running neighbor 5.5.5.5# address-family
rt5 running address-family# ipv4-vpn
rt5 running ipv4-vpn# ..
rt5 running address-family# ..
rt5 running neighbor 5.5.5.5# ..
rt5 running bgp# neighbor 9.9.9.9
rt5 running neighbor 9.9.9.9#! remote-as 65500
rt5 running neighbor 9.9.9.9# update-source 15.15.15.15
rt5 running neighbor 9.9.9.9# address-family
rt5 running address-family# ipv4-vpn
rt5 running ipv4-vpn# ..
rt5 running address-family# ..
rt5 running neighbor 15.15.15.15# ..
rt5 running bgp# ..
rt5 running routing# ..
rt5 running vrf main# l3vrf customer1
rt5 running l3vrf customer1#! table-id 10
rt5 running l3vrf customer1# # add l3vrf customer1 network configuration
rt5 running l3vrf customer1# routing bgp
rt5 running bgp# as 65500
rt5 running bgp# router-id 15.15.15.15
rt5 running bgp# # add neighbor information for customer1
rt5 running bgp# address-family ipv4-unicast
rt5 running ipv4-unicast# redistribute connected
rt5 running ipv4-unicast# l3vpn export route-distinguisher 65500:5
rt5 running ipv4-unicast# l3vpn export route-target 1:55
rt5 running ipv4-unicast# l3vpn export label 300
rt5 running ipv4-unicast# l3vpn export vpn true
rt5 running ipv4-unicast# l3vpn import route-target 1:55
rt5 running ipv4-unicast# l3vpn import vpn true
rt5 running ipv4-unicast# .. ..
rt5 running bgp# .. ..
rt5 running l3vrf customer1#

As can be seen, the network prefixes learned by each BGP instance are either learned (by adding extra CE configuration linked to each instance) or imported thanks to the L3VPN service. The below command dumps the vpnv4 entries:

rt1> show bgp ipv4 vpn
 BGP table version is 27, local router ID is 5.5.5.5, vrf id 0
 Default local pref 100, local AS 65500
 Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath,
                i internal, r RIB-failure, S Stale, R Removed
 Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
 Origin codes:  i - IGP, e - EGP, ? - incomplete

    Network          Next Hop            Metric LocPrf Weight Path
 Route Distinguisher: 65500:1
 *> 10.101.0.0/24    1.1.1.2@7<              100      0 i
 *> 10.101.1.0/24    1.1.1.2@7<              100      0 i
 *> 10.101.2.0/24    1.1.1.2@7<              100      0 i
 [..]
 Route Distinguisher: 65500:3
 *>i10.101.11.0/24   9.9.9.9              100      0 i
 *>i10.101.12.0/24   9.9.9.9              100      0 i
 *>i10.101.13.0/24   9.9.9.9              100      0 i
 [..]
 Route Distinguisher: 65500:5
 *>i10.101.21.0/24   15.15.15.15          100      0 i
 *>i10.101.22.0/24   15.15.15.15          100      0 i
 *>i10.101.23.0/24   15.15.15.15          100      0 i
 [..]
 Route Distinguisher: 65500:2
 *> 10.201.0.0/24    2.2.2.3@8<              100      0 i
 *> 10.201.1.0/24    2.2.2.3@8<              100      0 i
 *> 10.201.2.0/24    2.2.2.3@8<              100      0 i
 *> 10.201.3.0/24    2.2.2.3@8<              100      0 i
 [..]
 Route Distinguisher: 65500:4
 *>i10.201.11.0/24   9.9.9.9              100      0 i
 *>i10.201.12.0/24   9.9.9.9              100      0 i
 *>i10.201.13.0/24   9.9.9.9              100      0 i
 *>i10.201.14.0/24   9.9.9.9              100      0 i
 [..]

The three emphasized entries are locally exported entries. The 1.1.1.2 Next Hop attribute is located in the L3VRF customer1. The relationship between the L3VRF name and the L3VRF identifier displayed (the @7 value) can be done using the following command:

rt1> show bgp l3vrfs
Type  Id     routerId          #PeersCfg  #PeersEstb  Name
             L3-VNI            RouterMAC              Interface
DFLT  0      5.5.5.5           1          1           main
             0                 00:00:00:00:00:00      unknown
 VRF  7      1.1.1.1           1          1           customer1
             0                 00:00:00:00:00:00      unknown
 VRF  8      2.2.2.2           1          1           customer2
             0                 00:00:00:00:00:00      unknown

To get more details about the NLRI information of prefixes, use the following show command. The exposed 80 label value is the exported value sent to the remote BGP peers.

rt2> show bgp ipv4 vpn prefix 10.101.0.0/24
 BGP routing table entry for 1:55:10.101.0.0/24
 not allocated
 Paths: (1 available, best #1)
   Not advertised to any peer
   Local
     5.5.5.5 from 5.5.5.5 (5.5.5.5)
       Origin IGP, localpref 100, valid, internal, best (First path received)
       Extended Community: RT:1:55
       Remote label: 80
       Last update: Mon Aug 23 12:32:45 2021

Use the below command to check the available entries in the customer1 L3VRF:

rt1> show bgp l3vrf customer1 ipv4
 BGP table version is 32, local router ID is 1.1.1.1, vrf id 2
 Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath,
                i internal, r RIB-failure, S Stale, R Removed
 Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
 Origin codes:  i - IGP, e - EGP, ? - incomplete

    Network          Next Hop            Metric LocPrf Weight Path
 *>i10.101.0.0/24    1.1.1.2                       100      0 i
 *>i10.101.1.0/24    1.1.1.2                       100      0 i
 *>i10.101.2.0/24    1.1.1.2                       100      0 i
 [..]
 *> 10.101.11.0/24   9.9.9.9@0<                    100      0 i
 *> 10.101.12.0/24   9.9.9.9@0<                    100      0 i
 *> 10.101.13.0/24   9.9.9.9@0<                    100      0 i
 [..]
 *> 10.101.22.0/24   15.15.15.15@0<                100      0 i
 *> 10.101.23.0/24   15.15.15.15@0<                100      0 i
 *> 10.101.24.0/24   15.15.15.15@0<                100      0 i
 [..]

The three emphasized entries are ipv4-vpn entries which have their underlay next hop located in a remote site (9.9.9.9 IP address). @0< indicates that this is a L3VRF route leak going to a remote site. In the above example, only 1.1.1.2 ip address is a locally reachable IP address.

Routing output

From the above example, the following output can be extracted from the L3VRF routing table. As illustrated, the installed route entry performs a double MPLS encapsulation (20/80). The inner value is the negotiated BGP value. The 20 value is the one negotiated MPLS value on the backbone.

rt1> show ipv4-routes l3vrf customer1
 Codes: K - kernel route, C - connected, S - static, R - RIP,
        O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
        T - Table, A - Babel, D - SHARP, F - PBR, f - OpenFabric,
        > - selected route, * - FIB route, q - queued, r - rejected, b - backup
        t - trapped, o - offload failure

 VRF customer1:
 B>* 10.101.0.0/24 [200/0] via 1.1.1.2, eth0_0, 00:27:24
 B>* 10.101.1.0/24 [200/0] via 1.1.1.2, eth0_0, 00:27:24
 B>* 10.101.2.0/24 [200/0] via 1.1.1.2, eth0_0, 00:27:24
 [..]
 B>  10.101.11.0/24 [200/0] via 9.9.9.9(vrf vrf0) (recursive), label 80, 00:05:01
   *                          via 6.6.6.3, eth0_0(vrf main), label 20/80, 00:05:01
 B>  10.101.12.0/24 [200/0] via 9.9.9.9(vrf vrf0) (recursive), label 80, 00:05:01
   *                          via 6.6.6.3, eth0_0(vrf main), label 20/80, 00:05:01
 B>  10.101.13.0/24 [200/0] via 9.9.9.9(vrf vrf0) (recursive), label 80, 00:05:01
   *                          via 6.6.6.3, eth0_0(vrf main), label 20/80, 00:05:01
 [..]
 B>  10.101.22.0/24 [200/0] via 15.15.15.15(vrf vrf0) (recursive), label 300, 00:07:58
   *                          via 6.6.6.3, eth0_0(vrf main), label 21/300, 00:07:58
 B>  10.101.23.0/24 [200/0] via 15.15.15.15(vrf main) (recursive), label 300, 00:07:58
   *                          via 6.6.6.3, eth0_0(vrf main), label 21/300, 00:07:58
 B>  10.101.24.0/24 [200/0] via 15.15.15.15(vrf main) (recursive), label 300, 00:07:58
   *                          via 6.6.6.3, eth0_0(vrf main), label 21/300, 00:07:58
 [..]

The next command dumps the LFIB of the backbone. The relevant part deals with incoming traffic from remote VPNs. As the device is a LER, the LDP label is implicit-null, and incoming traffic comes with the BGP chosen label, here 80 or 81. That traffic is being popped and redirected to the interface representation of the L3VRF instance.

rt1> show mpls-table
 Inbound Label  Type  Nexthop   Outbound Label
 -----------------------------------------------
 16             LDP   6.6.6.3   implicit-null
 17             LDP   6.6.6.3   implicit-null
 18             LDP   6.6.6.3   16
 19             LDP   6.6.6.3   17
 20             LDP   6.6.6.3   18
 21             LDP   6.6.6.3   20
 22             LDP   6.6.6.3   21
 80             BGP   customer1 -
 81             BGP   customer2 -

Interconnection policies options

The previous networking configuration is reused to interconnect all A networks with all B networks. The black arrow illustrates it on the diagram.

The below configurations show the BGP configuration changes. Usually, that kind of configuration can be used, when some resources are shared with the other L3VRFs: A video server located on that shared resource, or a management L3VRF that is only able to access the other L3VRFs.

rt1

rt1 running config# vrf main
rt1 running vrf main# routing bgp
rt1 running bgp# l3vrf customer1
rt1 running l3vrf customer1# address-family ipv4-unicast
rt1 running ipv4-unicast# del l3vpn import route-target 1:55
rt1 running ipv4-unicast#! l3vpn import route-target 1:55 route-target 2:55
rt1 running ipv4-unicast# maximum-path ebgp 4
rt1 running ipv4-unicast# maximum-path ibgp 4
rt1 running ipv4-unicast# .. ..
rt1 running l3vrf customer1# ..
rt1 running bgp# l3vrf customer2
rt1 running l3vrf customer2# address-family ipv4-unicast
rt1 running ipv4-unicast# del l3vpn import route-target 2:55
rt1 running ipv4-unicast#! l3vpn import route-target 1:55 route-target 2:55
rt1 running ipv4-unicast# .. ..
rt1 running l3vrf customer2# ..
rt1 running bgp#

rt2

rt2 running config# vrf main
rt2 running vrf main# routing bgp
rt2 running bgp# l3vrf customer1
rt2 running l3vrf customer1# address-family ipv4-unicast
rt2 running ipv4-unicast# del l3vpn import route-target 1:55
rt2 running ipv4-unicast#! l3vpn import route-target 1:55 route-target 2:55
rt2 running ipv4-unicast# .. ..
rt2 running l3vrf customer1# ..
rt2 running bgp# l3vrf customer2
rt2 running l3vrf customer2# address-family ipv4-unicast
rt2 running ipv4-unicast# del l3vpn import route-target 2:55
rt2 running ipv4-unicast#! l3vpn import route-target 1:55 route-target 2:55
rt2 running ipv4-unicast# .. ..
rt2 running l3vrf customer2# ..
rt2 running bgp#

rt5

rt5 running config# vrf main
rt5 running vrf main# routing bgp
rt5 running bgp# l3vrf customer1
rt5 running l3vrf customer1# address-family ipv4-unicast
rt5 running ipv4-unicast# del l3vpn import route-target 1:55
rt5 running ipv4-unicast#! l3vpn import route-target 1:55 route-target 2:55
rt5 running ipv4-unicast# .. ..
rt5 running l3vrf customer1# ..
rt5 running bgp#

The following show extract on rt1 dumps the routing entries of A and B located on rt2. Two different MPLS labels chosen by BGP of rt2, are received by rt1, for each network. Those labels are visible by executing the show bgp ipv4 vpn prefix command.

rt1> show bgp ipv4 vpn
BGP table version is 20, local router ID is 9.9.9.9, vrf id 0
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath,
               i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes:  i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 2:55
[..]
*>i10.101.11.0/24   9.9.9.9              100      0 i
*>i10.101.12.0/24   9.9.9.9              100      0 i
*>i10.101.13.0/24   9.9.9.9              100      0 i
[..]
*>i10.201.11.0/24   9.9.9.9              100      0 i
*>i10.201.12.0/24   9.9.9.9              100      0 i
*>i10.201.13.0/24   9.9.9.9              100      0 i
[..]

rt1> show bgp ipv4 vpn prefix 10.101.11.0/24
BGP routing table entry for 1:55:10.201.11.0/24
not allocated
Paths: (1 available, best #1)
  Not advertised to any peer
  Local
    9.9.9.9 from 9.9.9.9 (9.9.9.9)
      Origin IGP, localpref 100, valid, internal, best (First path received)
      Extended Community: RT:1:55
      Remote label: 80
      Last update: Mon Aug 23 14:08:23 2021

rt1> show bgp ipv4 vpn prefix 10.201.11.0/24
BGP routing table entry for 2:55:10.101.11.0/24
not allocated
Paths: (1 available, best #1)
  Not advertised to any peer
  Local
    9.9.9.9 from 9.9.9.9 (9.9.9.9)
      Origin IGP, localpref 100, valid, internal, best (First path received)
      Extended Community: RT:2:55
      Remote label: 80
      Last update: Mon Aug 23 14:09:10 2021

When applied to the underlying system, the encapsulation for packets leaving the customer1 L3VRF contains different labels, depending if the traffic is headed to remote customer1 or remote customer2.

rt1> show ipv4-routes l3vrf customer1
 Codes: K - kernel route, C - connected, S - static, R - RIP,
        O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
        T - Table, A - Babel, D - SHARP, F - PBR, f - OpenFabric,
        > - selected route, * - FIB route, q - queued, r - rejected, b - backup
        t - trapped, o - offload failure

 VRF customer1:
 B>  10.101.11.0/24 [200/0] via 9.9.9.9(vrf main) (recursive), label 80, 00:20:10
   *                          via 6.6.6.3, r1-eth2(vrf main), label 21/80, 00:20:10
 B>  10.101.12.0/24 [200/0] via 9.9.9.9(vrf main) (recursive), label 80, 00:20:10
   *                          via 6.6.6.3, r1-eth2(vrf main), label 21/80, 00:20:10
 B>  10.101.13.0/24 [200/0] via 9.9.9.9(vrf main) (recursive), label 80, 00:20:10
   *                          via 6.6.6.3, r1-eth2(vrf main), label 21/80, 00:20:10
 [..]
 B>  10.201.11.0/24 [200/0] via 9.9.9.9(vrf main) (recursive), label 81, 00:20:57
   *                          via 6.6.6.3, r1-eth2(vrf main), label 20/81, 00:20:57
 B>  10.201.12.0/24 [200/0] via 9.9.9.9(vrf main) (recursive), label 81, 00:20:57
   *                          via 6.6.6.3, r1-eth2(vrf main), label 20/81, 00:20:57
 B>  10.201.13.0/24 [200/0] via 9.9.9.9(vrf main) (recursive), label 81, 00:20:57
   *                          via 6.6.6.3, r1-eth2(vrf main), label 20/81, 00:20:57

rt1> show mpls table
 Inbound                            Outbound
   Label     Type          Nexthop     Label
--------  -------  ---------------  --------
      80      BGP         r1-cust1
      81      BGP         r1-cust2
      82      BGP          6.6.6.3        18
      83      BGP          6.6.6.3        18
      84      BGP          6.6.6.3        18
      85      BGP          6.6.6.3        18

An additional specificity of the setup is the possibility to import ECMP entries coming from 2 separate locations; here, some 32-bit host routes are retrieved. Some of the entries stand for a server with the same IP, but geographically at 2 different places, namely rt2 and rt5. This kind of scenario can be used for servers that require availability, or where the load is split in two, to avoid starvation of the resources of one of the machines.

rt1> show bgp ipv4 vpn
BGP table version is 20, local router ID is 9.9.9.9, vrf id 0
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath,
               i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes:  i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*=i10.101.18.10/32   15.15.15.15              100      0 i
*>i                 9.9.9.9              100      0 i
*=i10.101.19.10/32   15.15.15.15              100      0 i
*>i                 9.9.9.9              100      0 i
*=i10.101.20.10/32   15.15.15.15              100      0 i
*>i                 9.9.9.9              100      0 i

rt1> show ipv4-routes l3vrf customer1
 Codes: K - kernel route, C - connected, S - static, R - RIP,
        O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
        T - Table, A - Babel, D - SHARP, F - PBR, f - OpenFabric,
        > - selected route, * - FIB route, q - queued, r - rejected, b - backup
        t - trapped, o - offload failure

 VRF customer1:
 B>  10.101.18.10/32 [200/0] via 15.15.15.15(vrf main) (recursive), label 300, 00:20:10
   *                          via 6.6.6.3, r1-eth2(vrf main), label 21/300, 00:20:10
                            via 9.9.9.9(vrf main) (recursive), label 80, 00:20:10
   *                          via 6.6.6.3, r1-eth2(vrf main), label 20/80, 00:20:10
 B>  10.101.19.10/32 [200/0] via 15.15.15.15(vrf main) (recursive), label 300, 00:20:10
   *                          via 6.6.6.3, r1-eth2(vrf main), label 21/300, 00:20:10
                            via 9.9.9.9(vrf main) (recursive), label 80, 00:20:10
   *                          via 6.6.6.3, r1-eth2(vrf main), label 20/80, 00:20:10
 B>  10.101.20.10/32 [200/0] via 15.15.15.15(vrf main) (recursive), label 300, 00:20:10
   *                          via 6.6.6.3, r1-eth2(vrf main), label 21/300, 00:20:10
                            via 9.9.9.9(vrf main) (recursive), label 80, 00:20:10
   *                          via 6.6.6.3, r1-eth2(vrf main), label 20/80, 00:20:10

Interconnect traffic between direct connections

Having an MPLS backbone requires using the LDP service. It may require increasing the configuration on the local devices, but also on the whole interconnection network devices. One solution to simplify the configuration and to not rely on any external dependencies is to reduce to 1 hop the distance between the two devices that should interconnect. The following use cases can apply:

  • device acting as ASBR, and directly connecting to remote ASBR. In that case, eBGP is used.

  • a GRE tunnel is used to interconnect two devices. As GRE tunnel is a point-to-point technology, then the traffic flowing across that interface is reduced to 1 hop only, independently of the framework to be crossed.

The below examples illustrate how the MPLS configuration can be simplified when having to interconnect L3VPN networks over a GRE interface.

Using LDP as MPLS backbone

The below configuration illustrates the configuration of the two interconnected devices using LDP. The configuration does not include the VPN configuration part.

rt1

rt1 running config# vrf main
rt1 running vrf main# interface
rt1 running interface# loopback loop1
rt1 running loopback loop1# ipv4 address 5.5.5.5/32
rt1 running loopback loop1# ..
rt1 running interface# physical eth1
rt1 running physical eth1#! port pci-b0s4
rt1 running physical eth1# ipv4 address 10.1.2.5/24
rt1 running physical eth1# ..
rt1 running interface# gre gre1
rt1 running gre gre1#! link-interface eth1
rt1 running gre gre1#! ipv4 address 40.0.0.5/24
rt1 running gre gre1#! local 10.1.2.5
rt1 running gre gre1# remote 10.3.2.9
rt1 running gre gre1# ..
rt1 running interface# ..
rt1 running vrf main# routing ospf
rt1 running ospf# router-id 10.1.2.5
rt1 running ospf# network 10.1.2.0/24 area 0.0.0.0
rt1 running ospf# ..
rt1 running routing# ..
rt1 running vrf main# routing static
rt1 running static# ipv4-route 9.9.9.9/32 next-hop 40.0.0.9
rt1 running static# ..
rt1 running routing# ..
rt1 running vrf main# routing mpls ldp
rt1 running ldp# router-id 5.5.5.5
rt1 running ldp# address-family ipv4
rt1 running ipv4#! discovery transport-address 5.5.5.5
rt1 running ipv4# interface gre1
rt1 running interface gre1# ..
rt1 running ipv4# ..
rt1 running address-family# .. ..
rt1 running mpls# ..
rt1 running routing# bgp
rt1 running bgp# router-id 5.5.5.5
rt1 running bgp# as 65500
rt1 running bgp# neighbor 9.9.9.9 remote-as 65500
rt1 running bgp# neighbor 9.9.9.9 update-source 5.5.5.5
rt1 running bgp# neighbor 9.9.9.9 address-family ipv4-unicast enabled false
rt1 running bgp# neighbor 9.9.9.9 address-family ipv4-vpn enabled true
rt1 running bgp# .. ..
rt1 running vrf main# l3vrf customer1
rt1 running l3vrf customer1#! table-id 10
rt1 running l3vrf customer1# routing bgp
rt1 running bgp# router-id 5.5.5.5
rt1 running bgp# as 65500
rt1 running bgp# # append bgp configuration for customer1 : l3vpn, ...
rt1 running bgp#

rt2

rt2 running config# vrf main
rt2 running vrf main# interface
rt2 running interface# loopback loop1
rt2 running loopback loop1# ipv4 address 9.9.9.9/32
rt2 running loopback loop1# ..
rt2 running interface# physical eth1
rt2 running physical eth1#! port pci-b0s4
rt2 running physical eth1# ipv4 address 10.3.2.9/24
rt2 running physical eth1# ..
rt2 running interface# gre gre1
rt2 running gre gre1#! link-interface eth1
rt2 running gre gre1#! ipv4 address 40.0.0.9/24
rt2 running gre gre1#! local 10.3.2.9
rt2 running gre gre1# remote 10.1.2.5
rt2 running gre gre1# ..
rt2 running interface# ..
rt2 running vrf main# routing ospf
rt2 running ospf# router-id 10.3.2.9
rt2 running ospf# network 10.3.2.0/24 area 0.0.0.0
rt2 running ospf# ..
rt2 running routing# ..
rt2 running vrf main# routing static
rt2 running static# ipv4-route 5.5.5.5/32 next-hop 40.0.0.5
rt2 running static# ..
rt2 running routing# ..
rt2 running vrf main# routing mpls ldp
rt2 running ldp# router-id 9.9.9.9
rt2 running ldp# address-family ipv4
rt2 running ipv4#! discovery transport-address 9.9.9.9
rt2 running ipv4# interface gre1
rt2 running interface gre1# ..
rt2 running ipv4# ..
rt2 running address-family# ..
rt2 running ldp# .. ..
rt2 running routing# bgp
rt2 running bgp#! router-id 9.9.9.9
rt2 running bgp#! as 65500
rt2 running bgp# neighbor 5.5.5.5 remote-as 65500
rt2 running bgp# neighbor 5.5.5.5 update-source 9.9.9.9
rt2 running bgp# neighbor 5.5.5.5 address-family ipv4-unicast enabled false
rt2 running bgp# neighbor 5.5.5.5 address-family ipv4-vpn enabled true
rt2 running bgp# .. ..
rt2 running vrf main# l3vrf customer2
rt2 running l3vrf customer2#! table-id 10
rt2 running l3vrf customer2# routing bgp
rt2 running bgp# router-id 9.9.9.9
rt2 running bgp# as 65500
rt2 running bgp# # append bgp configuration for customer1 : l3vpn, ...
rt2 running bgp#

As shown below, an implicit-null label has been bound to the route to the next hop of the remote BGP speaker. You can note that the 40.0.0.9 IP address is the gateway used to reach the remote 9.9.9.9 IP address.

 rt1> show ipv4-routing
  Codes: K - kernel route, C - connected, S - static, R - RIP,
         O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
         T - Table, A - Babel, D - SHARP, F - PBR, f - OpenFabric,
         > - selected route, * - FIB route, q - queued, r - rejected, b - backup
         t - trapped, o - offload failure

  VRF default:
  [..]
  C>* 5.5.5.5/32 is directly connected, loop1, 00:02:21
  S>* 9.9.9.9/32 [1/0] via 40.0.0.9, gre1, label implicit-null, weight 1, 00:00:19
  C>* 40.0.0.0/24 is directly connected, gre1, 00:02:21
  [..]

By adding the VPN configuration which is not present on the above configuration, one will have two VPN labels negotiated between BGP independently of LDP.

rt1> show bgp ipv4 vpn
BGP table version is 2, local router ID is 5.5.5.5, vrf id 0
Default local pref 100, local AS 65500
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath,
               i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes:  i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 65500:5
*> 172.16.1.0/24    0.0.0.0@6<         0         32768 ?
[..]
Route Distinguisher: 65500:9
*>i172.16.2.0/24    9.9.9.9         0    100      0 ?
[..]

The below show ipv4-routes command gives the information on the encapsulation used by a packet going from the local L3VRF customer1 to the remote L3VRF customer2. Outgoing packets leaving the rt1 device are encapsulated with the 144/implicit-null labels. The additional implicit-null label is chosen since it stands for the label to use when wanting to reach the nexthop of the VPN route. Actually, 9.9.9.9 is reachable via 40.0.0.9 by using the implicit-null label.

rt1> show ipv4-routes l3vrf customer1
 Codes: K - kernel route, C - connected, S - static, R - RIP,
        O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
        T - Table, A - Babel, D - SHARP, F - PBR, f - OpenFabric,
        > - selected route, * - FIB route, q - queued, r - rejected, b - backup
        t - trapped, o - offload failure

 VRF customer1:
 C>* 172.16.1.0/24 is directly connected, r1-eth0, 00:11:15
 B>  172.16.2.0/24 [20/0] via 9.9.9.9 (vrf default) (recursive), label 80, weight 1, 00:15:29
   *                        via 40.0.0.9, gre1 (vrf default), label implicit-null/80, weight 1, 00:15:29

Convey VPN labels coming from eBGP over a connected interface

When an inter-AS BGP peering between connected peers is formed, forging an MPLS backbone to transport VPN labels is not always feasible. When peering with a connected eBGP peer, the incoming BGP L3VPN updates have the next hop resolved over the connected routes of the interface. To authorize those MPLS routes to be installed on the FIB, the following command under the routing interface context is enabled.

vsr running config# vrf main
vsr running vrf main# interface physical eth2
vsr running physical eth2#! port pci-b0s5
vsr running physical eth2# ipv4-address 10.125.0.1
vsr running physical eth2# / vrf main routing interface eth2
vsr running interface eth2# bgp mpls accept-connected-ebgp-routes true
vsr running interface eth2# ..
vsr running routing# bgp
vsr running bgp#! as 65500
vsr running bgp# router-id 10.125.0.1
vsr running bgp# neighbor 10.125.0.2 remote-as 65502
vsr running bgp# neighbor 10.125.0.2 address-family ipv4-vpn enabled true
vsr running bgp#

The BGP routes received from the eBGP neighbor are installed, without requiring any label distribution protocol.

See also

  • RFC 4364 section 10.b using the connected approach:

Note

The value of this parameter is ignored when using any label distribution protocol other than BGP over that interface.

Using Static Label configuration

The below configuration proposes to replace the LDP configuration with a static route that contains a labeled next hop. The MPLS label value is 3 and stands for the implicit-null value for ipv4 traffic.

rt1

rt1 running config# vrf main
rt1 running vrf main# interface loopback loop1
rt1 running loopback loop1# ipv4 address 5.5.5.5/32
rt1 running loopback loop1# ..
rt1 running interface# physical eth1
rt1 running physical eth1#! port pci-b0s4
rt1 running physical eth1# ipv4 address 10.1.2.5/24
rt1 running physical eth1# ..
rt1 running interface# gre gre1
rt1 running gre gre1#! link-interface eth1
rt1 running gre gre1#! ipv4 address 40.0.0.5/24
rt1 running gre gre1#! local 10.1.2.5
rt1 running gre gre1# remote 10.3.2.9
rt1 running gre gre1# ..
rt1 running interface# ..
rt1 running vrf main# routing ospf
rt1 running ospf# router-id 10.1.2.5
rt1 running ospf# network 10.1.2.0/24 area 0.0.0.0
rt1 running ospf# ..
rt1 running routing# ..
rt1 running vrf main# routing static
rt1 running static# ipv4-route 9.9.9.9/32 next-hop 40.0.0.9%gre1 label 3
rt1 running static#
rt1 running static# ..
rt1 running routing# bgp
rt1 running bgp#! router-id 5.5.5.5
rt1 running bgp#! as 65500
rt1 running bgp# neighbor 9.9.9.9 remote-as 65500
rt1 running bgp# neighbor 9.9.9.9 update-source 5.5.5.5
rt1 running bgp# neighbor 9.9.9.9 address-family ipv4-unicast enabled false
rt1 running bgp# neighbor 9.9.9.9 address-family ipv4-vpn enabled true
rt1 running bgp# .. ..
rt1 running vrf main# l3vrf customer1
rt1 running l3vrf customer1#! table-id 10
rt1 running l3vrf customer1# routing bgp
rt1 running bgp# router-id 5.5.5.5
rt1 running bgp# as 65500
rt1 running bgp# # append bgp configuration for customer1 : l3vpn, ...
rt1 running bgp#

rt2

rt2 running config# vrf main
rt2 running vrf main# interface loopback loop1
rt2 running loopback loop1# ipv4 address 9.9.9.9/32
rt2 running loopback loop1# ..
rt2 running interface# physical eth1
rt2 running physical eth1#! port pci-b0s4
rt2 running physical eth1# ipv4 address 10.3.2.9/24
rt2 running physical eth1# ..
rt2 running interface# gre gre1
rt2 running gre gre1#! link-interface eth1
rt2 running gre gre1#! ipv4 address 40.0.0.9/24
rt2 running gre gre1#! local 10.3.2.9
rt2 running gre gre1# remote 10.1.2.5
rt2 running gre gre1# ..
rt2 running interface# ..
rt2 running vrf main# routing ospf
rt2 running ospf# router-id 10.3.2.9
rt2 running ospf# network 10.3.2.0/24 area 0.0.0.0
rt2 running ospf# ..
rt2 running routing# ..
rt2 running vrf main# routing static
rt2 running static# ipv4-route 5.5.5.5/32 next-hop 40.0.0.5%gre1 label 3
rt2 running static# ..
rt2 running routing# bgp
rt2 running bgp#! router-id 5.5.5.5
rt2 running bgp#! as 65500
rt2 running bgp# neighbor 9.9.9.9 remote-as 65500
rt2 running bgp# neighbor 9.9.9.9 update-source 5.5.5.5
rt2 running bgp# neighbor 9.9.9.9 address-family ipv4-unicast enabled false
rt2 running bgp# neighbor 9.9.9.9 address-family ipv4-vpn enabled true
rt2 running bgp# .. ..
rt2 running vrf main# l3vrf customer1
rt2 running l3vrf customer1#! table-id 10
rt2 running l3vrf customer1# routing bgp
rt2 running bgp# router-id 5.5.5.5
rt2 running bgp# as 65500
rt2 running bgp# # append bgp configuration for customer1 : l3vpn, ...
rt2 running bgp#

The above configuration results in having an mpls based route to reach the 40.0.0.9 BGP next hop. When importing the L3VRF entry in the customer1 L3VRF, the VPN label is appended with the implicit-null label. The show commands from above example are the same with or without LDP.

Relaxing Labelled path over GRE interfaces

The below configuration proposes to not configure any MPLS backbone. This mechanism works only if the BGP next hop entry selects a route that uses a GRE tunnel as outgoing interface. In those conditions, the L3VPN entries will not need the next hop to select a labelled path.

This setup is possible by using a specific route-map command called set l3vpn next-hop encapsulation gre. Because this command is available, and because the calculated path to install the BGP entry to is a GRE tunnel, then the installation is authorized.

rt1

rt1 running config# routing
rt1 running routing# route-map rmap
rt1 running route-map rmap#! seq 1
rt1 running seq 1#! policy permit
rt1 running seq 1# set l3vpn next-hop encapsulation gre
rt1 running seq 1# ..
rt1 running route-map rmap# ..
rt1 running routing# ..
rt1 running config# vrf main
rt1 running vrf main# routing bgp
rt1 running bgp#! router-id 5.5.5.5
rt1 running bgp#! as 65500
rt1 running bgp# neighbor 9.9.9.9 remote-as 65500
rt1 running bgp# neighbor 9.9.9.9 update-source 5.5.5.5
rt1 running bgp# neighbor 9.9.9.9 address-family ipv4-unicast enabled false
rt1 running bgp# neighbor 9.9.9.9 address-family ipv4-vpn enabled true
rt1 running bgp# neighbor 9.9.9.9 address-family ipv4-vpn route-map in route-map-name rmap
rt1 running bgp# .. ..
rt1 running vrf main# interface loopback loop1
rt1 running loopback loop1# ipv4 address 5.5.5.5/32
rt1 running loopback loop1# ..
rt1 running interface# physical eth1
rt1 running physical eth1#! port pci-b0s4
rt1 running physical eth1# ipv4 address 10.1.2.5/24
rt1 running physical eth1# ..
rt1 running interface# gre gre1
rt1 running gre gre1#! link-interface eth1
rt1 running gre gre1#! ipv4 address 40.0.0.5/24
rt1 running gre gre1#! local 10.1.2.5
rt1 running gre gre1# remote 10.3.2.9
rt1 running gre gre1# ..
rt1 running interface# ..
rt1 running vrf main# routing ospf
rt1 running ospf# router-id 10.1.2.5
rt1 running ospf# network 10.1.2.0/24 area 0.0.0.0
rt1 running ospf# ..
rt1 running routing# ..
rt1 running vrf main# routing static
rt1 running static# ipv4-route 9.9.9.9/32 next-hop 40.0.0.9%gre1
rt1 running static# ..
rt1 running routing# ..
rt1 running vrf main# l3vrf customer1
rt1 running l3vrf customer1#! table-id 10
rt1 running l3vrf customer1# routing bgp
rt1 running bgp# router-id 5.5.5.5
rt1 running bgp# as 65500
rt1 running bgp# # append bgp configuration for customer1 : l3vpn, ...
rt1 running bgp# ..
rt1 running routing# ..
rt1 running l3vrf customer1#

rt2

rt2 running config# routing
rt2 running routing# route-map rmap
rt2 running route-map rmap#! seq 1
rt2 running seq 1#! policy permit
rt2 running seq 1# set l3vpn next-hop encapsulation gre
rt2 running seq 1# ..
rt2 running route-map rmap# ..
rt2 running routing# ..
rt2 running config# vrf main
rt2 running vrf main# routing bgp
rt2 running bgp#! router-id 9.9.9.9
rt2 running bgp#! as 65500
rt2 running bgp# neighbor 5.5.5.5 remote-as 65500
rt2 running bgp# neighbor 5.5.5.5 update-source 9.9.9.9
rt2 running bgp# neighbor 5.5.5.5 address-family ipv4-unicast enabled false
rt2 running bgp# neighbor 5.5.5.5 address-family ipv4-vpn enabled true
rt2 running bgp# neighbor 5.5.5.5 address-family ipv4-vpn route-map in route-map-name rmap
rt2 running bgp# .. ..
rt2 running vrf main# interface
rt2 running interface# loopback loop1
rt2 running loopback loop1# ipv4 address 9.9.9.9/32
rt2 running loopback loop1# ..
rt2 running interface# physical eth1
rt2 running physical eth1#! port pci-b0s4
rt2 running physical eth1# ipv4 address 10.3.2.9/24
rt2 running physical eth1# ..
rt2 running interface# gre gre1
rt2 running gre gre1#! link-interface eth1
rt2 running gre gre1#! ipv4 address 40.0.0.9/24
rt2 running gre gre1#! local 10.3.2.9
rt2 running gre gre1# remote 10.1.2.5
rt2 running gre gre1# ..
rt2 running interface# ..
rt2 running vrf main# routing ospf
rt2 running ospf# router-id 10.3.2.9
rt2 running ospf# network 10.3.2.0/24 area 0.0.0.0
rt2 running ospf# ..
rt2 running routing# ..
rt2 running vrf main# routing static
rt2 running static# ipv4-route 5.5.5.5/32 next-hop 40.0.0.5%gre1
rt2 running static# ..
rt2 running routing# ..
rt2 running vrf main# l3vrf customer2
rt2 running l3vrf customer2#! table-id 10
rt2 running l3vrf customer2# routing bgp
rt2 running bgp# router-id 9.9.9.9
rt2 running bgp# as 65500
rt2 running bgp# # append bgp configuration for customer2 : l3vpn, ...
rt2 running bgp# ..
rt2 running routing#

L3VPN advanced configuration

Avoid routing loops by using SoO tag

There are situations, where VPNs are connected to various PEs. For example, a CE, or a group of devices forming an AS group needs to be connected to multiple PEs. This can also be the case for multihoming purposes. The below figure can help:

../../../../../_images/l3vpn-soo.svg

BGP l3vpn filtering prefixes with SOO use case example

The above diagram shows that a given prefix originated from ce1 is transmitted to pe1, then pe2. That prefix is also sent to ce2, which already knows that this prefix is reachable via its own private AS. One way to mitigate that situation is to use SOO extended communities on the PE. The below configuration can be used as example:

pe1

pe1 running config# routing
pe1 running routing# route-map add_soo_ext_community
pe1 running route-map add_soo_ext_community#! seq 1
pe1 running seq 1#! policy permit
pe1 running seq 1# set
pe1 running set# extcommunity
pe1 running extcommunity# soo 65500:0
pe1 running extcommunity# ..
pe1 running set# ..
pe1 running seq 1# ..
pe1 running route-map add_soo_ext_community# ..
pe1 running routing# ..
pe1 running config# vrf main
pe1 running vrf main# interface physical eth2
pe1 running physical eth2#! port pci-b0s5
pe1 running physical eth2# ipv4
pe1 running ipv4# address 10.125.0.1/24
pe1 running ipv4# ..
pe1 running physical eth2# ..
pe1 running interface# ..
pe1 running vrf main# routing bgp
pe1 running bgp#! as 65500
pe1 running bgp# router-id 1.1.1.1
pe1 running bgp# neighbor 10.125.0.2
pe1 running neighbor 10.125.0.2#! remote-as 65500
pe1 running neighbor 10.125.0.2# address-family
pe1 running address-family# ipv4-unicast
pe1 running ipv4-unicast# enabled false
pe1 running ipv4-unicast# ..
pe1 running address-family# ipv4-vpn
pe1 running ipv4-vpn# ..
pe1 running address-family# ..
pe1 running neighbor 10.125.0.2# ..
pe1 running bgp# .. ..
pe1 running vrf main# l3vrf customer2
pe1 running l3vrf customer2#! table-id 10
pe1 running l3vrf customer2# routing
pe1 running routing# bgp
pe1 running bgp# network-import-check false
pe1 running bgp# address-family
pe1 running address-family# ipv4-unicast
pe1 running ipv4-unicast# network 9.9.9.0/24
pe1 running network 9.9.9.0/24# ..
pe1 running ipv4-unicast# l3vpn
pe1 running l3vpn# export
pe1 running export# vpn true
pe1 running export# route-target 1:1
pe1 running export# route-distinguisher 65500:1
pe1 running export# route-map add_soo_ext_community
pe1 running export# ..
pe1 running l3vpn# import
pe1 running import#! vpn true
pe1 running import#! route-target 1:1
pe1 running import# ..
pe1 running l3vpn# ..
pe1 running ipv4-unicast# .. ..
pe1 running bgp# .. ..
pe1 running l3vrf customer2# interface
pe1 running interface# physical eth1
pe1 running physical eth1#! port pci-b0s4
pe1 running physical eth1# ipv4
pe1 running ipv4# address 9.9.9.0/24
pe1 running ipv4#

pe2

pe2 running routing# route-map filter_soo_from_65500_0
pe2 running route-map filter_soo_from_65500_0#! seq 1
pe2 running seq 1#! policy deny
pe2 running seq 1# match
pe2 running match# extcommunity filter_soo_comm_list
pe2 running match# ..
pe2 running seq 1# ..
pe2 running route-map filter_soo_from_65500_0# seq 2 policy permit
pe2 running route-map filter_soo_from_65500_0# ..
pe2 running routing# bgp
pe2 running bgp# extcommunity-list filter_soo_comm_list
pe2 running extcommunity-list filter_soo_comm_list#! policy 1 permit soo 65500:0
pe2 running extcommunity-list filter_soo_comm_list# ..
pe2 running bgp# ..
pe2 running routing# ..
pe2 running config# vrf main
pe2 running vrf main# routing
pe2 running routing# bgp
pe2 running bgp#! as 65500
pe2 running bgp# router-id 2.2.2.2
pe2 running bgp# neighbor 10.125.0.1
pe2 running neighbor 10.125.0.1#! remote-as 65500
pe2 running neighbor 10.125.0.1# address-family
pe2 running address-family# ipv4-unicast
pe2 running ipv4-unicast# enabled false
pe2 running ipv4-unicast# ..
pe2 running address-family# ipv4-vpn
pe2 running ipv4-vpn# ..
pe2 running address-family# ..
pe2 running neighbor 10.125.0.1# ..
pe2 running bgp# .. ..
pe2 running vrf main#
pe2 running vrf main# interface
pe2 running interface# physical eth2
pe2 running physical eth2#! port pci-b0s5
pe2 running physical eth2# ipv4
pe2 running ipv4# address 10.125.0.2/24
pe2 running ipv4# ..
pe2 running physical eth2# ..
pe2 running interface# ..
pe2 running vrf main# l3vrf customer2
pe2 running l3vrf customer2#! table-id 10
pe2 running l3vrf customer2# routing bgp
pe2 running bgp# address-family
pe2 running address-family# ipv4-unicast
pe2 running ipv4-unicast# l3vpn
pe2 running l3vpn# export
pe2 running export# vpn true
pe2 running export# label auto
pe2 running export# route-target 1:1
pe2 running export# route-distinguisher 65500:2
pe2 running export# ..
pe2 running l3vpn# import
pe2 running import#! vpn true
pe2 running import#! route-target 1:1
pe2 running import# route-map filter_soo_from_65500_0
pe2 running import# ..
pe2 running l3vpn# ..
pe2 running ipv4-unicast# ..
pe2 running address-family# ..
pe2 running bgp# ..
pe2 running routing# ..
pe2 running l3vrf customer2# interface
pe2 running interface# physical eth1
pe2 running physical eth2#! port pci-b0s4
pe2 running physical eth2# ipv4
pe2 running ipv4# address 9.9.9.0/24
pe2 running ipv4# ..
pe2 running physical eth2# ..
pe2 running interface#

As can be seen, when exporting prefixes from customer2 L3VRF, a SOO extended community is appended to BGP update and transmitted to the interconnection network. At reception, pe2 will filter incoming updates matching the SOO extended community. The update will be ignored in L3VRF for ce2. Below dump shows how to display the extended community-list attached to an incoming ipv4-vpn prefix. When detaching the route-map at inbound from L3VPN, the network 9.9.9.0/24 prefix becomes available again.

pe2 running config# show bgp ipv4 vpn prefix 9.9.9.0/24
BGP routing table entry for 65500:1:9.9.9.0/24
not allocated
Paths: (1 available, best #1)
  Not advertised to any peer
  Local
    10.125.0.1 from 10.125.0.1 (1.1.1.1)
      Origin incomplete, metric 0, localpref 100, valid, internal, best (First path received)
      Extended Community: RT:1:1 SoO:65500:0
      Remote label: 3
      Last update: Mon Nov 22 17:05:42 2021
pe2 running config# show bgp l3vrf customer2 ipv4 prefix 9.9.9.0/24
BGP routing table entry for 9.9.9.0/24
Paths: (1 available, no best path)
  Not advertised to any peer
  Imported from 65500:1:
  Local
    10.125.0.1 (inaccessible) from 0.0.0.0 (2.2.2.2) vrf 0 announce-nh-self
      Origin IGP, metric 0, localpref 100, invalid, sourced, local
      Extended Community: RT:1:1 SoO:65500:0
      Remote label: 3
      Last update: Mon Nov 22 17:29:00 2021

How to create half duplex connections

The previous chapters explained how to connect multiple VPNs by leaking traffic either locally or remotely. The same technology can be used to forbid traffic to leak locally. This can be the case with a PE device with multiple subscribers connected locally in separate VPNs, and where the ISPs want to monitor traffic on a central PE located in a separate place. There are some situations where PE hosts many subscribers, and authorizing local connectivity between subscribers may lead to additional complexity in monitoring such configuration. Redirecting traffic similarly to a hub and spoke way, permits reducing complexity on local PE. This situation can happen on large-scale wholesale service provider environments.

This can be done by instantiating VPN policies on each direction, for a given client. The wording half-duplex refers to the ability to associate different policies to traffic flow for each direction for a given client. The traffic is redirected to that central PE, similary to a star topology, where traffic is forwarded to the central PE before being redirected to the same PE, but in another VPN.

The below configuration illustrates the customer1 VPN that wants to access to the customer2 VPN. Two devices are configured, rt1 stands for the PE hosting the two VPNs, while rt2 acts as the central PE. The proposed way to implement such policies consists in creating 2 L3VRFs for each client.

  • one for hosting the client interface.

  • one for handling the upstream traffic. This one is associated to a first VPN policy that will import the remote networks. Usually, this L3VRF will host the default route to reach the central PE. This can also be any summary routes configured on the central PE.

  • one for handling the downstream traffic. This one is associated to a second VPN policy that will export the client network. The local interface is associated to that L3VRF. This permits to propagate networking information from OSPF into BGP for instance. Note that upstream is also received on that L3VRF, but is redirected to upstream L3VRF thanks to a default route leak.

rt1

rt1 running config# vrf main
rt1 running vrf main# interface physical eth1
rt1 running physical eth1#! port pci-b0s4
rt1 running physical eth1# ..
rt1 running interface# physical eth0
rt1 running physical eth0#! port pci-b0s5
rt1 running physical eth0# ipv4
rt1 running ipv4# address 10.125.0.1/24
rt1 running ipv4# .. ..
rt1 running interface# loopback loop1
rt1 running loopback loop1# ipv4
rt1 running ipv4# address 1.1.1.1/32
rt1 running ipv4# ..
rt1 running loopback loop1# ..
rt1 running interface# ..
rt1 running vrf main# routing ospf
rt1 running ospf# router-id 1.1.1.1
rt1 running ospf# network 1.1.1.1/32 area 0.0.0.0
rt1 running ospf# network 10.125.0.0/24 area 0.0.0.0
rt1 running ospf# ..
rt1 running routing# ..
rt1 running vrf main# routing mpls ldp
rt1 running ldp# router-id 1.1.1.1
rt1 running ldp# address-family ipv4
rt1 running ipv4#! discovery transport-address 1.1.1.1
rt1 running ipv4# interface eth0
rt1 running interface eth0# / vrf main routing bgp
rt1 running bgp#! as 65500
rt1 running bgp# router-id 1.1.1.1
rt1 running bgp# address-family
rt1 running address-family# ipv4-unicast
rt1 running ipv4-unicast# ..
rt1 running address-family# ..
rt1 running bgp# neighbor 2.2.2.2
rt1 running neighbor 2.2.2.2#! remote-as 65500
rt1 running neighbor 2.2.2.2# update-source 1.1.1.1
rt1 running neighbor 2.2.2.2# address-family
rt1 running address-family# ipv4-unicast
rt1 running ipv4-unicast# enabled false
rt1 running ipv4-unicast# ..
rt1 running address-family# ipv4-vpn
rt1 running ipv4-vpn# / vrf main l3vrf customer1_up
rt1 running l3vrf customer1_up#! table-id 100
rt1 running l3vrf customer1_up# routing bgp
rt1 running bgp# address-family
rt1 running address-family# ipv4-unicast
rt1 running ipv4-unicast# l3vpn
rt1 running l3vpn# import
rt1 running import#! vpn true
rt1 running import#! route-target 100:65500
rt1 running import# route-target 200:65500
rt1 running import# / vrf main l3vrf customer2_up
rt1 running l3vrf customer2_up#! table-id 200
rt1 running l3vrf customer2_up# routing bgp
rt1 running bgp# address-family
rt1 running address-family# ipv4-unicast
rt1 running ipv4-unicast# l3vpn
rt1 running l3vpn# import
rt1 running import#! vpn true
rt1 running import#! route-target 200:65500
rt1 running import# route-target 100:65500
rt1 running import# / vrf main l3vrf customer1_down
rt1 running l3vrf customer1_down#! table-id 101
rt1 running l3vrf customer1_down# routing
rt1 running routing# static
rt1 running static# ipv4-route 0.0.0.0/0
rt1 running ipv4-route 0.0.0.0/0#! next-hop customer1_up nexthop-l3vrf customer1_up
rt1 running ipv4-route 0.0.0.0/0# .. ..
rt1 running routing# bgp
rt1 running bgp# address-family
rt1 running address-family# ipv4-unicast
rt1 running ipv4-unicast# network 10.100.0.0/24
rt1 running network 10.100.0.0/24# ..
rt1 running ipv4-unicast# l3vpn
rt1 running l3vpn# export
rt1 running export# vpn true
rt1 running export# label auto
rt1 running export# route-target 101:65500
rt1 running export# route-distinguisher 101:65500
rt1 running export# nexthop 1.1.1.1
rt1 running export# / vrf main l3vrf customer1_down
rt1 running l3vrf customer1_down# interface vlan eth1.10
rt1 running vlan eth1.10#! description customer1
rt1 running vlan eth1.10#! ipv4
rt1 running ipv4#! address 10.100.0.1/24
rt1 running ipv4#! ..
rt1 running vlan eth1.10#! vlan-id 10
rt1 running vlan eth1.10#! link-interface eth1
rt1 running vlan eth1.10# / vrf main l3vrf customer2_down
rt1 running l3vrf customer2_down#! table-id 201
rt1 running l3vrf customer2_down# routing
rt1 running routing# static
rt1 running static# ipv4-route 0.0.0.0/0
rt1 running ipv4-route 0.0.0.0/0#! next-hop customer2_up nexthop-l3vrf customer2_up
rt1 running ipv4-route 0.0.0.0/0# .. ..
rt1 running routing# bgp
rt1 running bgp# address-family
rt1 running address-family# ipv4-unicast
rt1 running ipv4-unicast# network 10.101.0.0/24
rt1 running network 10.101.0.0/24# ..
rt1 running ipv4-unicast# l3vpn
rt1 running l3vpn# export
rt1 running export# vpn true
rt1 running export# label auto
rt1 running export# route-target 201:65500
rt1 running export# route-distinguisher 201:65500
rt1 running export# nexthop 1.1.1.1
rt1 running export# / vrf main l3vrf customer2_down
rt1 running l3vrf customer2_down# interface vlan eth1.20
rt1 running vlan eth1.20#! ipv4
rt1 running ipv4#! address 10.101.0.1/24
rt1 running ipv4#! ..
rt1 running vlan eth1.20#! vlan-id 20
rt1 running vlan eth1.20#! link-interface eth1
rt1 running vlan eth1.20# ..
rt1 running interface# ..
rt1 running l3vrf customer2_down#

The below rt2 configuration reflects a typical configuration where a given L3VRF is being associated a single VPN policy. This VPN will receive networks from the remote downstream L3VRFs, and also push default route to the remote upstream L3VRFs.

rt2

rt2 running config# vrf main
rt2 running vrf main# interface
rt2 running interface# physical eth0
rt2 running physical eth0#! port pci-b0s5
rt2 running physical eth0# ipv4
rt2 running ipv4# address 10.125.0.2/24
rt2 running ipv4# ..
rt2 running physical eth0# ..
rt2 running interface# loopback loop1
rt2 running loopback loop1# ipv4
rt2 running ipv4# address 2.2.2.2/32
rt2 running ipv4# ..
rt2 running loopback loop1# ..
rt2 running interface# ..
rt2 running vrf main# routing
rt2 running routing# bgp
rt2 running bgp#! as 65500
rt2 running bgp# router-id 2.2.2.2
rt2 running bgp# address-family
rt2 running address-family# ipv4-unicast
rt2 running ipv4-unicast# enabled false
rt2 running ipv4-unicast# ..
rt2 running address-family# ..
rt2 running bgp# neighbor 1.1.1.1
rt2 running neighbor 1.1.1.1#! remote-as 65500
rt2 running neighbor 1.1.1.1# update-source 2.2.2.2
rt2 running neighbor 1.1.1.1# address-family
rt2 running address-family# ipv4-unicast
rt2 running ipv4-unicast# enabled false
rt2 running ipv4-unicast# ..
rt2 running address-family# ipv4-vpn
rt2 running ipv4-vpn# ..
rt2 running address-family# ..
rt2 running neighbor 1.1.1.1# ..
rt2 running bgp# ..
rt2 running routing# ospf
rt2 running ospf# router-id 2.2.2.2
rt2 running ospf# network 2.2.2.2/32 area 0
rt2 running ospf# network 10.125.0.0/24 area 0
rt2 running ospf# ..
rt2 running routing# mpls ldp
rt2 running ldp# router-id 2.2.2.2
rt2 running ldp# address-family ipv4
rt2 running ipv4#! discovery transport-address 2.2.2.2
rt2 running ipv4# interface eth0
rt2 running interface eth0# / vrf main
rt2 running vrf main# l3vrf customer1
rt2 running l3vrf customer1#! table-id 100
rt2 running l3vrf customer1# routing bgp
rt2 running bgp# address-family
rt2 running address-family# ipv4-unicast
rt2 running ipv4-unicast# network 0.0.0.0/0
rt2 running network 0.0.0.0/0# ..
rt2 running ipv4-unicast# l3vpn
rt2 running l3vpn# import
rt2 running import#! vpn true
rt2 running import#! route-target 101:65500
rt2 running import# route-target 201:65500
rt2 running import# ..
rt2 running l3vpn# ..
rt2 running ipv4-unicast# l3vpn
rt2 running l3vpn# export
rt2 running export# vpn true
rt2 running export# label auto
rt2 running export# route-target 100:65500
rt2 running export# route-distinguisher 100:65500
rt2 running export# ..
rt2 running l3vpn# ..
rt2 running ipv4-unicast# ..
rt2 running address-family# ..
rt2 running bgp# ..
rt2 running routing# ..
rt2 running l3vrf customer1# ..
rt2 running vrf main# l3vrf customer2
rt2 running l3vrf customer2#! table-id 200
rt2 running l3vrf customer2# routing bgp
rt2 running bgp# address-family
rt2 running address-family# ipv4-unicast
rt2 running ipv4-unicast# network 0.0.0.0/0
rt2 running network 0.0.0.0/0# ..
rt2 running ipv4-unicast# l3vpn
rt2 running l3vpn# import
rt2 running import#! vpn true
rt2 running import#! route-target 101:65500
rt2 running import# route-target 201:65500
rt2 running import# ..
rt2 running l3vpn# ..
rt2 running ipv4-unicast# l3vpn
rt2 running l3vpn# export
rt2 running export# vpn true
rt2 running export# label auto
rt2 running export# route-target 200:65500
rt2 running export# route-distinguisher 200:65500
rt2 running export# ..
rt2 running l3vpn# ..
rt2 running ipv4-unicast# ..
rt2 running address-family# ..
rt2 running bgp#

Such configuration on rt1 is completely scalable from both the configuration perspective, and the performance perspective.

Withdraw VPN prefixes which are not imported locally

It is possible to retain or not VPN prefixes that are not imported by local VRF configuration. This can be done via the following command in the context of the desired VPN address-family. By default, all prefixes are retained.

vsr running config# vrf main
vsr running vrf main# routing bgp
vsr running bgp# as 65500
vsr running bgp# address-family ipv4-vpn
vsr running ipv4-vpn# retain route-target all false
vsr running ipv4-vpn#