BGP for L3VPN¶
L3VPN basic configuration¶
L3VPN is one of the VPN interconnection services that BGP offers. This chapter explains how to configure L3VPN service.
Configuring BGP L3VPN service¶
The below configuration shows how to configure L3VPN service on the main BGP instance, with a given neighbor:
vsr running config# / vrf main routing bgp as 65500
vsr running config# / vrf main routing bgp neighbor 2.2.2.2 remote-as 65500
vsr running config# / vrf main routing bgp neighbor 2.2.2.2 address-family ipv4-vpn
Configuring BGP route distribution¶
The below configuration shows how to configure a RD for all IP information from a given L3VRF. A separate BGP instance needs to be configured.
vsr running config# / vrf main l3vrf customer1 table-id 10
vsr running config# / vrf main l3vrf customer1 routing bgp as 65500
vsr running config# / vrf main l3vrf customer1 routing bgp address-family ipv4-unicast l3vpn export route-distinguisher 65500:55
The below configuration shows how to configure a RT for a given L3VRF:
vsr running config# / vrf main l3vrf customer1 table-id 10
vsr running config# / vrf main l3vrf customer1 routing bgp as 65500
vsr running config# / vrf main l3vrf customer1 routing bgp address-family ipv4-unicast l3vpn export route-target 11:22
vsr running config# / vrf main l3vrf customer1 routing bgp address-family ipv4-unicast l3vpn import route-target 11:22 route-target 22:44
L3VPN use case example¶
The above drawing illustrates a setup made up of 2 symmetrical sites. Each site is separated with a PE device. In the case the site is a data center, the PE could be replaced with a DC-GW. To simplify, each site is made up of 2 distinct VPNs. L3VPN service will be gradually configured to do the following:
Interconnect traffic from different VPNs on the same site. This method is recommended only if the user has also some interest in using the same service for interconnecting sites. Otherwise, the user may use BGP peerings between VRFs.
See also
Interconnect traffic between two VPNs located on remote sites.
Configure locally route leaking between VPNs¶
BGP configuration¶
The below configuration illustrates a setup with two VPNs. A BGP instance
is set up in each of the two L3VRFs instances and on the default BGP
instance.
Also, each L3VRF BGP instance has its own RD. The RT export settings
of each L3VRF is being matched by the RT import settings of the other
L3VRF.
vsr running config# / vrf main routing bgp as 65500
vsr running config# / vrf main routing bgp router-id 1.1.1.1
vsr running config# / vrf main l3vrf customer1 table-id 10
vsr running config# / vrf main l3vrf customer1 interface physical eth1 port pci-b0s4
vsr running config# / vrf main l3vrf customer1 interface physical eth1 ipv4 address 192.168.3.1/24
vsr running config# / vrf main l3vrf customer1 routing bgp as 65500
vsr running config# / vrf main l3vrf customer1 routing bgp router-id 1.1.1.1
vsr running config# / vrf main l3vrf customer1 routing bgp address-family ipv4-unicast l3vpn export vpn true
vsr running config# / vrf main l3vrf customer1 routing bgp address-family ipv4-unicast l3vpn export route-target 11:22
vsr running config# / vrf main l3vrf customer1 routing bgp address-family ipv4-unicast l3vpn export route-distinguisher 1:55
vsr running config# / vrf main l3vrf customer1 routing bgp address-family ipv4-unicast l3vpn import vpn true
vsr running config#! / vrf main l3vrf customer1 routing bgp address-family ipv4-unicast l3vpn import route-target 11:22 route-target 22:44
vsr running config# / vrf main l3vrf customer1 routing bgp address-family ipv4-unicast redistribute connected
vsr running config# / vrf main l3vrf customer2 table-id 20
vsr running config# / vrf main l3vrf customer2 interface physical eth2 port pci-b0s5
vsr running config# / vrf main l3vrf customer2 interface physical eth2 ipv4 address 192.168.2.1/24
vsr running config# / vrf main l3vrf customer2 routing bgp as 65500
vsr running config# / vrf main l3vrf customer2 routing bgp router-id 1.1.1.1
vsr running config# / vrf main l3vrf customer2 routing bgp address-family ipv4-unicast l3vpn export vpn true
vsr running config# / vrf main l3vrf customer2 routing bgp address-family ipv4-unicast l3vpn export route-target 22:44
vsr running config# / vrf main l3vrf customer2 routing bgp address-family ipv4-unicast l3vpn export route-distinguisher 2:55
vsr running config# / vrf main l3vrf customer2 routing bgp address-family ipv4-unicast l3vpn import vpn true
vsr running config#! / vrf main l3vrf customer2 routing bgp address-family ipv4-unicast l3vpn import route-target 11:22 route-target 22:44
vsr running config# / vrf main l3vrf customer2 routing bgp address-family ipv4-unicast redistribute connected
Note
It is not recommended to set the same values for both RD and RT. RD stands for a given VPN identifier of the local ISP. RT stands for the exchanged information with the remote BGP peer, and is set accordingly by operators.
With the above configuration applied, L3VRF route leaking is possible.
Subsequently, if BGP peering is done between a CE and the BGP instance
of each L3VRF instance, then route importation and exportation occurs. The
below output demonstrates that the routes from the customer2 L3VRF have
been imported to the customer1 VRF. The L3VRF route leaks are visible
with the @xx< indicating that the route entry is originated from the
customer2 L3VRF.
vsr> show bgp l3vrf customer1 ipv4
BGP table version is 3, local router ID is 1.1.1.1, vrf id 7
Default local pref 100, local AS 65500
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found
Network Next Hop Metric LocPrf Weight Path
*> 192.168.2.0/24 0.0.0.0@8< 0 32768 ?
*> 192.168.3.0/24 0.0.0.0 0 32768 ?
Displayed 2 routes and 2 total paths
Note
The route leak entry contains a number just after @ character. This number
stands for the identifier of the L3VRF where the prefix is originated from.
To get the associated L3VRF, the below command gives the Id identifier
associated with the L3VRF name.
vsr> show bgp l3vrfs
Type Id routerId #PeersCfg #PeersEstb Name
L3-VNI RouterMAC Interface
DFLT 0 1.1.1.1 0 0 default
0 00:00:00:00:00:00 unknown
VRF 7 1.1.1.1 0 0 customer1
0 00:00:00:00:00:00 unknown
VRF 8 1.1.1.1 0 0 customer2
0 00:00:00:00:00:00 unknown
Total number of VRFs (including default): 3
Routing output¶
The following output displays the routing entries available in the customer1
L3VRF routing table. The local route leak is emphasized in the below output.
Generated local route leaks are not using a label for encapsulation as the
route is directly connected to the targeted L3VRF instance.
Note
Ensure that the entries are selected in the bgp RIB. For instance, the
learned route may not be selected if the BGP instance from the customer2
L3VRF used the network command. By default, the network import-check
command is turned on and may invalidate the entry.
vsr> show ipv4-routes l3vrf customer1
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, N - NHRP, T - Table
> - selected route, * - FIB route, r - rejected, b - backup
L3VRF customer1:
K>* 0.0.0.0/0 [255/8192] unreachable (ICMP unreachable), 00:00:04
B>* 192.168.2.0/24 [20/0] is directly connected, customer2 (l3vrf customer2), weight 1, 00:00:02
C>* 192.168.3.0/24 is directly connected, eth1, 00:00:04
3 routes displayed.
Note
When doing local route leaking, the ECMP routing is not supported. Ensure that there are no overlapping addresses across the VPNs to interconnect.
Interconnect two geographically separate VPNs¶
Connecting remote VPNs together is made with route leaks using MPLS labels as encapsulation. Conveying MPLS frames requires having an MPLS backbone. The MPLS backbone is generally based on an IP based network, interconnected by using an IGP protocol like OSPF or IS-IS. Then MPLS is turned on on this network. If the MPLS backbone is a non SR network, then LDP service needs to be configured. Otherwise, an IGP like IS-IS that leverages SR capability can be used.
See also
LDP configuration, see LDP configuration.
IS-IS SR configuration, see ISIS Segment Routing configuration.
In order to create route leaks between each VPN, the ipv4-vpn address-family
must be turned on on the default BGP instance. (respectively ipv6-vpn
address-family for IPv6 VPN traffic.
vsr running config# vrf main
vsr running vrf main# interface physical eth0
vsr running physical eth0#! port pci-b0s5
vsr running physical eth0# ipv4 address 1.1.1.1/32
vsr running physical eth0# .. ..
vsr running vrf main# routing bgp
vsr running bgp# as 65500
vsr running bgp# neighbor 2.2.2.2 remote-as 65500
vsr running bgp# neighbor 2.2.2.2 update-source 1.1.1.1
vsr running bgp# neighbor 2.2.2.2 address-family ipv4-vpn enabled true
vsr running bgp#
Consequently, having an ipv4-vpn peering will trigger the importation of
L3VPN entries. For instance, the presence of remote VPNs and associated
prefixes from the 2.2.2.2 peer will trigger prefixes importation in the
relevant L3VRFs. Those prefixes will be imported in the associated L3VRF,
if the remote VPNs match the local VPNs. Route leaks will be injected to
cross the border between the default L3VRF, and the given L3VRF.
In order for BGP to be able to export its own labels, BGP must be configured
so as to rely on its own labels, either automatically, or by choosing its own.
The below configuration illustrates the automatic label chosen by the customer1
L3VRF, while the customer2 L3VRF chooses to hard-set its exportation label
to 300.
vsr running config# vrf main
vsr running vrf main# l3vrf customer1
vsr running l3vrf customer1# routing bgp
vsr running bgp# as 65500
vsr running bgp# address-family ipv4-unicast
vsr running ipv4-unicast# l3vpn export label auto
vsr running ipv4-unicast# .. ..
vsr running bgp# .. ..
vsr running l3vrf customer1# ..
vsr running vrf main# l3vrf customer2
vsr running l3vrf customer2# routing bgp
vsr running bgp# as 65500
vsr running bgp# address-family ipv4-unicast
vsr running ipv4-unicast# l3vpn export label 300
vsr running ipv4-unicast#
The below show bgp ipv4 vpn command dumps the remotely learned prefixes,
while the show ipv4-routes l3vrf customer1 command dumps the imported
prefixes that are installed on the local L3VRF.
vsr> show bgp ipv4 vpn
BGP table version is 5, local router ID is 1.1.1.1, vrf id 0
Default local pref 100, local AS 65500
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 1:55
*> 192.168.3.0/24 0.0.0.0@7< 0 32768 ?
*>i192.168.4.0/24 2.2.2.2 0 100 0 ?
Route Distinguisher: 2:55
*> 192.168.2.0/24 0.0.0.0@8< 0 32768 ?
*>i192.168.5.0/24 2.2.2.2 0 100 0 ?
vsr> show ipv4-routes l3vrf customer1
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, A - Babel, D - SHARP, F - PBR, f - OpenFabric,
> - selected route, * - FIB route, q - queued, r - rejected, b - backup
t - trapped, o - offload failure
VRF customer1:
B>* 192.168.2.0/24 [200/0] is directly connected, eth2(vrf customer2), 00:05:26
C>* 192.168.3.0/24 is directly connected, eth1, 00:19:06
B> 192.168.4.0/24 [200/0] via 2.2.2.2(vrf main) (recursive), label 16, 00:05:26
* via 2.2.2.2, eth0(vrf main) onlink, label 16, 00:05:26
B> 192.168.5.0/24 [200/0] via 2.2.2.2(vrf main) (recursive), label 600, 00:01:14
* via 2.2.2.2, eth0(vrf main) onlink, label 600, 00:01:14
The following show bgp ipv4 vpn prefix gives detailed information about a specific
prefix and its associated attributes. The exported emphasized prefix has a hard set
label set to 300.
vsr> show bgp ipv4 vpn prefix 192.168.2.0/24
BGP routing table entry for 2:55:192.168.2.0/24
not allocated
Paths: (1 available, best #1)
Advertised to non peer-group peers:
2.2.2.2
Local
0.0.0.0 from 0.0.0.0 (1.1.1.1) vrf 8 announce-nh-self
Origin incomplete, metric 0, weight 32768, valid, sourced, local, best (First path received)
Extended Community: RT:22:44
Originator: 1.1.1.1
Remote label: 300
Last update: Mon Aug 23 07:34:44 2021
Setup Example¶
The above diagram illustrates a topology made up of an MPLS-based backbone,
with some LSR devices: rt3 and rt4, and LER devices. Each LER
device has a BGP core instance that has ipv4-vpn address-family enabled.
Next to each BGP instance, a BGP instance is created in each L3VRF.
Each L3VRF stands for a private network, either A or B. The color
semantic explains the relationship between the private networks on the
various LER devices.
The displayed arrows indicate the interconnection between the VPNs defined on each site. The below configuration permits to do the following:
The green arrow connects all
Bnetworks together.Bnetwork fromrt2is interconnected withBnetwork fromrt1.The orange arrow connects all
Anetworks together.Anetwork fromrt2is interconnected withAnetwork fromrt1andAnetwork fromrt5.
rt1
rt1 running config# vrf main
rt1 running vrf main# interface
rt1 running interface# physical eth0_0
rt1 running physical eth0_0#! port pci-b0s6
rt1 running physical eth0_0# ipv4
rt1 running ipv4# address 6.6.6.1/24
rt1 running ipv4# ..
rt1 running physical eth0_0# ..
rt1 running interface# loopback loop1
rt1 running loopback loop1# ipv4
rt1 running ipv4# address 5.5.5.5/32
rt1 running ipv4# ..
rt1 running loopback loop1# ..
rt1 running interface# ..
rt1 running vrf main# routing
rt1 running routing# mpls
rt1 running mpls# ldp
rt1 running ldp# router-id 5.5.5.5
rt1 running ldp# dual-stack
rt1 running dual-stack# transport-preference ipv4
rt1 running dual-stack# ..
rt1 running ldp# address-family
rt1 running address-family# ipv4
rt1 running ipv4#! discovery
rt1 running discovery#! transport-address 5.5.5.5
rt1 running discovery# ..
rt1 running ipv4# interface eth0_0
rt1 running interface eth0_0# ..
rt1 running ipv4# ..
rt1 running address-family# ..
rt1 running ldp# ..
rt1 running mpls# ..
rt1 running routing# ospf
rt1 running ospf# network 6.6.6.0/24 area 0
rt1 running ospf# network 5.5.5.5/32 area 0
rt1 running ospf# redistribute connected
rt1 running ospf# ..
rt1 running routing# bgp
rt1 running bgp#! as 65500
rt1 running bgp# router-id 5.5.5.5
rt1 running bgp# neighbor 9.9.9.9
rt1 running neighbor 9.9.9.9#! remote-as 65500
rt1 running neighbor 9.9.9.9# update-source 5.5.5.5
rt1 running neighbor 9.9.9.9# address-family
rt1 running address-family# ipv4-vpn
rt1 running ipv4-vpn# ..
rt1 running address-family# ..
rt1 running neighbor 9.9.9.9# ..
rt1 running bgp# neighbor 15.15.15.15
rt1 running neighbor 15.15.15.15#! remote-as 65500
rt1 running neighbor 15.15.15.15# update-source 5.5.5.5
rt1 running neighbor 15.15.15.15# address-family
rt1 running address-family# ipv4-vpn
rt1 running ipv4-vpn# ..
rt1 running address-family# ..
rt1 running neighbor 15.15.15.15# ..
rt1 running bgp# ..
rt1 running routing# ..
rt1 running vrf main# l3vrf customer1
rt1 running l3vrf customer1#! table-id 10
rt1 running l3vrf customer1# # add l3vrf customer1 network configuration
rt1 running l3vrf customer1# routing bgp
rt1 running bgp# as 65500
rt1 running bgp# router-id 5.5.5.5
rt1 running bgp# # add neighbor information for customer1
rt1 running bgp# address-family ipv4-unicast
rt1 running ipv4-unicast# redistribute connected
rt1 running ipv4-unicast# l3vpn export route-distinguisher 65500:1
rt1 running ipv4-unicast# l3vpn export route-target 1:55
rt1 running ipv4-unicast# l3vpn export label auto
rt1 running ipv4-unicast# l3vpn export vpn true
rt1 running ipv4-unicast# l3vpn import route-target 1:55
rt1 running ipv4-unicast# l3vpn import vpn true
rt1 running ipv4-unicast# .. ..
rt1 running bgp# .. ..
rt1 running l3vrf customer1# ..
rt1 running vrf main# l3vrf customer2
rt1 running l3vrf customer2#! table-id 20
rt1 running l3vrf customer2# # add l3vrf customer2 network configuration
rt1 running l3vrf customer2# routing bgp
rt1 running bgp# as 65500
rt1 running bgp# router-id 5.5.5.5
rt1 running bgp# # add neighbor information for customer2
rt1 running bgp# address-family ipv4-unicast
rt1 running ipv4-unicast# redistribute connected
rt1 running ipv4-unicast# l3vpn export route-distinguisher 65500:2
rt1 running ipv4-unicast# l3vpn export route-target 2:55
rt1 running ipv4-unicast# l3vpn export label auto
rt1 running ipv4-unicast# l3vpn export vpn true
rt1 running ipv4-unicast# l3vpn import route-target 2:55
rt1 running ipv4-unicast# l3vpn import vpn true
rt1 running ipv4-unicast#
rt2
rt2 running config# vrf main
rt2 running vrf main# interface
rt2 running interface# physical eth0_0
rt2 running physical eth0_0#! port pci-b0s5
rt2 running physical eth0_0# ipv4
rt2 running ipv4# address 8.8.8.2/24
rt2 running ipv4# ..
rt2 running physical eth0_0# ..
rt2 running interface# loopback loop1
rt2 running loopback loop1# ipv4
rt2 running ipv4# address 9.9.9.9/32
rt2 running ipv4# ..
rt2 running loopback loop1# ..
rt2 running interface# ..
rt2 running vrf main# routing
rt2 running routing# mpls
rt2 running mpls# ldp
rt2 running ldp# router-id 9.9.9.9
rt2 running ldp# dual-stack
rt2 running dual-stack# transport-preference ipv4
rt2 running dual-stack# ..
rt2 running ldp# address-family
rt2 running address-family# ipv4
rt2 running ipv4#! discovery
rt2 running discovery#! transport-address 9.9.9.9
rt2 running discovery# ..
rt2 running ipv4# interface eth0_0
rt2 running interface eth0_0# ..
rt2 running ipv4# ..
rt2 running address-family# ..
rt2 running ldp# ..
rt2 running mpls# ..
rt2 running routing# ospf
rt2 running ospf# network 8.8.8.0/24 area 0
rt2 running ospf# network 9.9.9.9/32 area 0
rt2 running ospf# redistribute connected
rt2 running ospf# ..
rt2 running routing# bgp
rt2 running bgp#! as 65500
rt2 running bgp# router-id 9.9.9.9
rt2 running bgp# neighbor 5.5.5.5
rt2 running neighbor 5.5.5.5#! remote-as 65500
rt2 running neighbor 5.5.5.5# update-source 9.9.9.9
rt2 running neighbor 5.5.5.5# address-family
rt2 running address-family# ipv4-vpn
rt2 running ipv4-vpn# ..
rt2 running address-family# ..
rt2 running neighbor 5.5.5.5# ..
rt2 running bgp# neighbor 15.15.15.15
rt2 running neighbor 15.15.15.15#! remote-as 65500
rt2 running neighbor 15.15.15.15# update-source 9.9.9.9
rt2 running neighbor 15.15.15.15# address-family
rt2 running address-family# ipv4-vpn
rt2 running ipv4-vpn# ..
rt2 running address-family# ..
rt2 running neighbor 15.15.15.15# ..
rt2 running bgp# ..
rt2 running routing# ..
rt2 running vrf main# l3vrf customer1
rt2 running l3vrf customer1#! table-id 10
rt2 running l3vrf customer1# # add l3vrf customer1 network configuration
rt2 running l3vrf customer1# routing bgp
rt2 running bgp# as 65500
rt2 running bgp# router-id 9.9.9.9
rt2 running bgp# # add neighbor information for customer1
rt2 running bgp# address-family ipv4-unicast
rt2 running ipv4-unicast# redistribute connected
rt2 running ipv4-unicast# l3vpn export route-distinguisher 65500:3
rt2 running ipv4-unicast# l3vpn export route-target 1:55
rt2 running ipv4-unicast# l3vpn export label auto
rt2 running ipv4-unicast# l3vpn export vpn true
rt2 running ipv4-unicast# l3vpn import route-target 1:55
rt2 running ipv4-unicast# l3vpn import vpn true
rt2 running ipv4-unicast# .. ..
rt2 running bgp# .. ..
rt2 running l3vrf customer1# ..
rt2 running vrf main# l3vrf customer2
rt2 running l3vrf customer2#! table-id 20
rt2 running l3vrf customer2# # add l3vrf customer2 network configuration
rt2 running l3vrf customer2# routing bgp
rt2 running bgp# as 65500
rt2 running bgp# router-id 9.9.9.9
rt2 running bgp# # add neighbor information for customer2
rt2 running bgp# address-family ipv4-unicast
rt2 running ipv4-unicast# redistribute connected
rt2 running ipv4-unicast# l3vpn export route-distinguisher 65500:4
rt2 running ipv4-unicast# l3vpn export route-target 2:55
rt2 running ipv4-unicast# l3vpn export label auto
rt2 running ipv4-unicast# l3vpn export vpn true
rt2 running ipv4-unicast# l3vpn import route-target 2:55
rt2 running ipv4-unicast# l3vpn import vpn true
rt2 running ipv4-unicast#
rt5
rt5 running config# vrf main
rt5 running vrf main# interface
rt5 running interface# physical eth0_0
rt5 running physical eth0_0#! port pci-b0s5
rt5 running physical eth0_0# ipv4
rt5 running ipv4# address 12.12.12.0/24
rt5 running ipv4# ..
rt5 running physical eth0_0# ..
rt5 running interface# loopback loop1
rt5 running loopback loop1# ipv4
rt5 running ipv4# address 15.15.15.15/32
rt5 running ipv4# ..
rt5 running loopback loop1# ..
rt5 running interface# ..
rt5 running vrf main# routing
rt5 running routing# mpls
rt5 running mpls# ldp
rt5 running ldp# router-id 15.15.15.15
rt5 running ldp# dual-stack
rt5 running dual-stack# transport-preference ipv4
rt5 running dual-stack# ..
rt5 running ldp# address-family
rt5 running address-family# ipv4
rt5 running ipv4#! discovery
rt5 running discovery#! transport-address 15.15.15.15
rt5 running discovery# ..
rt5 running ipv4# interface eth0_0
rt5 running interface eth0_0# ..
rt5 running ipv4# ..
rt5 running address-family# ..
rt5 running ldp# ..
rt5 running mpls# ..
rt5 running routing# ospf
rt5 running ospf# network 12.12.12.0/24 area 0
rt5 running ospf# network 15.15.15.15/32 area 0
rt5 running ospf# redistribute connected
rt5 running ospf# ..
rt5 running routing# bgp
rt5 running bgp#! as 65500
rt5 running bgp# router-id 15.15.15.15
rt5 running bgp# neighbor 5.5.5.5
rt5 running neighbor 5.5.5.5#! remote-as 65500
rt5 running neighbor 5.5.5.5# update-source 15.15.15.15
rt5 running neighbor 5.5.5.5# address-family
rt5 running address-family# ipv4-vpn
rt5 running ipv4-vpn# ..
rt5 running address-family# ..
rt5 running neighbor 5.5.5.5# ..
rt5 running bgp# neighbor 9.9.9.9
rt5 running neighbor 9.9.9.9#! remote-as 65500
rt5 running neighbor 9.9.9.9# update-source 15.15.15.15
rt5 running neighbor 9.9.9.9# address-family
rt5 running address-family# ipv4-vpn
rt5 running ipv4-vpn# ..
rt5 running address-family# ..
rt5 running neighbor 15.15.15.15# ..
rt5 running bgp# ..
rt5 running routing# ..
rt5 running vrf main# l3vrf customer1
rt5 running l3vrf customer1#! table-id 10
rt5 running l3vrf customer1# # add l3vrf customer1 network configuration
rt5 running l3vrf customer1# routing bgp
rt5 running bgp# as 65500
rt5 running bgp# router-id 15.15.15.15
rt5 running bgp# # add neighbor information for customer1
rt5 running bgp# address-family ipv4-unicast
rt5 running ipv4-unicast# redistribute connected
rt5 running ipv4-unicast# l3vpn export route-distinguisher 65500:5
rt5 running ipv4-unicast# l3vpn export route-target 1:55
rt5 running ipv4-unicast# l3vpn export label 300
rt5 running ipv4-unicast# l3vpn export vpn true
rt5 running ipv4-unicast# l3vpn import route-target 1:55
rt5 running ipv4-unicast# l3vpn import vpn true
rt5 running ipv4-unicast# .. ..
rt5 running bgp# .. ..
rt5 running l3vrf customer1#
As can be seen, the network prefixes learned by each BGP instance are
either learned (by adding extra CE configuration linked to each instance)
or imported thanks to the L3VPN service. The below command dumps the vpnv4
entries:
rt1> show bgp ipv4 vpn
BGP table version is 27, local router ID is 5.5.5.5, vrf id 0
Default local pref 100, local AS 65500
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 65500:1
*> 10.101.0.0/24 1.1.1.2@7< 100 0 i
*> 10.101.1.0/24 1.1.1.2@7< 100 0 i
*> 10.101.2.0/24 1.1.1.2@7< 100 0 i
[..]
Route Distinguisher: 65500:3
*>i10.101.11.0/24 9.9.9.9 100 0 i
*>i10.101.12.0/24 9.9.9.9 100 0 i
*>i10.101.13.0/24 9.9.9.9 100 0 i
[..]
Route Distinguisher: 65500:5
*>i10.101.21.0/24 15.15.15.15 100 0 i
*>i10.101.22.0/24 15.15.15.15 100 0 i
*>i10.101.23.0/24 15.15.15.15 100 0 i
[..]
Route Distinguisher: 65500:2
*> 10.201.0.0/24 2.2.2.3@8< 100 0 i
*> 10.201.1.0/24 2.2.2.3@8< 100 0 i
*> 10.201.2.0/24 2.2.2.3@8< 100 0 i
*> 10.201.3.0/24 2.2.2.3@8< 100 0 i
[..]
Route Distinguisher: 65500:4
*>i10.201.11.0/24 9.9.9.9 100 0 i
*>i10.201.12.0/24 9.9.9.9 100 0 i
*>i10.201.13.0/24 9.9.9.9 100 0 i
*>i10.201.14.0/24 9.9.9.9 100 0 i
[..]
The three emphasized entries are locally exported entries. The 1.1.1.2 Next Hop
attribute is located in the L3VRF customer1. The relationship between the L3VRF
name and the L3VRF identifier displayed (the @7 value) can be done using the
following command:
rt1> show bgp l3vrfs
Type Id routerId #PeersCfg #PeersEstb Name
L3-VNI RouterMAC Interface
DFLT 0 5.5.5.5 1 1 main
0 00:00:00:00:00:00 unknown
VRF 7 1.1.1.1 1 1 customer1
0 00:00:00:00:00:00 unknown
VRF 8 2.2.2.2 1 1 customer2
0 00:00:00:00:00:00 unknown
To get more details about the NLRI information of prefixes, use the following
show command. The exposed 80 label value is the exported value sent to the
remote BGP peers.
rt2> show bgp ipv4 vpn prefix 10.101.0.0/24
BGP routing table entry for 1:55:10.101.0.0/24
not allocated
Paths: (1 available, best #1)
Not advertised to any peer
Local
5.5.5.5 from 5.5.5.5 (5.5.5.5)
Origin IGP, localpref 100, valid, internal, best (First path received)
Extended Community: RT:1:55
Remote label: 80
Last update: Mon Aug 23 12:32:45 2021
Use the below command to check the available entries in the customer1 L3VRF:
rt1> show bgp l3vrf customer1 ipv4
BGP table version is 32, local router ID is 1.1.1.1, vrf id 2
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*>i10.101.0.0/24 1.1.1.2 100 0 i
*>i10.101.1.0/24 1.1.1.2 100 0 i
*>i10.101.2.0/24 1.1.1.2 100 0 i
[..]
*> 10.101.11.0/24 9.9.9.9@0< 100 0 i
*> 10.101.12.0/24 9.9.9.9@0< 100 0 i
*> 10.101.13.0/24 9.9.9.9@0< 100 0 i
[..]
*> 10.101.22.0/24 15.15.15.15@0< 100 0 i
*> 10.101.23.0/24 15.15.15.15@0< 100 0 i
*> 10.101.24.0/24 15.15.15.15@0< 100 0 i
[..]
The three emphasized entries are ipv4-vpn entries which have their underlay
next hop located in a remote site (9.9.9.9 IP address). @0< indicates that
this is a L3VRF route leak going to a remote site. In the above example, only
1.1.1.2 ip address is a locally reachable IP address.
Routing output¶
From the above example, the following output can be extracted from the L3VRF
routing table. As illustrated, the installed route entry performs a double MPLS
encapsulation (20/80). The inner value is the negotiated BGP value. The 20
value is the one negotiated MPLS value on the backbone.
rt1> show ipv4-routes l3vrf customer1
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, A - Babel, D - SHARP, F - PBR, f - OpenFabric,
> - selected route, * - FIB route, q - queued, r - rejected, b - backup
t - trapped, o - offload failure
VRF customer1:
B>* 10.101.0.0/24 [200/0] via 1.1.1.2, eth0_0, 00:27:24
B>* 10.101.1.0/24 [200/0] via 1.1.1.2, eth0_0, 00:27:24
B>* 10.101.2.0/24 [200/0] via 1.1.1.2, eth0_0, 00:27:24
[..]
B> 10.101.11.0/24 [200/0] via 9.9.9.9(vrf vrf0) (recursive), label 80, 00:05:01
* via 6.6.6.3, eth0_0(vrf main), label 20/80, 00:05:01
B> 10.101.12.0/24 [200/0] via 9.9.9.9(vrf vrf0) (recursive), label 80, 00:05:01
* via 6.6.6.3, eth0_0(vrf main), label 20/80, 00:05:01
B> 10.101.13.0/24 [200/0] via 9.9.9.9(vrf vrf0) (recursive), label 80, 00:05:01
* via 6.6.6.3, eth0_0(vrf main), label 20/80, 00:05:01
[..]
B> 10.101.22.0/24 [200/0] via 15.15.15.15(vrf vrf0) (recursive), label 300, 00:07:58
* via 6.6.6.3, eth0_0(vrf main), label 21/300, 00:07:58
B> 10.101.23.0/24 [200/0] via 15.15.15.15(vrf main) (recursive), label 300, 00:07:58
* via 6.6.6.3, eth0_0(vrf main), label 21/300, 00:07:58
B> 10.101.24.0/24 [200/0] via 15.15.15.15(vrf main) (recursive), label 300, 00:07:58
* via 6.6.6.3, eth0_0(vrf main), label 21/300, 00:07:58
[..]
The next command dumps the LFIB of the backbone. The relevant part deals with
incoming traffic from remote VPNs. As the device is a LER, the LDP label is
implicit-null, and incoming traffic comes with the BGP chosen label, here 80
or 81. That traffic is being popped and redirected to the interface representation
of the L3VRF instance.
rt1> show mpls-table
Inbound Label Type Nexthop Outbound Label
-----------------------------------------------
16 LDP 6.6.6.3 implicit-null
17 LDP 6.6.6.3 implicit-null
18 LDP 6.6.6.3 16
19 LDP 6.6.6.3 17
20 LDP 6.6.6.3 18
21 LDP 6.6.6.3 20
22 LDP 6.6.6.3 21
80 BGP customer1 -
81 BGP customer2 -
Interconnection policies options¶
The previous networking configuration is reused to interconnect all A
networks with all B networks. The black arrow illustrates it on the
diagram.
The below configurations show the BGP configuration changes. Usually,
that kind of configuration can be used, when some resources are shared with
the other L3VRFs: A video server located on that shared resource, or a
management L3VRF that is only able to access the other L3VRFs.
rt1
rt1 running config# vrf main
rt1 running vrf main# routing bgp
rt1 running bgp# l3vrf customer1
rt1 running l3vrf customer1# address-family ipv4-unicast
rt1 running ipv4-unicast# del l3vpn import route-target 1:55
rt1 running ipv4-unicast#! l3vpn import route-target 1:55 route-target 2:55
rt1 running ipv4-unicast# maximum-path ebgp 4
rt1 running ipv4-unicast# maximum-path ibgp 4
rt1 running ipv4-unicast# .. ..
rt1 running l3vrf customer1# ..
rt1 running bgp# l3vrf customer2
rt1 running l3vrf customer2# address-family ipv4-unicast
rt1 running ipv4-unicast# del l3vpn import route-target 2:55
rt1 running ipv4-unicast#! l3vpn import route-target 1:55 route-target 2:55
rt1 running ipv4-unicast# .. ..
rt1 running l3vrf customer2# ..
rt1 running bgp#
rt2
rt2 running config# vrf main
rt2 running vrf main# routing bgp
rt2 running bgp# l3vrf customer1
rt2 running l3vrf customer1# address-family ipv4-unicast
rt2 running ipv4-unicast# del l3vpn import route-target 1:55
rt2 running ipv4-unicast#! l3vpn import route-target 1:55 route-target 2:55
rt2 running ipv4-unicast# .. ..
rt2 running l3vrf customer1# ..
rt2 running bgp# l3vrf customer2
rt2 running l3vrf customer2# address-family ipv4-unicast
rt2 running ipv4-unicast# del l3vpn import route-target 2:55
rt2 running ipv4-unicast#! l3vpn import route-target 1:55 route-target 2:55
rt2 running ipv4-unicast# .. ..
rt2 running l3vrf customer2# ..
rt2 running bgp#
rt5
rt5 running config# vrf main
rt5 running vrf main# routing bgp
rt5 running bgp# l3vrf customer1
rt5 running l3vrf customer1# address-family ipv4-unicast
rt5 running ipv4-unicast# del l3vpn import route-target 1:55
rt5 running ipv4-unicast#! l3vpn import route-target 1:55 route-target 2:55
rt5 running ipv4-unicast# .. ..
rt5 running l3vrf customer1# ..
rt5 running bgp#
The following show extract on rt1 dumps the routing entries of A and B
located on rt2. Two different MPLS labels chosen by BGP of rt2, are
received by rt1, for each network. Those labels are visible by executing
the show bgp ipv4 vpn prefix command.
rt1> show bgp ipv4 vpn
BGP table version is 20, local router ID is 9.9.9.9, vrf id 0
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 2:55
[..]
*>i10.101.11.0/24 9.9.9.9 100 0 i
*>i10.101.12.0/24 9.9.9.9 100 0 i
*>i10.101.13.0/24 9.9.9.9 100 0 i
[..]
*>i10.201.11.0/24 9.9.9.9 100 0 i
*>i10.201.12.0/24 9.9.9.9 100 0 i
*>i10.201.13.0/24 9.9.9.9 100 0 i
[..]
rt1> show bgp ipv4 vpn prefix 10.101.11.0/24
BGP routing table entry for 1:55:10.201.11.0/24
not allocated
Paths: (1 available, best #1)
Not advertised to any peer
Local
9.9.9.9 from 9.9.9.9 (9.9.9.9)
Origin IGP, localpref 100, valid, internal, best (First path received)
Extended Community: RT:1:55
Remote label: 80
Last update: Mon Aug 23 14:08:23 2021
rt1> show bgp ipv4 vpn prefix 10.201.11.0/24
BGP routing table entry for 2:55:10.101.11.0/24
not allocated
Paths: (1 available, best #1)
Not advertised to any peer
Local
9.9.9.9 from 9.9.9.9 (9.9.9.9)
Origin IGP, localpref 100, valid, internal, best (First path received)
Extended Community: RT:2:55
Remote label: 80
Last update: Mon Aug 23 14:09:10 2021
When applied to the underlying system, the encapsulation for packets leaving
the customer1 L3VRF contains different labels, depending if the traffic
is headed to remote customer1 or remote customer2.
rt1> show ipv4-routes l3vrf customer1
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, A - Babel, D - SHARP, F - PBR, f - OpenFabric,
> - selected route, * - FIB route, q - queued, r - rejected, b - backup
t - trapped, o - offload failure
VRF customer1:
B> 10.101.11.0/24 [200/0] via 9.9.9.9(vrf main) (recursive), label 80, 00:20:10
* via 6.6.6.3, r1-eth2(vrf main), label 21/80, 00:20:10
B> 10.101.12.0/24 [200/0] via 9.9.9.9(vrf main) (recursive), label 80, 00:20:10
* via 6.6.6.3, r1-eth2(vrf main), label 21/80, 00:20:10
B> 10.101.13.0/24 [200/0] via 9.9.9.9(vrf main) (recursive), label 80, 00:20:10
* via 6.6.6.3, r1-eth2(vrf main), label 21/80, 00:20:10
[..]
B> 10.201.11.0/24 [200/0] via 9.9.9.9(vrf main) (recursive), label 81, 00:20:57
* via 6.6.6.3, r1-eth2(vrf main), label 20/81, 00:20:57
B> 10.201.12.0/24 [200/0] via 9.9.9.9(vrf main) (recursive), label 81, 00:20:57
* via 6.6.6.3, r1-eth2(vrf main), label 20/81, 00:20:57
B> 10.201.13.0/24 [200/0] via 9.9.9.9(vrf main) (recursive), label 81, 00:20:57
* via 6.6.6.3, r1-eth2(vrf main), label 20/81, 00:20:57
rt1> show mpls table
Inbound Outbound
Label Type Nexthop Label
-------- ------- --------------- --------
80 BGP r1-cust1
81 BGP r1-cust2
82 BGP 6.6.6.3 18
83 BGP 6.6.6.3 18
84 BGP 6.6.6.3 18
85 BGP 6.6.6.3 18
An additional specificity of the setup is the possibility to import ECMP
entries coming from 2 separate locations; here, some 32-bit host routes are
retrieved. Some of the entries stand for a server with the same IP, but
geographically at 2 different places, namely rt2 and rt5. This kind of
scenario can be used for servers that require availability, or where the
load is split in two, to avoid starvation of the resources of one of the
machines.
rt1> show bgp ipv4 vpn
BGP table version is 20, local router ID is 9.9.9.9, vrf id 0
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*=i10.101.18.10/32 15.15.15.15 100 0 i
*>i 9.9.9.9 100 0 i
*=i10.101.19.10/32 15.15.15.15 100 0 i
*>i 9.9.9.9 100 0 i
*=i10.101.20.10/32 15.15.15.15 100 0 i
*>i 9.9.9.9 100 0 i
rt1> show ipv4-routes l3vrf customer1
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, A - Babel, D - SHARP, F - PBR, f - OpenFabric,
> - selected route, * - FIB route, q - queued, r - rejected, b - backup
t - trapped, o - offload failure
VRF customer1:
B> 10.101.18.10/32 [200/0] via 15.15.15.15(vrf main) (recursive), label 300, 00:20:10
* via 6.6.6.3, r1-eth2(vrf main), label 21/300, 00:20:10
via 9.9.9.9(vrf main) (recursive), label 80, 00:20:10
* via 6.6.6.3, r1-eth2(vrf main), label 20/80, 00:20:10
B> 10.101.19.10/32 [200/0] via 15.15.15.15(vrf main) (recursive), label 300, 00:20:10
* via 6.6.6.3, r1-eth2(vrf main), label 21/300, 00:20:10
via 9.9.9.9(vrf main) (recursive), label 80, 00:20:10
* via 6.6.6.3, r1-eth2(vrf main), label 20/80, 00:20:10
B> 10.101.20.10/32 [200/0] via 15.15.15.15(vrf main) (recursive), label 300, 00:20:10
* via 6.6.6.3, r1-eth2(vrf main), label 21/300, 00:20:10
via 9.9.9.9(vrf main) (recursive), label 80, 00:20:10
* via 6.6.6.3, r1-eth2(vrf main), label 20/80, 00:20:10
Interconnect traffic between direct connections¶
Having an MPLS backbone requires using the LDP service. It may require increasing the configuration on the local devices, but also on the whole interconnection network devices. One solution to simplify the configuration and to not rely on any external dependencies is to reduce to 1 hop the distance between the two devices that should interconnect. The following use cases can apply:
device acting as ASBR, and directly connecting to remote ASBR. In that case, eBGP is used.
a GRE tunnel is used to interconnect two devices. As GRE tunnel is a point-to-point technology, then the traffic flowing across that interface is reduced to 1 hop only, independently of the framework to be crossed.
The below examples illustrate how the MPLS configuration can be simplified when having to interconnect L3VPN networks over a GRE interface.
Using LDP as MPLS backbone¶
The below configuration illustrates the configuration of the two interconnected devices using LDP. The configuration does not include the VPN configuration part.
rt1
rt1 running config# vrf main
rt1 running vrf main# interface
rt1 running interface# loopback loop1
rt1 running loopback loop1# ipv4 address 5.5.5.5/32
rt1 running loopback loop1# ..
rt1 running interface# physical eth1
rt1 running physical eth1#! port pci-b0s4
rt1 running physical eth1# ipv4 address 10.1.2.5/24
rt1 running physical eth1# ..
rt1 running interface# gre gre1
rt1 running gre gre1#! link-interface eth1
rt1 running gre gre1#! ipv4 address 40.0.0.5/24
rt1 running gre gre1#! local 10.1.2.5
rt1 running gre gre1# remote 10.3.2.9
rt1 running gre gre1# ..
rt1 running interface# ..
rt1 running vrf main# routing ospf
rt1 running ospf# router-id 10.1.2.5
rt1 running ospf# network 10.1.2.0/24 area 0.0.0.0
rt1 running ospf# ..
rt1 running routing# ..
rt1 running vrf main# routing static
rt1 running static# ipv4-route 9.9.9.9/32 next-hop 40.0.0.9
rt1 running static# ..
rt1 running routing# ..
rt1 running vrf main# routing mpls ldp
rt1 running ldp# router-id 5.5.5.5
rt1 running ldp# address-family ipv4
rt1 running ipv4#! discovery transport-address 5.5.5.5
rt1 running ipv4# interface gre1
rt1 running interface gre1# ..
rt1 running ipv4# ..
rt1 running address-family# .. ..
rt1 running mpls# ..
rt1 running routing# bgp
rt1 running bgp# router-id 5.5.5.5
rt1 running bgp# as 65500
rt1 running bgp# neighbor 9.9.9.9 remote-as 65500
rt1 running bgp# neighbor 9.9.9.9 update-source 5.5.5.5
rt1 running bgp# neighbor 9.9.9.9 address-family ipv4-unicast enabled false
rt1 running bgp# neighbor 9.9.9.9 address-family ipv4-vpn enabled true
rt1 running bgp# .. ..
rt1 running vrf main# l3vrf customer1
rt1 running l3vrf customer1#! table-id 10
rt1 running l3vrf customer1# routing bgp
rt1 running bgp# router-id 5.5.5.5
rt1 running bgp# as 65500
rt1 running bgp# # append bgp configuration for customer1 : l3vpn, ...
rt1 running bgp#
rt2
rt2 running config# vrf main
rt2 running vrf main# interface
rt2 running interface# loopback loop1
rt2 running loopback loop1# ipv4 address 9.9.9.9/32
rt2 running loopback loop1# ..
rt2 running interface# physical eth1
rt2 running physical eth1#! port pci-b0s4
rt2 running physical eth1# ipv4 address 10.3.2.9/24
rt2 running physical eth1# ..
rt2 running interface# gre gre1
rt2 running gre gre1#! link-interface eth1
rt2 running gre gre1#! ipv4 address 40.0.0.9/24
rt2 running gre gre1#! local 10.3.2.9
rt2 running gre gre1# remote 10.1.2.5
rt2 running gre gre1# ..
rt2 running interface# ..
rt2 running vrf main# routing ospf
rt2 running ospf# router-id 10.3.2.9
rt2 running ospf# network 10.3.2.0/24 area 0.0.0.0
rt2 running ospf# ..
rt2 running routing# ..
rt2 running vrf main# routing static
rt2 running static# ipv4-route 5.5.5.5/32 next-hop 40.0.0.5
rt2 running static# ..
rt2 running routing# ..
rt2 running vrf main# routing mpls ldp
rt2 running ldp# router-id 9.9.9.9
rt2 running ldp# address-family ipv4
rt2 running ipv4#! discovery transport-address 9.9.9.9
rt2 running ipv4# interface gre1
rt2 running interface gre1# ..
rt2 running ipv4# ..
rt2 running address-family# ..
rt2 running ldp# .. ..
rt2 running routing# bgp
rt2 running bgp#! router-id 9.9.9.9
rt2 running bgp#! as 65500
rt2 running bgp# neighbor 5.5.5.5 remote-as 65500
rt2 running bgp# neighbor 5.5.5.5 update-source 9.9.9.9
rt2 running bgp# neighbor 5.5.5.5 address-family ipv4-unicast enabled false
rt2 running bgp# neighbor 5.5.5.5 address-family ipv4-vpn enabled true
rt2 running bgp# .. ..
rt2 running vrf main# l3vrf customer2
rt2 running l3vrf customer2#! table-id 10
rt2 running l3vrf customer2# routing bgp
rt2 running bgp# router-id 9.9.9.9
rt2 running bgp# as 65500
rt2 running bgp# # append bgp configuration for customer1 : l3vpn, ...
rt2 running bgp#
As shown below, an implicit-null label has been bound to the route to the
next hop of the remote BGP speaker. You can note that the 40.0.0.9 IP
address is the gateway used to reach the remote 9.9.9.9 IP address.
rt1> show ipv4-routing
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, A - Babel, D - SHARP, F - PBR, f - OpenFabric,
> - selected route, * - FIB route, q - queued, r - rejected, b - backup
t - trapped, o - offload failure
VRF default:
[..]
C>* 5.5.5.5/32 is directly connected, loop1, 00:02:21
S>* 9.9.9.9/32 [1/0] via 40.0.0.9, gre1, label implicit-null, weight 1, 00:00:19
C>* 40.0.0.0/24 is directly connected, gre1, 00:02:21
[..]
By adding the VPN configuration which is not present on the above configuration, one will have two VPN labels negotiated between BGP independently of LDP.
rt1> show bgp ipv4 vpn
BGP table version is 2, local router ID is 5.5.5.5, vrf id 0
Default local pref 100, local AS 65500
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 65500:5
*> 172.16.1.0/24 0.0.0.0@6< 0 32768 ?
[..]
Route Distinguisher: 65500:9
*>i172.16.2.0/24 9.9.9.9 0 100 0 ?
[..]
The below show ipv4-routes command gives the information on the encapsulation used
by a packet going from the local L3VRF customer1 to the remote L3VRF customer2.
Outgoing packets leaving the rt1 device are encapsulated with the 144/implicit-null
labels. The additional implicit-null label is chosen since it stands for the label
to use when wanting to reach the nexthop of the VPN route.
Actually, 9.9.9.9 is reachable via 40.0.0.9 by using the implicit-null label.
rt1> show ipv4-routes l3vrf customer1
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, A - Babel, D - SHARP, F - PBR, f - OpenFabric,
> - selected route, * - FIB route, q - queued, r - rejected, b - backup
t - trapped, o - offload failure
VRF customer1:
C>* 172.16.1.0/24 is directly connected, r1-eth0, 00:11:15
B> 172.16.2.0/24 [20/0] via 9.9.9.9 (vrf default) (recursive), label 80, weight 1, 00:15:29
* via 40.0.0.9, gre1 (vrf default), label implicit-null/80, weight 1, 00:15:29
Convey VPN labels coming from eBGP over a connected interface¶
When an inter-AS BGP peering between connected peers is formed, forging an
MPLS backbone to transport VPN labels is not always feasible. When peering
with a connected eBGP peer, the incoming BGP L3VPN updates have the next
hop resolved over the connected routes of the interface. To authorize those
MPLS routes to be installed on the FIB, the following command under the
routing interface context is enabled.
vsr running config# vrf main
vsr running vrf main# interface physical eth2
vsr running physical eth2#! port pci-b0s5
vsr running physical eth2# ipv4-address 10.125.0.1
vsr running physical eth2# / vrf main routing interface eth2
vsr running interface eth2# bgp mpls accept-connected-ebgp-routes true
vsr running interface eth2# ..
vsr running routing# bgp
vsr running bgp#! as 65500
vsr running bgp# router-id 10.125.0.1
vsr running bgp# neighbor 10.125.0.2 remote-as 65502
vsr running bgp# neighbor 10.125.0.2 address-family ipv4-vpn enabled true
vsr running bgp#
The BGP routes received from the eBGP neighbor are installed, without requiring any label distribution protocol.
Note
The value of this parameter is ignored when using any label distribution protocol other than BGP over that interface.
Note
The value of this parameter is effective only for devices acting both as ASBR and as PE. Because the latter device has L3VRF BGP instances, VPN routes are imported locally and can be installed.
See also
RFC 4364 section
10.busing theconnectedapproach: when the configured ASBR has no L3VRF instances, labeled VPN routes must be redistributed between the PEs located in the interior domain, and the facing ASBR. Because the next-hop needs to be modified along with the incoming label value, theswitch-l3vpn-labelscommand under therouting interface bgpnode helps in binding a new label value to the VPN route’s original label value. The new label value is redistributed, and return traffic is switched to the original label value. More information: Redistribute L3VPN traffic between ASBRs from 2 domains.RFC 4364 section
10.cusing thelabeled-unicastaddress family. The labeled BGP VPN routes are carried using the BGP labeled unicast address family. More information: L3VPN service over EBGP.
Using Static Label configuration¶
The below configuration proposes to replace the LDP configuration with a static
route that contains a labeled next hop. The MPLS label implicit-null which
corresponds to the value 3 for ipv4 traffic.
rt1
rt1 running config# vrf main
rt1 running vrf main# interface loopback loop1
rt1 running loopback loop1# ipv4 address 5.5.5.5/32
rt1 running loopback loop1# ..
rt1 running interface# physical eth1
rt1 running physical eth1#! port pci-b0s4
rt1 running physical eth1# ipv4 address 10.1.2.5/24
rt1 running physical eth1# ..
rt1 running interface# gre gre1
rt1 running gre gre1#! link-interface eth1
rt1 running gre gre1#! ipv4 address 40.0.0.5/24
rt1 running gre gre1#! local 10.1.2.5
rt1 running gre gre1# remote 10.3.2.9
rt1 running gre gre1# ..
rt1 running interface# ..
rt1 running vrf main# routing ospf
rt1 running ospf# router-id 10.1.2.5
rt1 running ospf# network 10.1.2.0/24 area 0.0.0.0
rt1 running ospf# ..
rt1 running routing# ..
rt1 running vrf main# routing static
rt1 running static# ipv4-route 9.9.9.9/32 next-hop 40.0.0.9%gre1 mpls-label implicit-null
rt1 running static#
rt1 running static# ..
rt1 running routing# bgp
rt1 running bgp#! router-id 5.5.5.5
rt1 running bgp#! as 65500
rt1 running bgp# neighbor 9.9.9.9 remote-as 65500
rt1 running bgp# neighbor 9.9.9.9 update-source 5.5.5.5
rt1 running bgp# neighbor 9.9.9.9 address-family ipv4-unicast enabled false
rt1 running bgp# neighbor 9.9.9.9 address-family ipv4-vpn enabled true
rt1 running bgp# .. ..
rt1 running vrf main# l3vrf customer1
rt1 running l3vrf customer1#! table-id 10
rt1 running l3vrf customer1# routing bgp
rt1 running bgp# router-id 5.5.5.5
rt1 running bgp# as 65500
rt1 running bgp# # append bgp configuration for customer1 : l3vpn, ...
rt1 running bgp#
rt2
rt2 running config# vrf main
rt2 running vrf main# interface loopback loop1
rt2 running loopback loop1# ipv4 address 9.9.9.9/32
rt2 running loopback loop1# ..
rt2 running interface# physical eth1
rt2 running physical eth1#! port pci-b0s4
rt2 running physical eth1# ipv4 address 10.3.2.9/24
rt2 running physical eth1# ..
rt2 running interface# gre gre1
rt2 running gre gre1#! link-interface eth1
rt2 running gre gre1#! ipv4 address 40.0.0.9/24
rt2 running gre gre1#! local 10.3.2.9
rt2 running gre gre1# remote 10.1.2.5
rt2 running gre gre1# ..
rt2 running interface# ..
rt2 running vrf main# routing ospf
rt2 running ospf# router-id 10.3.2.9
rt2 running ospf# network 10.3.2.0/24 area 0.0.0.0
rt2 running ospf# ..
rt2 running routing# ..
rt2 running vrf main# routing static
rt2 running static# ipv4-route 5.5.5.5/32 next-hop 40.0.0.5%gre1 mpls-label implicit-null
rt2 running static# ..
rt2 running routing# bgp
rt2 running bgp#! router-id 5.5.5.5
rt2 running bgp#! as 65500
rt2 running bgp# neighbor 9.9.9.9 remote-as 65500
rt2 running bgp# neighbor 9.9.9.9 update-source 5.5.5.5
rt2 running bgp# neighbor 9.9.9.9 address-family ipv4-unicast enabled false
rt2 running bgp# neighbor 9.9.9.9 address-family ipv4-vpn enabled true
rt2 running bgp# .. ..
rt2 running vrf main# l3vrf customer1
rt2 running l3vrf customer1#! table-id 10
rt2 running l3vrf customer1# routing bgp
rt2 running bgp# router-id 5.5.5.5
rt2 running bgp# as 65500
rt2 running bgp# # append bgp configuration for customer1 : l3vpn, ...
rt2 running bgp#
The above configuration results in having an mpls based route to reach the
40.0.0.9 BGP next hop. When importing the L3VRF entry in the customer1
L3VRF, the VPN label is appended with the implicit-null label. The show
commands from above example are the same with or without LDP.
Relaxing Labelled path over GRE interfaces¶
The below configuration proposes to not configure any MPLS backbone. This mechanism works only if the BGP next hop entry selects a route that uses a GRE tunnel as outgoing interface. In those conditions, the L3VPN entries will not need the next hop to select a labelled path.
This setup is possible by using a specific route-map command called
set l3vpn next-hop encapsulation gre. Because this command is available, and
because the calculated path to install the BGP entry to is a GRE tunnel,
then the installation is authorized.
rt1
rt1 running config# routing
rt1 running routing# route-map rmap
rt1 running route-map rmap#! seq 1
rt1 running seq 1#! policy permit
rt1 running seq 1# set l3vpn next-hop encapsulation gre
rt1 running seq 1# ..
rt1 running route-map rmap# ..
rt1 running routing# ..
rt1 running config# vrf main
rt1 running vrf main# routing bgp
rt1 running bgp#! router-id 5.5.5.5
rt1 running bgp#! as 65500
rt1 running bgp# neighbor 9.9.9.9 remote-as 65500
rt1 running bgp# neighbor 9.9.9.9 update-source 5.5.5.5
rt1 running bgp# neighbor 9.9.9.9 address-family ipv4-unicast enabled false
rt1 running bgp# neighbor 9.9.9.9 address-family ipv4-vpn enabled true
rt1 running bgp# neighbor 9.9.9.9 address-family ipv4-vpn route-map in route-map-name rmap
rt1 running bgp# .. ..
rt1 running vrf main# interface loopback loop1
rt1 running loopback loop1# ipv4 address 5.5.5.5/32
rt1 running loopback loop1# ..
rt1 running interface# physical eth1
rt1 running physical eth1#! port pci-b0s4
rt1 running physical eth1# ipv4 address 10.1.2.5/24
rt1 running physical eth1# ..
rt1 running interface# gre gre1
rt1 running gre gre1#! link-interface eth1
rt1 running gre gre1#! ipv4 address 40.0.0.5/24
rt1 running gre gre1#! local 10.1.2.5
rt1 running gre gre1# remote 10.3.2.9
rt1 running gre gre1# ..
rt1 running interface# ..
rt1 running vrf main# routing ospf
rt1 running ospf# router-id 10.1.2.5
rt1 running ospf# network 10.1.2.0/24 area 0.0.0.0
rt1 running ospf# ..
rt1 running routing# ..
rt1 running vrf main# routing static
rt1 running static# ipv4-route 9.9.9.9/32 next-hop 40.0.0.9%gre1
rt1 running static# ..
rt1 running routing# ..
rt1 running vrf main# l3vrf customer1
rt1 running l3vrf customer1#! table-id 10
rt1 running l3vrf customer1# routing bgp
rt1 running bgp# router-id 5.5.5.5
rt1 running bgp# as 65500
rt1 running bgp# # append bgp configuration for customer1 : l3vpn, ...
rt1 running bgp# ..
rt1 running routing# ..
rt1 running l3vrf customer1#
rt2
rt2 running config# routing
rt2 running routing# route-map rmap
rt2 running route-map rmap#! seq 1
rt2 running seq 1#! policy permit
rt2 running seq 1# set l3vpn next-hop encapsulation gre
rt2 running seq 1# ..
rt2 running route-map rmap# ..
rt2 running routing# ..
rt2 running config# vrf main
rt2 running vrf main# routing bgp
rt2 running bgp#! router-id 9.9.9.9
rt2 running bgp#! as 65500
rt2 running bgp# neighbor 5.5.5.5 remote-as 65500
rt2 running bgp# neighbor 5.5.5.5 update-source 9.9.9.9
rt2 running bgp# neighbor 5.5.5.5 address-family ipv4-unicast enabled false
rt2 running bgp# neighbor 5.5.5.5 address-family ipv4-vpn enabled true
rt2 running bgp# neighbor 5.5.5.5 address-family ipv4-vpn route-map in route-map-name rmap
rt2 running bgp# .. ..
rt2 running vrf main# interface
rt2 running interface# loopback loop1
rt2 running loopback loop1# ipv4 address 9.9.9.9/32
rt2 running loopback loop1# ..
rt2 running interface# physical eth1
rt2 running physical eth1#! port pci-b0s4
rt2 running physical eth1# ipv4 address 10.3.2.9/24
rt2 running physical eth1# ..
rt2 running interface# gre gre1
rt2 running gre gre1#! link-interface eth1
rt2 running gre gre1#! ipv4 address 40.0.0.9/24
rt2 running gre gre1#! local 10.3.2.9
rt2 running gre gre1# remote 10.1.2.5
rt2 running gre gre1# ..
rt2 running interface# ..
rt2 running vrf main# routing ospf
rt2 running ospf# router-id 10.3.2.9
rt2 running ospf# network 10.3.2.0/24 area 0.0.0.0
rt2 running ospf# ..
rt2 running routing# ..
rt2 running vrf main# routing static
rt2 running static# ipv4-route 5.5.5.5/32 next-hop 40.0.0.5%gre1
rt2 running static# ..
rt2 running routing# ..
rt2 running vrf main# l3vrf customer2
rt2 running l3vrf customer2#! table-id 10
rt2 running l3vrf customer2# routing bgp
rt2 running bgp# router-id 9.9.9.9
rt2 running bgp# as 65500
rt2 running bgp# # append bgp configuration for customer2 : l3vpn, ...
rt2 running bgp# ..
rt2 running routing#
Redistribute L3VPN traffic between ASBRs from 2 domains¶
Generally, an ASBR configuration only redistributes traffic between VPNs located
on neighbor devices. The Virtual Service Router can receive L3VPN updates. Then it can redistribute
these updates either when peering with an iBGP peer by using the nexthop-self
command, or when peering with an eBGP peer. In that case, the MPLS label value from
incoming updates must be modified, and redistributed with a new label value. The
switch-l3vpn-labels command must be configured on the interfaces where the L3VPN
traffic has to be sent. The command will create an MPLS entry to swap the new label
value with the original one, and send the MPLS traffic to the original BGP speaker.
The drawing below illustrates a setup with two distinct domains connected using two facing ASBRs. In this setup, only the PEs internally connected to the ASBRs have L3VPN services available. The ASBR responsibility is to redistribute L3VPN updates and L3VPN traffic between the internal and the external domains.
BGP setup with two facing ASBRs redistributing L3VPN traffic.¶
The configuration of the four devices is given below:
pe1
pe1 running config# / vrf main l3vrf L3VRF1 table-id 10
pe1 running config# / vrf main l3vrf L3VRF1 interface physical eth1 port pci-b0s4
pe1 running config# / vrf main l3vrf L3VRF1 interface physical eth1 ipv4 address 172.16.0.254/24
pe1 running config# / vrf main l3vrf L3VRF1 routing bgp address-family ipv4-unicast l3vpn export vpn true
pe1 running config#! / vrf main l3vrf L3VRF1 routing bgp address-family ipv4-unicast l3vpn export label auto
pe1 running config#! / vrf main l3vrf L3VRF1 routing bgp address-family ipv4-unicast l3vpn export route-target 64696:0
pe1 running config#! / vrf main l3vrf L3VRF1 routing bgp address-family ipv4-unicast l3vpn export route-distinguisher 64696:1
pe1 running config#! / vrf main l3vrf L3VRF1 routing bgp address-family ipv4-unicast l3vpn import vpn true
pe1 running config#! / vrf main l3vrf L3VRF1 routing bgp address-family ipv4-unicast l3vpn import route-target 64500:0
pe1 running config#! / vrf main l3vrf L3VRF1 routing bgp address-family ipv4-unicast l3vpn import route-target 64696:0
pe1 running config#! / vrf main l3vrf L3VRF1 routing bgp address-family ipv4-unicast redistribute connected
pe1 running config#! / vrf main interface physical eth2 port pci-b0s5
pe1 running config#! / vrf main interface physical eth2 ipv4 address 10.0.0.1/24
pe1 running config#! / vrf main interface loopback lo1 ipv4 address 192.168.2.1/32
pe1 running config#! / vrf main routing bgp as 64696
pe1 running config# / vrf main routing bgp router-id 192.168.2.1
pe1 running config# / vrf main routing bgp address-family ipv4-unicast network 192.168.2.1/32
pe1 running network 192.168.2.1/32# / vrf main routing bgp neighbor 10.0.0.2 remote-as internal
pe1 running network 192.168.2.1/32# / vrf main routing bgp neighbor 10.0.0.2 address-family ipv4-unicast enabled false
pe1 running network 192.168.2.1/32# / vrf main routing bgp neighbor 10.0.0.2 address-family ipv4-labeled-unicast
pe1 running ipv4-labeled-unicast# / vrf main routing bgp neighbor 192.168.2.2 remote-as internal
pe1 running ipv4-labeled-unicast# / vrf main routing bgp neighbor 192.168.2.2 update-source 192.168.2.1
pe1 running ipv4-labeled-unicast# / vrf main routing bgp neighbor 192.168.2.2 address-family ipv4-unicast enabled false
pe1 running ipv4-labeled-unicast# / vrf main routing bgp neighbor 192.168.2.2 address-family ipv4-vpn
asbr1
asbr1 running config# / vrf main interface physical eth1 port pci-b0s4
asbr1 running config# / vrf main interface physical eth1 ipv4 address 10.0.1.2/24
asbr1 running config# / vrf main interface physical eth2 port pci-b0s5
asbr1 running config# / vrf main interface physical eth2 ipv4 address 10.0.0.2/24
asbr1 running config# / vrf main interface loopback lo1 ipv4 address 192.168.2.2/32
asbr1 running config# / vrf main routing interface eth1 bgp mpls switch-l3vpn-labels true
asbr1 running config# / vrf main routing interface eth2 bgp mpls switch-l3vpn-labels true
asbr1 running config# / vrf main routing bgp as 64696
asbr1 running config# / vrf main routing bgp router-id 192.168.2.2
asbr1 running config# / vrf main routing bgp ebgp-requires-policy false
asbr1 running config# / vrf main routing bgp address-family ipv4-unicast network 192.168.2.2/32
asbr1 running network 192.168.2.2/32# / vrf main routing bgp neighbor 10.0.0.1 remote-as internal
asbr1 running network 192.168.2.2/32# / vrf main routing bgp neighbor 10.0.0.1 address-family ipv4-unicast enabled false
asbr1 running network 192.168.2.2/32# / vrf main routing bgp neighbor 10.0.0.1 address-family ipv4-labeled-unicast
asbr1 running ipv4-labeled-unicast# / vrf main routing bgp neighbor 10.0.1.3 remote-as external
asbr1 running ipv4-labeled-unicast# / vrf main routing bgp neighbor 10.0.1.3 address-family ipv4-unicast enabled false
asbr1 running ipv4-labeled-unicast# / vrf main routing bgp neighbor 10.0.1.3 address-family ipv4-vpn
asbr1 running ipv4-vpn# / vrf main routing bgp neighbor 192.168.2.1 remote-as internal
asbr1 running ipv4-vpn# / vrf main routing bgp neighbor 192.168.2.1 update-source lo1
asbr1 running ipv4-vpn# / vrf main routing bgp neighbor 192.168.2.1 address-family ipv4-unicast enabled false
asbr1 running ipv4-vpn# / vrf main routing bgp neighbor 192.168.2.1 address-family ipv4-vpn nexthop-self
asbr2
asbr2 running config# / vrf main interface physical eth1 port pci-b0s4
asbr2 running config# / vrf main interface physical eth1 ipv4 address 10.0.1.3/24
asbr2 running config# / vrf main interface physical eth2 port pci-b0s5
asbr2 running config# / vrf main interface physical eth2 ipv4 address 10.0.2.3/24
asbr2 running config# / vrf main interface loopback lo1 ipv4 address 192.168.2.3/32
asbr2 running config# / vrf main routing interface eth1 bgp mpls switch-l3vpn-labels true
asbr2 running config# / vrf main routing interface eth2 bgp mpls switch-l3vpn-labels true
asbr2 running config# / vrf main routing bgp as 64500
asbr2 running config# / vrf main routing bgp router-id 192.168.2.3
asbr2 running config# / vrf main routing bgp ebgp-requires-policy false
asbr2 running config# / vrf main routing bgp address-family ipv4-unicast network 192.168.2.3/32
asbr2 running network 192.168.2.3/32# / vrf main routing bgp neighbor 10.0.1.2 remote-as external
asbr2 running network 192.168.2.3/32# / vrf main routing bgp neighbor 10.0.1.2 address-family ipv4-unicast enabled false
asbr2 running network 192.168.2.3/32# / vrf main routing bgp neighbor 10.0.1.2 address-family ipv4-vpn
asbr2 running ipv4-vpn# / vrf main routing bgp neighbor 10.0.2.4 remote-as internal
asbr2 running ipv4-vpn# / vrf main routing bgp neighbor 10.0.2.4 address-family ipv4-unicast enabled false
asbr2 running ipv4-vpn# / vrf main routing bgp neighbor 10.0.2.4 address-family ipv4-labeled-unicast
asbr2 running ipv4-labeled-unicast# / vrf main routing bgp neighbor 192.168.2.4 remote-as internal
asbr2 running ipv4-labeled-unicast# / vrf main routing bgp neighbor 192.168.2.4 update-source lo1
asbr2 running ipv4-labeled-unicast# / vrf main routing bgp neighbor 192.168.2.4 address-family ipv4-unicast enabled false
asbr2 running ipv4-labeled-unicast# / vrf main routing bgp neighbor 192.168.2.4 address-family ipv4-vpn nexthop-self
pe2
pe2 running config# / vrf main l3vrf L3VRF1 table-id 10
pe2 running config# / vrf main l3vrf L3VRF1 interface physical eth1 port pci-b0s4
pe2 running config# / vrf main l3vrf L3VRF1 interface physical eth1 ipv4 address 172.16.128.254/24
pe2 running config# / vrf main l3vrf L3VRF1 routing bgp address-family ipv4-unicast l3vpn export vpn true
pe2 running config#! / vrf main l3vrf L3VRF1 routing bgp address-family ipv4-unicast l3vpn export label 200
pe2 running config#! / vrf main l3vrf L3VRF1 routing bgp address-family ipv4-unicast l3vpn export route-target 64500:0
pe2 running config#! / vrf main l3vrf L3VRF1 routing bgp address-family ipv4-unicast l3vpn export route-distinguisher 64500:1
pe2 running config#! / vrf main l3vrf L3VRF1 routing bgp address-family ipv4-unicast l3vpn import vpn true
pe2 running config#! / vrf main l3vrf L3VRF1 routing bgp address-family ipv4-unicast l3vpn import route-target 64500:0
pe2 running config#! / vrf main l3vrf L3VRF1 routing bgp address-family ipv4-unicast l3vpn import route-target 64696:0
pe2 running config#! / vrf main l3vrf L3VRF1 routing bgp address-family ipv4-unicast redistribute connected
pe2 running config#! / vrf main interface physical eth2 port pci-b0s5
pe2 running config#! / vrf main interface physical eth2 ipv4 address 10.0.2.4/24
pe2 running config#! / vrf main interface loopback lo1 ipv4 address 192.168.2.4/32
pe2 running config#! / vrf main routing bgp as 64500
pe2 running config# / vrf main routing bgp router-id 192.168.2.4
pe2 running config# / vrf main routing bgp address-family ipv4-unicast network 192.168.2.4/32
pe2 running network 192.168.2.4/32# / vrf main routing bgp neighbor 10.0.2.3 remote-as internal
pe2 running network 192.168.2.4/32# / vrf main routing bgp neighbor 10.0.2.3 address-family ipv4-unicast enabled false
pe2 running network 192.168.2.4/32# / vrf main routing bgp neighbor 10.0.2.3 address-family ipv4-labeled-unicast
pe2 running ipv4-labeled-unicast# / vrf main routing bgp neighbor 192.168.2.3 remote-as internal
pe2 running ipv4-labeled-unicast# / vrf main routing bgp neighbor 192.168.2.3 update-source 192.168.2.1
pe2 running ipv4-labeled-unicast# / vrf main routing bgp neighbor 192.168.2.3 address-family ipv4-unicast enabled false
pe2 running ipv4-labeled-unicast# / vrf main routing bgp neighbor 192.168.2.3 address-family ipv4-vpn
As can be seen, the 172.16.128.0/24 network is redistributed by the pe2 device
with the 200 label value. The BGP update is received on asbr2:
asbr2> show bgp ipv4 vpn prefix 172.16.128.0/24
BGP routing table entry for 64500:1:172.16.128.0/24, version 1
not allocated
Paths: (1 available, best #1)
Advertised to non peer-group peers:
10.0.1.2
Local
192.168.2.4 from 192.168.2.4 (192.168.2.4)
Origin incomplete, metric 0, localpref 100, valid, internal, best (First path received)
Extended Community: RT:64500:0
Remote label: 200
Last update: Tue Jul 9 14:56:45 2024
An advertisement of the 172.16.128.0/24 network is sent to the asbr1 eBGP device.
The label value is changed, along with the next-hop value, as can be seen on the asbr1
device:
asbr1> show bgp ipv4 vpn prefix 172.16.128.0/24
BGP routing table entry for 64500:1:172.16.128.0/24, version 1
not allocated
Paths: (1 available, best #1)
Advertised to non peer-group peers:
10.0.1.3 192.168.2.1
64500
10.0.1.3 from 10.0.1.3 (192.168.2.3)
Origin incomplete, valid, external, best (First path received)
Extended Community: RT:64500:0
Remote label: 18
Last update: Tue Jul 9 14:56:44 2024
The 18 label value has been allocated by the asbr2 device, and used as
MPLS entry to handle the return traffic. The 18 label value is swapped
and replaces by the 3/200 label stack. The 3 label value is the labeled
unicast value provided by pe2, as the labeled unicast labels are used as
label transport.
asbr2> show mpls table 18
Local label: 18 (installed)
type: BGP remote label: 3/200 distance: 20
via 10.0.2.4 dev eth2 (installed)
The same routing mechanism applies to the asbr1 and pe1 devices which
advertise the 172.16.0.0/24 network.
asbr2> show bgp ipv4 vpn prefix 172.16.0.0/24
BGP routing table entry for 64696:1:172.16.0.0/24, version 1
not allocated
Paths: (1 available, best #1)
Advertised to non peer-group peers:
10.0.1.2 192.168.2.4
64696
10.0.1.2 from 10.0.1.2 (192.168.2.2)
Origin incomplete, valid, external, best (First path received)
Extended Community: RT:64696:0
Remote label: 17
Last update: Tue Jul 9 14:56:37 2024
The asbr2 device will redistribute this prefix with a new local label to the
pe2 device, as the nexthop-self option has been used.
pe2> show bgp ipv4 vpn prefix 172.16.0.0/24
BGP routing table entry for 64696:1:172.16.0.0/24, version 1
not allocated
Paths: (1 available, best #1)
Not advertised to any peer
64696
192.168.2.3 from 192.168.2.3 (192.168.2.3)
Origin incomplete, localpref 100, valid, internal, best (First path received)
Extended Community: RT:64696:0
Remote label: 16
Last update: Tue Jul 9 14:56:45 2024
The traffic from pe2 to pe1 is switched on the asbr2 device.
asbr2> show mpls table 17
Local label: 17 (installed)
type: BGP remote label: 3 distance: 20
via 10.0.2.4 dev eth2 (installed)
The below show command dumps the L3VPN allocated label values when receiving
incoming L3VPN updates. Each label value is bound to an incoming label value,
and an incoming next hop value.
asbr2> show bgp l3vpn label-incoming-nexthop-cache detail
Current BGP mpls-vpn nexthop label bind cache, VRF default
10.0.1.2, label 17, local label 16 #paths 1
interface eth1
Last update: Tue Jul 9 14:56:37 2024
Paths:
1/3 172.16.0.0/24 VRF default flags 0x20418
192.168.2.4, label 200, local label 18 #paths 1
interface eth2
Last update: Tue Jul 9 14:56:45 2024
Paths:
1/3 172.16.128.0/24 VRF default flags 0x20418
Interconnect IPv6 L3VRF instances over an IPv4 MPLS network (6vpe)¶
6vpe stands for IPv6 on VPN to Provider Edge Router, and defines an approach
derived from the 6pe approach, where a mapping between a given IPv4 address and
the IPv4 mapped IPv6 address is done. Contrary to 6pe, it applies to L3VRFs:
from the control plane perspective, the BGP ipv6-vpn address family is used,
instead of the ipv6-labeled-unicast address family. From the data plane perspective,
the IPv6 traffic is encapsulated with the service VPN label; the recursive IPv6
route uses an IPv4 mapped IPv6 address as next-hop, which is derived from the
IPv4 next hop IP address calculated by the IGP.
See also
The following example illustrates two PEs devices connected to an MPLS IPv4
based network. The rt1 and rt4 devices are dual stack based, and are able to
convey IPv6 VPN traffic over the existing IPv4 network.
iBGP setup connecting two IPv6 L3VRFs over an IPv4 MPLS backbone¶
rt1
rt1 running config# vrf main l3vrf l3vrf1
rt1 running l3vrf l3vrf1#! table-id 10
rt1 running l3vrf l3vrf1# interface physical eth2
rt1 running physical eth2#! port pci-b0s5
rt1 running physical eth2# ipv6 address fd00:100::1/64
rt1 running physical eth2# .. .. routing bgp
rt1 running bgp#! as 65500
rt1 running bgp#! router-id 198.51.100.1
rt1 running bgp#! address-family ipv6-unicast
rt1 running ipv6-unicast#! network fd00:100::/64
rt1 running network fd00:100::/64#! ..
rt1 running ipv6-unicast#! l3vpn export vpn true
rt1 running ipv6-unicast#! l3vpn export label 101
rt1 running ipv6-unicast#! l3vpn export route-target 65500:1
rt1 running ipv6-unicast#! l3vpn export route-distinguisher 65500:20
rt1 running ipv6-unicast#! l3vpn import vpn true
rt1 running ipv6-unicast#! l3vpn import route-target 65500:1
rt1 running ipv6-unicast#! / vrf main interface physical eth1
rt1 running physical eth1#! ipv4 address 192.168.0.1/24
rt1 running physical eth1#! ipv6 address ::ffff:192.168.0.1/120
rt1 running physical eth1#! port pci-b0s4
rt1 running physical eth1#! .. loopback loop1
rt1 running loopback loop1#! ipv4 address 198.51.100.1/32
rt1 running loopback loop1#! .. .. routing interface loop1
rt1 running interface loop1#! isis area-tag 1
rt1 running interface loop1#! isis ipv4-routing true
rt1 running interface loop1#! .. interface eth1
rt1 running interface eth1#! isis area-tag 1
rt1 running interface eth1#! isis ipv4-routing true
rt1 running interface eth1#! .. bgp
rt1 running bgp#! as 65500
rt1 running bgp#! router-id 198.51.100.1
rt1 running bgp#! neighbor 198.51.100.4
rt1 running neighbor 198.51.100.4#! remote-as 65500
rt1 running neighbor 198.51.100.4#! update-source loop1
rt1 running neighbor 198.51.100.4#! address-family ipv4-unicast enabled false
rt1 running neighbor 198.51.100.4#! address-family ipv6-vpn enabled true
rt1 running neighbor 198.51.100.4#! .. .. isis instance 1
rt1 running instance 1# is-type level-2
rt1 running instance 1# area-address 49.0000.0007.e901.1111.00
rt1 running instance 1# traffic-engineering ipv4-router-address 198.51.100.1
rt1 running instance 1# segment-routing
rt1 running segment-routing# enabled true
rt1 running segment-routing# label-blocks srgb lower-bound 1000 upper-bound 2000
rt1 running segment-routing# label-blocks srlb lower-bound 31000 upper-bound 31999
rt1 running segment-routing# msd node-msd 8
rt1 running segment-routing# prefix-sid-map 198.51.100.1/32 sid-value 11
rt1 running segment-routing#
rt2
rt2 running config# vrf main interface physical eth1
rt2 running physical eth1#! ipv4 address 192.168.1.2/24
rt2 running physical eth1#! port pci-b0s4
rt2 running physical eth1# ..
rt2 running interface# physical eth2
rt2 running physical eth2#! port pci-b0s5
rt2 running physical eth2# ipv4 address 192.168.0.2/24
rt2 running physical eth2# ..
rt2 running interface# loopback loop1
rt2 running loopback loop1# ipv4 address 198.51.100.2/32
rt2 running loopback loop1# .. .. routing interface loop1
rt2 running interface loop1# isis area-tag 1
rt2 running interface loop1#! isis ipv4-routing true
rt2 running interface loop1#! .. interface eth1
rt2 running interface eth1#! isis area-tag 1
rt2 running interface eth1#! isis ipv4-routing true
rt2 running interface eth1#! .. interface eth2
rt2 running interface eth2#! isis area-tag 1
rt2 running interface eth2#! isis ipv4-routing true
rt2 running interface eth2#! .. isis instance 1
rt2 running instance 1# is-type level-2
rt2 running instance 1# area-address 49.0000.0007.e901.2222.00
rt2 running instance 1# traffic-engineering ipv4-router-address 198.51.100.2
rt2 running instance 1# segment-routing
rt2 running segment-routing# enabled true
rt2 running segment-routing# label-blocks srgb lower-bound 1000 upper-bound 2000
rt2 running segment-routing# label-blocks srlb lower-bound 30000 upper-bound 30999
rt2 running segment-routing# msd node-msd 8
rt2 running segment-routing# prefix-sid-map 198.51.100.2/32 sid-value 22
rt2 running segment-routing#
rt3
rt3 running config# vrf main interface physical eth1
rt3 running physical eth1#! ipv4 address 192.168.1.3/24
rt3 running physical eth1#! port pci-b0s4
rt3 running physical eth1# ..
rt3 running interface# physical eth2
rt3 running physical eth2#! port pci-b0s5
rt3 running physical eth2# ipv4 address 192.168.2.3/24
rt3 running physical eth2# ..
rt3 running interface# loopback loop1
rt3 running loopback loop1# ipv4 address 198.51.100.3/32
rt3 running loopback loop1# .. .. routing interface loop1
rt3 running interface loop1# isis area-tag 1
rt3 running interface loop1#! isis ipv4-routing true
rt3 running interface loop1#! .. interface eth1
rt3 running interface eth1#! isis area-tag 1
rt3 running interface eth1#! isis ipv4-routing true
rt3 running interface eth1#! .. interface eth2
rt3 running interface eth2#! isis area-tag 1
rt3 running interface eth2#! isis ipv4-routing true
rt3 running interface eth2#! .. isis instance 1
rt3 running instance 1# is-type level-2
rt3 running instance 1# area-address 49.0000.0007.e901.3333.00
rt3 running instance 1# traffic-engineering ipv4-router-address 198.51.100.3
rt3 running instance 1# segment-routing
rt3 running segment-routing# enabled true
rt3 running segment-routing# label-blocks srgb lower-bound 1000 upper-bound 2000
rt3 running segment-routing# label-blocks srlb lower-bound 32000 upper-bound 32999
rt3 running segment-routing# msd node-msd 8
rt3 running segment-routing# prefix-sid-map 198.51.100.3/32 sid-value 33
rt3 running segment-routing#
rt4
rt4 running config# vrf main l3vrf l3vrf1
rt4 running l3vrf l3vrf1#! table-id 10
rt4 running l3vrf l3vrf1# interface physical eth2
rt4 running physical eth2#! port pci-b0s5
rt4 running physical eth2# ipv6 address fd00:200::4/64
rt4 running physical eth2# .. .. routing bgp
rt4 running bgp#! as 65500
rt4 running bgp#! router-id 198.51.100.4
rt4 running bgp#! address-family ipv6-unicast
rt4 running ipv6-unicast#! network fd00:200::/64
rt4 running network fd00:200::/64#! ..
rt4 running ipv6-unicast#! l3vpn export vpn true
rt4 running ipv6-unicast#! l3vpn export label 104
rt4 running ipv6-unicast#! l3vpn export route-target 65500:1
rt4 running ipv6-unicast#! l3vpn export route-distinguisher 65500:10
rt4 running ipv6-unicast#! l3vpn import vpn true
rt4 running ipv6-unicast#! l3vpn import route-target 65500:1
rt4 running ipv6-unicast#! / vrf main interface physical eth1
rt4 running physical eth1#! ipv4 address 192.168.2.4/24
rt4 running physical eth1#! ipv6 address ::ffff:192.168.2.4/120
rt4 running physical eth1#! port pci-b0s4
rt4 running physical eth1#! .. loopback loop1
rt4 running loopback loop1#! ipv4 address 198.51.100.4/32
rt4 running loopback loop1#! .. .. routing interface loop1
rt4 running interface loop1#! isis area-tag 1
rt4 running interface loop1#! isis ipv4-routing true
rt4 running interface loop1#! .. interface eth1
rt4 running interface eth1#! isis area-tag 1
rt4 running interface eth1#! isis ipv4-routing true
rt4 running interface eth1#! .. bgp
rt4 running bgp#! as 65500
rt4 running bgp#! router-id 198.51.100.4
rt4 running bgp#! neighbor 198.51.100.1
rt4 running neighbor 198.51.100.4#! remote-as 65500
rt4 running neighbor 198.51.100.4#! update-source loop1
rt4 running neighbor 198.51.100.4#! address-family ipv4-unicast enabled false
rt4 running neighbor 198.51.100.4#! address-family ipv6-vpn enabled true
rt4 running neighbor 198.51.100.4#! .. .. isis instance 1
rt4 running instance 1# is-type level-2
rt4 running instance 1# area-address 49.0000.0007.e901.4444.00
rt4 running instance 1# traffic-engineering ipv4-router-address 198.51.100.4
rt4 running instance 1# segment-routing
rt4 running segment-routing# enabled true
rt4 running segment-routing# label-blocks srgb lower-bound 1000 upper-bound 2000
rt4 running segment-routing# label-blocks srlb lower-bound 33000 upper-bound 33999
rt4 running segment-routing# msd node-msd 8
rt4 running segment-routing# prefix-sid-map 198.51.100.4/32 sid-value 44
rt4 running segment-routing#
The rt1 device receives and imports the IPv6 prefix from rt4 with the
::ffff:c633:6404 next hop, and the 104 label value. The next hop is an
IPv4 mapped IPv6 address whose IPv4 address is 198.51.100.4.
rt1
rt1> show bgp ipv6 vpn
BGP table version is 0, local router ID is 198.51.100.1, vrf id 0
Default local pref 100, local AS 65500
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 65500:10
ifd00:200::/64 ::ffff:c633:6404
0 100 0 i
Route Distinguisher: 65500:20
*> fd00:100::/64 ::@13< 0 32768 i
rt1> show bgp l3vrf l3vrf1 ipv6
BGP table version is 4, local router ID is 198.51.100.1, vrf id 13
Default local pref 100, local AS 65500
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found
Network Next Hop Metric LocPrf Weight Path
*> fd00:100::/64 :: 0 32768 i
*> fd00:200::/64 ::ffff:c633:6404@0<
0 100 0 i
The rt1 device is able to resolve the next hop reachility over the ipv4 network by using
the 198.51.100.4 address. The resulting recursive route uses the 192.168.0.2 next hop
IP address. Because rt1 has an ipv4 mapped IPv6 address derived from the 192.168.0.0/24
network, the associated BGP IPv6 network can be resolved using a recursive route with the
::ffff:c0a8:2 IPv6 address.
rt1
rt1> show ipv4-routes to 198.51.100.4
Routing entry for 198.51.100.4/32
Known via "isis", distance 115, metric 40, best
Last update 00:44:48 ago
* 192.168.0.2, via eth1, label 1044, weight 1
rt1> show bgp nexthop detail
[..]
::ffff:c633:6404 valid [IGP metric 0], #paths 1
gate ::ffff:c0a8:2, if eth1
Last update: Wed Jun 21 13:06:09 2023
The resulting IPv6 route is a labeled IPv6 route whose recursive route uses the
::ffff:c0a8:2 next hop IPv6 address. The 104 label value is the VPN service
label from the prefix, and the 1044 label value is given by the IGP.
rt1
rt1> show ipv6-routes l3vrf l3vrf1
Codes: K - kernel route, C - connected, S - static, R - RIPng,
O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
A - Babel, D - SHARP, F - PBR, f - OpenFabric,
> - selected route, * - FIB route, q - queued, r - rejected, b - backup
t - trapped, o - offload failure
VRF l3vrf1:
K>* ::/0 [255/8192] unreachable (ICMP unreachable), 00:35:59
C>* fd00:100::/64 is directly connected, eth2, 00:35:58
B> fd00:200::/64 [200/0] via ::ffff:c633:6404 (vrf default) (recursive), label 104, weight 1, 00:30:53
* via ::ffff:c0a8:2, eth1 (vrf default), label 1044/104, weight 1, 00:30:53
An intermediate route marked 6 is created, and reuses the same label value as the
matching IPv4 IGP route.
rt1> show ipv6-routes
Codes: K - kernel route, C - connected, S - static, R - RIPng,
O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
A - Babel, D - SHARP, F - PBR, f - OpenFabric,
> - selected route, * - FIB route, q - queued, r - rejected, b - backup
t - trapped, o - offload failure
C>* ::ffff:c0a8:0/120 is directly connected, eth1, 02:44:39
6>* ::ffff:c633:6404/128 [1/0] via ::ffff:c0a8:2, eth1, label 1044, weight 1, 00:32:28
L3VPN advanced configuration¶
Avoid routing loops by using SoO tag¶
There are situations, where VPNs are connected to various PEs. For example, a CE, or a group of devices forming an AS group needs to be connected to multiple PEs. This can also be the case for multihoming purposes. The below figure can help:
BGP l3vpn filtering prefixes with SOO use case example¶
The above diagram shows that a given prefix originated from ce1 is transmitted
to pe1, then pe2. That prefix is also sent to ce2, which already knows that
this prefix is reachable via its own private AS. One way to mitigate that situation
is to use SOO extended communities on the PE. The below configuration can be used
as example:
pe1
pe1 running config# routing
pe1 running routing# route-map add_soo_ext_community
pe1 running route-map add_soo_ext_community#! seq 1
pe1 running seq 1#! policy permit
pe1 running seq 1# set
pe1 running set# extcommunity
pe1 running extcommunity# soo 65500:0
pe1 running extcommunity# ..
pe1 running set# ..
pe1 running seq 1# ..
pe1 running route-map add_soo_ext_community# ..
pe1 running routing# ..
pe1 running config# vrf main
pe1 running vrf main# interface physical eth2
pe1 running physical eth2#! port pci-b0s5
pe1 running physical eth2# ipv4
pe1 running ipv4# address 10.125.0.1/24
pe1 running ipv4# ..
pe1 running physical eth2# ..
pe1 running interface# ..
pe1 running vrf main# routing bgp
pe1 running bgp#! as 65500
pe1 running bgp# router-id 1.1.1.1
pe1 running bgp# neighbor 10.125.0.2
pe1 running neighbor 10.125.0.2#! remote-as 65500
pe1 running neighbor 10.125.0.2# address-family
pe1 running address-family# ipv4-unicast
pe1 running ipv4-unicast# enabled false
pe1 running ipv4-unicast# ..
pe1 running address-family# ipv4-vpn
pe1 running ipv4-vpn# ..
pe1 running address-family# ..
pe1 running neighbor 10.125.0.2# ..
pe1 running bgp# .. ..
pe1 running vrf main# l3vrf customer2
pe1 running l3vrf customer2#! table-id 10
pe1 running l3vrf customer2# routing
pe1 running routing# bgp
pe1 running bgp# network-import-check false
pe1 running bgp# address-family
pe1 running address-family# ipv4-unicast
pe1 running ipv4-unicast# network 9.9.9.0/24
pe1 running network 9.9.9.0/24# ..
pe1 running ipv4-unicast# l3vpn
pe1 running l3vpn# export
pe1 running export# vpn true
pe1 running export# route-target 1:1
pe1 running export# route-distinguisher 65500:1
pe1 running export# route-map add_soo_ext_community
pe1 running export# ..
pe1 running l3vpn# import
pe1 running import#! vpn true
pe1 running import#! route-target 1:1
pe1 running import# ..
pe1 running l3vpn# ..
pe1 running ipv4-unicast# .. ..
pe1 running bgp# .. ..
pe1 running l3vrf customer2# interface
pe1 running interface# physical eth1
pe1 running physical eth1#! port pci-b0s4
pe1 running physical eth1# ipv4
pe1 running ipv4# address 9.9.9.0/24
pe1 running ipv4#
pe2
pe2 running routing# route-map filter_soo_from_65500_0
pe2 running route-map filter_soo_from_65500_0#! seq 1
pe2 running seq 1#! policy deny
pe2 running seq 1# match
pe2 running match# extcommunity filter_soo_comm_list
pe2 running match# ..
pe2 running seq 1# ..
pe2 running route-map filter_soo_from_65500_0# seq 2 policy permit
pe2 running route-map filter_soo_from_65500_0# ..
pe2 running routing# bgp
pe2 running bgp# extcommunity-list filter_soo_comm_list
pe2 running extcommunity-list filter_soo_comm_list#! policy 1 permit soo 65500:0
pe2 running extcommunity-list filter_soo_comm_list# ..
pe2 running bgp# ..
pe2 running routing# ..
pe2 running config# vrf main
pe2 running vrf main# routing
pe2 running routing# bgp
pe2 running bgp#! as 65500
pe2 running bgp# router-id 2.2.2.2
pe2 running bgp# neighbor 10.125.0.1
pe2 running neighbor 10.125.0.1#! remote-as 65500
pe2 running neighbor 10.125.0.1# address-family
pe2 running address-family# ipv4-unicast
pe2 running ipv4-unicast# enabled false
pe2 running ipv4-unicast# ..
pe2 running address-family# ipv4-vpn
pe2 running ipv4-vpn# ..
pe2 running address-family# ..
pe2 running neighbor 10.125.0.1# ..
pe2 running bgp# .. ..
pe2 running vrf main#
pe2 running vrf main# interface
pe2 running interface# physical eth2
pe2 running physical eth2#! port pci-b0s5
pe2 running physical eth2# ipv4
pe2 running ipv4# address 10.125.0.2/24
pe2 running ipv4# ..
pe2 running physical eth2# ..
pe2 running interface# ..
pe2 running vrf main# l3vrf customer2
pe2 running l3vrf customer2#! table-id 10
pe2 running l3vrf customer2# routing bgp
pe2 running bgp# address-family
pe2 running address-family# ipv4-unicast
pe2 running ipv4-unicast# l3vpn
pe2 running l3vpn# export
pe2 running export# vpn true
pe2 running export# label auto
pe2 running export# route-target 1:1
pe2 running export# route-distinguisher 65500:2
pe2 running export# ..
pe2 running l3vpn# import
pe2 running import#! vpn true
pe2 running import#! route-target 1:1
pe2 running import# route-map filter_soo_from_65500_0
pe2 running import# ..
pe2 running l3vpn# ..
pe2 running ipv4-unicast# ..
pe2 running address-family# ..
pe2 running bgp# ..
pe2 running routing# ..
pe2 running l3vrf customer2# interface
pe2 running interface# physical eth1
pe2 running physical eth2#! port pci-b0s4
pe2 running physical eth2# ipv4
pe2 running ipv4# address 9.9.9.0/24
pe2 running ipv4# ..
pe2 running physical eth2# ..
pe2 running interface#
As can be seen, when exporting prefixes from customer2 L3VRF, a SOO extended
community is appended to BGP update and transmitted to the interconnection network.
At reception, pe2 will filter incoming updates matching the SOO extended community.
The update will be ignored in L3VRF for ce2. Below dump shows how to display the
extended community-list attached to an incoming ipv4-vpn prefix. When detaching the
route-map at inbound from L3VPN, the network 9.9.9.0/24 prefix becomes available
again.
pe2 running config# show bgp ipv4 vpn prefix 9.9.9.0/24
BGP routing table entry for 65500:1:9.9.9.0/24
not allocated
Paths: (1 available, best #1)
Not advertised to any peer
Local
10.125.0.1 from 10.125.0.1 (1.1.1.1)
Origin incomplete, metric 0, localpref 100, valid, internal, best (First path received)
Extended Community: RT:1:1 SoO:65500:0
Remote label: 3
Last update: Mon Nov 22 17:05:42 2021
pe2 running config# show bgp l3vrf customer2 ipv4 prefix 9.9.9.0/24
BGP routing table entry for 9.9.9.0/24
Paths: (1 available, no best path)
Not advertised to any peer
Imported from 65500:1:
Local
10.125.0.1 (inaccessible) from 0.0.0.0 (2.2.2.2) vrf 0 announce-nh-self
Origin IGP, metric 0, localpref 100, invalid, sourced, local
Extended Community: RT:1:1 SoO:65500:0
Remote label: 3
Last update: Mon Nov 22 17:29:00 2021
How to create half duplex connections¶
The previous chapters explained how to connect multiple VPNs by leaking traffic either locally or remotely. The same technology can be used to forbid traffic to leak locally. This can be the case with a PE device with multiple subscribers connected locally in separate VPNs, and where the ISPs want to monitor traffic on a central PE located in a separate place. There are some situations where PE hosts many subscribers, and authorizing local connectivity between subscribers may lead to additional complexity in monitoring such configuration. Redirecting traffic similarly to a hub and spoke way, permits reducing complexity on local PE. This situation can happen on large-scale wholesale service provider environments.
This can be done by instantiating VPN policies on each direction, for a given client.
The wording half-duplex refers to the ability to associate different policies to traffic
flow for each direction for a given client. The traffic is redirected to that central PE,
similary to a star topology, where traffic is forwarded to the central PE before being
redirected to the same PE, but in another VPN.
The below configuration illustrates the customer1 VPN that wants to access to the customer2
VPN. Two devices are configured, rt1 stands for the PE hosting the two VPNs, while rt2
acts as the central PE. The proposed way to implement such policies consists in creating 2
L3VRFs for each client.
one for hosting the client interface.
one for handling the upstream traffic. This one is associated to a first VPN policy that will import the remote networks. Usually, this L3VRF will host the default route to reach the central PE. This can also be any summary routes configured on the central PE.
one for handling the downstream traffic. This one is associated to a second VPN policy that will export the client network. The local interface is associated to that L3VRF. This permits to propagate networking information from OSPF into BGP for instance. Note that upstream is also received on that L3VRF, but is redirected to upstream L3VRF thanks to a default route leak.
rt1
rt1 running config# vrf main
rt1 running vrf main# interface physical eth1
rt1 running physical eth1#! port pci-b0s4
rt1 running physical eth1# ..
rt1 running interface# physical eth0
rt1 running physical eth0#! port pci-b0s5
rt1 running physical eth0# ipv4
rt1 running ipv4# address 10.125.0.1/24
rt1 running ipv4# .. ..
rt1 running interface# loopback loop1
rt1 running loopback loop1# ipv4
rt1 running ipv4# address 1.1.1.1/32
rt1 running ipv4# ..
rt1 running loopback loop1# ..
rt1 running interface# ..
rt1 running vrf main# routing ospf
rt1 running ospf# router-id 1.1.1.1
rt1 running ospf# network 1.1.1.1/32 area 0.0.0.0
rt1 running ospf# network 10.125.0.0/24 area 0.0.0.0
rt1 running ospf# ..
rt1 running routing# ..
rt1 running vrf main# routing mpls ldp
rt1 running ldp# router-id 1.1.1.1
rt1 running ldp# address-family ipv4
rt1 running ipv4#! discovery transport-address 1.1.1.1
rt1 running ipv4# interface eth0
rt1 running interface eth0# / vrf main routing bgp
rt1 running bgp#! as 65500
rt1 running bgp# router-id 1.1.1.1
rt1 running bgp# address-family
rt1 running address-family# ipv4-unicast
rt1 running ipv4-unicast# ..
rt1 running address-family# ..
rt1 running bgp# neighbor 2.2.2.2
rt1 running neighbor 2.2.2.2#! remote-as 65500
rt1 running neighbor 2.2.2.2# update-source 1.1.1.1
rt1 running neighbor 2.2.2.2# address-family
rt1 running address-family# ipv4-unicast
rt1 running ipv4-unicast# enabled false
rt1 running ipv4-unicast# ..
rt1 running address-family# ipv4-vpn
rt1 running ipv4-vpn# / vrf main l3vrf customer1_up
rt1 running l3vrf customer1_up#! table-id 100
rt1 running l3vrf customer1_up# routing bgp
rt1 running bgp# address-family
rt1 running address-family# ipv4-unicast
rt1 running ipv4-unicast# l3vpn
rt1 running l3vpn# import
rt1 running import#! vpn true
rt1 running import#! route-target 100:65500
rt1 running import# route-target 200:65500
rt1 running import# / vrf main l3vrf customer2_up
rt1 running l3vrf customer2_up#! table-id 200
rt1 running l3vrf customer2_up# routing bgp
rt1 running bgp# address-family
rt1 running address-family# ipv4-unicast
rt1 running ipv4-unicast# l3vpn
rt1 running l3vpn# import
rt1 running import#! vpn true
rt1 running import#! route-target 200:65500
rt1 running import# route-target 100:65500
rt1 running import# / vrf main l3vrf customer1_down
rt1 running l3vrf customer1_down#! table-id 101
rt1 running l3vrf customer1_down# routing
rt1 running routing# static
rt1 running static# ipv4-route 0.0.0.0/0
rt1 running ipv4-route 0.0.0.0/0#! next-hop customer1_up nexthop-l3vrf customer1_up
rt1 running ipv4-route 0.0.0.0/0# .. ..
rt1 running routing# bgp
rt1 running bgp# address-family
rt1 running address-family# ipv4-unicast
rt1 running ipv4-unicast# network 10.100.0.0/24
rt1 running network 10.100.0.0/24# ..
rt1 running ipv4-unicast# l3vpn
rt1 running l3vpn# export
rt1 running export# vpn true
rt1 running export# label auto
rt1 running export# route-target 101:65500
rt1 running export# route-distinguisher 101:65500
rt1 running export# nexthop 1.1.1.1
rt1 running export# / vrf main l3vrf customer1_down
rt1 running l3vrf customer1_down# interface vlan eth1.10
rt1 running vlan eth1.10#! description customer1
rt1 running vlan eth1.10#! ipv4
rt1 running ipv4#! address 10.100.0.1/24
rt1 running ipv4#! ..
rt1 running vlan eth1.10#! vlan-id 10
rt1 running vlan eth1.10#! link-interface eth1
rt1 running vlan eth1.10# / vrf main l3vrf customer2_down
rt1 running l3vrf customer2_down#! table-id 201
rt1 running l3vrf customer2_down# routing
rt1 running routing# static
rt1 running static# ipv4-route 0.0.0.0/0
rt1 running ipv4-route 0.0.0.0/0#! next-hop customer2_up nexthop-l3vrf customer2_up
rt1 running ipv4-route 0.0.0.0/0# .. ..
rt1 running routing# bgp
rt1 running bgp# address-family
rt1 running address-family# ipv4-unicast
rt1 running ipv4-unicast# network 10.101.0.0/24
rt1 running network 10.101.0.0/24# ..
rt1 running ipv4-unicast# l3vpn
rt1 running l3vpn# export
rt1 running export# vpn true
rt1 running export# label auto
rt1 running export# route-target 201:65500
rt1 running export# route-distinguisher 201:65500
rt1 running export# nexthop 1.1.1.1
rt1 running export# / vrf main l3vrf customer2_down
rt1 running l3vrf customer2_down# interface vlan eth1.20
rt1 running vlan eth1.20#! ipv4
rt1 running ipv4#! address 10.101.0.1/24
rt1 running ipv4#! ..
rt1 running vlan eth1.20#! vlan-id 20
rt1 running vlan eth1.20#! link-interface eth1
rt1 running vlan eth1.20# ..
rt1 running interface# ..
rt1 running l3vrf customer2_down#
The below rt2 configuration reflects a typical configuration where a given L3VRF is
being associated a single VPN policy. This VPN will receive networks from the remote
downstream L3VRFs, and also push default route to the remote upstream L3VRFs.
rt2
rt2 running config# vrf main
rt2 running vrf main# interface
rt2 running interface# physical eth0
rt2 running physical eth0#! port pci-b0s5
rt2 running physical eth0# ipv4
rt2 running ipv4# address 10.125.0.2/24
rt2 running ipv4# ..
rt2 running physical eth0# ..
rt2 running interface# loopback loop1
rt2 running loopback loop1# ipv4
rt2 running ipv4# address 2.2.2.2/32
rt2 running ipv4# ..
rt2 running loopback loop1# ..
rt2 running interface# ..
rt2 running vrf main# routing
rt2 running routing# bgp
rt2 running bgp#! as 65500
rt2 running bgp# router-id 2.2.2.2
rt2 running bgp# address-family
rt2 running address-family# ipv4-unicast
rt2 running ipv4-unicast# enabled false
rt2 running ipv4-unicast# ..
rt2 running address-family# ..
rt2 running bgp# neighbor 1.1.1.1
rt2 running neighbor 1.1.1.1#! remote-as 65500
rt2 running neighbor 1.1.1.1# update-source 2.2.2.2
rt2 running neighbor 1.1.1.1# address-family
rt2 running address-family# ipv4-unicast
rt2 running ipv4-unicast# enabled false
rt2 running ipv4-unicast# ..
rt2 running address-family# ipv4-vpn
rt2 running ipv4-vpn# ..
rt2 running address-family# ..
rt2 running neighbor 1.1.1.1# ..
rt2 running bgp# ..
rt2 running routing# ospf
rt2 running ospf# router-id 2.2.2.2
rt2 running ospf# network 2.2.2.2/32 area 0
rt2 running ospf# network 10.125.0.0/24 area 0
rt2 running ospf# ..
rt2 running routing# mpls ldp
rt2 running ldp# router-id 2.2.2.2
rt2 running ldp# address-family ipv4
rt2 running ipv4#! discovery transport-address 2.2.2.2
rt2 running ipv4# interface eth0
rt2 running interface eth0# / vrf main
rt2 running vrf main# l3vrf customer1
rt2 running l3vrf customer1#! table-id 100
rt2 running l3vrf customer1# routing bgp
rt2 running bgp# address-family
rt2 running address-family# ipv4-unicast
rt2 running ipv4-unicast# network 0.0.0.0/0
rt2 running network 0.0.0.0/0# ..
rt2 running ipv4-unicast# l3vpn
rt2 running l3vpn# import
rt2 running import#! vpn true
rt2 running import#! route-target 101:65500
rt2 running import# route-target 201:65500
rt2 running import# ..
rt2 running l3vpn# ..
rt2 running ipv4-unicast# l3vpn
rt2 running l3vpn# export
rt2 running export# vpn true
rt2 running export# label auto
rt2 running export# route-target 100:65500
rt2 running export# route-distinguisher 100:65500
rt2 running export# ..
rt2 running l3vpn# ..
rt2 running ipv4-unicast# ..
rt2 running address-family# ..
rt2 running bgp# ..
rt2 running routing# ..
rt2 running l3vrf customer1# ..
rt2 running vrf main# l3vrf customer2
rt2 running l3vrf customer2#! table-id 200
rt2 running l3vrf customer2# routing bgp
rt2 running bgp# address-family
rt2 running address-family# ipv4-unicast
rt2 running ipv4-unicast# network 0.0.0.0/0
rt2 running network 0.0.0.0/0# ..
rt2 running ipv4-unicast# l3vpn
rt2 running l3vpn# import
rt2 running import#! vpn true
rt2 running import#! route-target 101:65500
rt2 running import# route-target 201:65500
rt2 running import# ..
rt2 running l3vpn# ..
rt2 running ipv4-unicast# l3vpn
rt2 running l3vpn# export
rt2 running export# vpn true
rt2 running export# label auto
rt2 running export# route-target 200:65500
rt2 running export# route-distinguisher 200:65500
rt2 running export# ..
rt2 running l3vpn# ..
rt2 running ipv4-unicast# ..
rt2 running address-family# ..
rt2 running bgp#
Such configuration on rt1 is completely scalable from both the configuration perspective,
and the performance perspective.
Withdraw VPN prefixes which are not imported locally¶
It is possible to retain or not VPN prefixes that are not imported by local VRF configuration. This can be done via the following command in the context of the desired VPN address-family. By default, all prefixes are retained.
vsr running config# vrf main
vsr running vrf main# routing bgp
vsr running bgp# as 65500
vsr running bgp# address-family ipv4-vpn
vsr running ipv4-vpn# retain route-target all false
vsr running ipv4-vpn#
Label allocation mode per next-hop¶
By default, a single label is allocated by the L3VRF instance, and is exported
along with the BGP updates in the NLRI message. The below vrf1 BGP
configuration exports the 300 label value, for all prefixes.
vsr running config# vrf main
vsr running vrf main# interface
vsr running interface# physical eth0
vsr running physical eth0#! port pci-b0s5
vsr running physical eth0# ipv4 address 192.0.2.1/24
vsr running physical eth0# .. ..
vsr running vrf main# routing bgp
vsr running bgp#! as 65500
vsr running bgp# ebgp-requires-policy false
vsr running bgp# router-id 192.0.2.1
vsr running bgp# neighbor 192.0.2.2 remote-as 65501
vsr running bgp# neighbor 192.0.2.2 address-family ipv4-unicast enabled false
vsr running bgp# neighbor 192.0.2.2 address-family ipv4-vpn enabled true
vsr running bgp# .. ..
vsr running vrf main# l3vrf vrf1
vsr running l3vrf vrf1#! table-id 10
vsr running l3vrf vrf1# interface physical eth1
vsr running physical eth1#! port pci-b0s4
vsr running physical eth&# ipv4 address 172.31.0.1/24
vsr running physical eth1# .. ..
vsr running l3vrf vrf1# routing bgp
vsr running bgp# as 65500
vsr running bgp# ebgp-requires-policy false
vsr running bgp# router-id 192.0.2.1
vsr running bgp# address-family ipv4-unicast
vsr running ipv4-unicast# redistribute connected
vsr running ipv4-unicast# network 192.168.0.0/24
vsr running network 192.168.0.0/24# .. ..
vsr running ipv4-unicast# l3vpn export route-distinguisher 65500:1
vsr running ipv4-unicast# l3vpn export route-target 65500:100
vsr running ipv4-unicast# l3vpn export label 300
vsr running ipv4-unicast# l3vpn export route-target 65500:100
vsr running ipv4-unicast# l3vpn import route-target 65500:100
vsr running ipv4-unicast# l3vpn import vpn true
vsr running ipv4-unicast# .. ..
The BGP updates are propagated to the remote peer, and an MPLS operation
is created in the dataplane to handle the return traffic. The incoming MPLS
packets are popped and transmitted to the vrf1 next-hop which handles
the packet routing in the vrf1 L3VRF. This MPLS operation is named
a pop-and-lookup operation.
vsr> show mpls table
Inbound Label Type Nexthop Outbound Label
----------------------------------------------
300 BGP vrf1 -
The per-nexthop allocation mode gets a unique label for each
resolved next-hop.
vsr running config# vrf main
vsr running vrf main# l3vrf vrf1
vsr running l3vrf vrf1# routing bgp
vsr running bgp# address-family ipv4-unicast
vsr running ipv4-unicast# l3vpn export label-allocation-mode per-nexthop
vsr running ipv4-unicast#
For instance, the 192.168.0.0/24 prefix which is reachable via the
172.31.0.2 next-hop will be assigned a new label value.
vsr> show bgp ipv4 vpn prefix 192.168.0.0/24
BGP routing table entry for 65500:1:192.168.0.0/24, version 6
not allocated
Paths: (1 available, best #1)
Not advertised to any peer
Local
0.0.0.0 from 0.0.0.0 (192.0.2.1) vrf vrf1(90) announce-nh-self
Origin IGP, metric 0, weight 32768, valid, sourced, local, best (First path received)
Extended Community: RT:1:1 RT:65500:1 RT:65500:100
Originator: 192.0.2.1
Remote label: 16
Last update: Thu Mar 16 09:52:44 2023
A new MPLS operation is created in the dataplane for the return MPLS
traffic with the 16 label value. The traffic is popped and sent out
to the 172.31.0.2 IP address. This MPLS operation is named
a pop-and-forward operation. The processing of the downstream traffic
is simplified, as it operates at the switching level, and not at the routing
level. Consequently, using that mode is recommended to save CPU cycles and
increase the performance of the Virtual Service Router.
vsr> show mpls table
Inbound Label Type Nexthop Outbound Label
----------------------------------------------
16 BGP 172.31.0.2 -
300 BGP vrf1 -
Note
Having unique MPLS values can help the operator to monitor
the MPLS traffic, by using some tools like ipfix.
See also
Some use cases require applying some routing services to the
downstream traffic. When the per-nexthop label allocation is used,
the pop-and-forward operations are used, and as consequence,
the whole routing services are bypassed. To force the MPLS
operation to pop the label and route the IP traffic to the L3VRF
routing table, the below pop-and-lookup command can be enabled
to implement that behavior.
vsr running config# vrf main
vsr running vrf main# l3vrf vrf1
vsr running l3vrf vrf1# routing bgp
vsr running bgp# address-family ipv4-unicast
vsr running ipv4-unicast# l3vpn export label-operation pop-and-lookup
vsr running ipv4-unicast#
All the MPLS traffic is popped and sent to the local L3VRF routing table.
vsr> show mpls table
Inbound Label Type Nexthop Outbound Label
----------------------------------------------
16 BGP vrf1 -
300 BGP vrf1 -
Note
The per-nexthop label allocation mode is used on topologies connecting
L3VRF networks directly connected behind PE devices: for instance,
downstream traffic from a given pe2 device will be switched directly
to the cpe1 router located behind the pe1 device.
In the case the destination address of the incoming MPLS packet is
located behind another PE device, then the receiving device acts as an
ASBR device and the switch-l3vpn-labels command should be used instead.
More information: Redistribute L3VPN traffic between ASBRs from 2
domains.
See also
RFC 8960: a yang data model for MPLS base, MPLS operations