BGP for L3VPN¶
BGP routing protocol is very rich, and permits exchanging more complex information. With the increasing usage of overlay information (widely used in data centers, but also deployed in ISPs), BGP evolves and is able to to carry tunneling information through L3VPN.
L3VPN overview¶
BGP l3vpn use case example¶
Above drawing illustrates a setup made up of 2 symmetrical sites. Each site is separated with a PE device. In the case the site is a data center, the PE could be replaced with a DC-GW. To simplify, each site is made up of 2 distinct VPNs. L3VPN functionality helps to exchange information about the 2 VPNs, between the 2 sites.
This functionality will subsequently enable data path forwarding. IP Traffic between the 2 PEs will be encapsulated into an MPLS label. On each site, traffic between each CE and the PE is standard IP over ethernet traffic.
From the drawing, the following use cases will be more in detail leveraged successively in that document:
how to interconnect traffic from different VPNs on a same site. This is a specific case from route-leaking. The basic L3VPN commands will be illustrated, as well as the commmands to create Cross-VRF interfaces across VRF.
how to interconnect traffic from a same VPN between sites. This use case will generalise usage of Cross-VRF interfaces with default VRF, as well as explaining how to configure backbone so as to carry MPLS labels.
how to interconnect traffic from different VPNs between sites This use case will generalise the L3VPN use case in an MPLS based framework.
L3VPN terminology¶
It is important to understand some L3VPN terminology. In this paragraph we will give the most important concepts.
- VPN
This acronym refers here to a routing entity, also called VRF Creating a VPN consists in creating a BGP instance in a separated VRF. More information in BGP VRF.
- Route Distinguisher, RD :
This attribute is specific for each VPN. This information is exported along with the L3VPN information of the BGP information
- L3VPN
This refers to creating an overlay with IP packets. In this chapter, L3VPN refers to encapsulating IP traffic over MPLS traffic.
- VPNv4, VPNv6
This refers to RFC 4364: that defines how BGP implements L3VPN with MPLS
- Route Target, RT:
RT and RD share the same format. A VPN can have 2 list of RTs. One is dedicated for import. This will help BGP to import incoming routing entries that come from a remote BGP entity. There is also a list for export, that is sent to remote BGP entities. Route Target is the key element for sharing information across VPNs.
- VRF route leaking:
This refers to the ability to share a route from a VPN to an other entity. See chapter BGP VRF rout leak. This ability requires that the VPN that shares a common route, do not have overlapping with the IP provisioning of the networks.
Configuring locally route leaking between VPN¶
BGP configuration¶
Below configuration illustrates a setup made up with 2 VPNs. As can be seen, there is a BGP instance in each of the 2 VR instances, plus the BGP core instance. The 2 instances are configured so as to create leaking between both VPNs. Actually configuration shows the following:
the RD is different, indicating that there are 2 distinct VPNs.
the RT export settings of each VPN is being matched by the RT import settings of the other VPN. This configuration means that route leaking will occur.
vrf main
routing bgp
as 65500
..
..
..
vrf customer1
routing bgp
as 65500
address-family ipv4-unicast
network 192.168.3.0/24
..
l3vpn export route-distinguisher 1:55
l3vpn export route-target 11:22
l3vpn export vpn true
l3vpn import route-target 11:22 route-target 22:44
l3vpn import vpn true
..
..
..
..
..
vrf customer2
routing bgp
as 65500
address-family ipv4-unicast
network 192.168.2.0/24
l3vpn export route-distinguisher 2:55
l3vpn export route-target 11:22
l3vpn export vpn true
l3vpn import route-target 11:22 route-target 22:44
l3vpn import vpn true
..
..
..
..
..
When using above configuration, it is mandatory to create BGP core instance. Also, despite RD and RT values could have been the same, it has been deliberately chosen to have distinct values to better understand the mechanisms put in place when dealing with L3VPN importation and exportation. Also, it is common, when a L3VPN setup is put in place between 2 ISPs, that the RD is self to each operator, while the RT will be chosen accordingly by operators.
Note
Above configuration details how routing information is exchanged, but does not explain in detail how data traffic is sent through. The product design choices opted for strongly isolating traffic across VPNs. To pass traffic across VPNs, it is required to create special interfaces by configuration. Those interfaces will make the connection between VPNs. More information will be given about how to create those interfaces in a separate chapter. Next chapters assume the user is familiar with interfaces used for crossing vrfs.
using IPless Virtual Ethernet interfaces¶
Using Cross-VRF interfaces to perform vrf route leaking with BGP requires a
specific semantic between VRs and interface names. VR naming must meet
the requirements of interface naming. Actually, the Cross-VRF interface name
chosen must be equal to the target VR the interface is connected to. To
illustrate, in order to reach VR foo from VR bar, an Cross-VRF interface
named foo has to be created in VR bar. Reversely, an Cross-VRF interface
named bar has to be created in VR foo. In this way, the interface foo
and the interface bar will be connected together. The naming convention
is not only done to reflect the intent of the interface. It is mandatory
to configure it in this way, if one wants to benefit from route leaking
across VRs, using Cross-VRF interfaces, and BGP.
From the user point of view, if a packet is emitted in one interface of a source VR, the same packet will be received in the associated veth interface of destination VR. More information about Cross-VRF interfaces can be found in XVRF Interface types. In order to have the setup working fine, the below configuration has to be appended to the former configuration of previous chapter (BGP VRF leak).
vrf customer1
interface xvrf customer2
link-interface customer1
link-vrf customer2
..
..
..
vrf customer2
interface xvrf customer1
link-interface customer2
link-vrf customer1
..
..
..
With the above configuration applied, VR route leaking is possible.
Subsequently, if BGP peering is done between a CE and the BGP instance
of each VR instance, then route importation and exportation occurs. Below
output demonstrates that the routes from customer2 have been imported to
customer1. The VR route leak are visible with the @1< indicating that
the route entry is originated from VR customer2.
Being able to interconnect between sites using L3VPN technology is now possible. As explained in the introduction of that chapter, the traffic between sites is encapsulated into an MPLS label. More exactly, BGP L3VPN will append a label that will stand for the encapsulation between 2 VPNs, in one direction. Adding to this, this label will be conveyed by an other framework. Currently, LDP offers that framework by conveying inner BGP labels in an outer LDP label. More information on chapter (LDP configuration).
rt1> show bgp vrf customer1 ipv4
BGP table version is 20, local router ID is 1.1.1.1, vrf id 2
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*>i10.101.0.0/24 1.1.1.2 100 0 i
*>i10.101.1.0/24 1.1.1.2 100 0 i
*>i10.101.2.0/24 1.1.1.2 100 0 i
[..]
*> 10.201.0.0/24 2.2.2.3@1< 100 0 i
*> 10.201.1.0/24 2.2.2.3@1< 100 0 i
*> 10.201.2.0/24 2.2.2.3@1< 100 0 i
[..]
Routing output¶
Once those entries selected in the bgp RIB, nothing prevents the
installation of those VRs routes in the underlying system. Packets going
from a VR to an other VR are using the Cross-VRF interface created. There
is no specific encapsulation, only a route using an interface as gateway.
From above example, the following output displays the routing entries
available in VRF routing table. As can be seen, the route to reach the
remote prefix first reaches the customer2 interface (first line directly
connected, customer2). Then, once the other VR reached, the original
BGP route of customer2 routes the traffic to the correct destination
(via 2.2.2.3 lines). The latter entry is only here for informational
purpose. To get information about routes in VR customer2, use the
associated show command to dump route entris in the associates VR.
rt1> show ipv4-routes vrf customer1
BGP table version is 20, local router ID is 1.1.1.1, vrf id 2
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
VRF customer1:
B>* 10.101.0.0/24 [200/0] via 1.1.1.2, eth1_0, 00:05:06
B>* 10.101.1.0/24 [200/0] via 1.1.1.2, eth1_0, 00:05:06
B>* 10.101.2.0/24 [200/0] via 1.1.1.2, eth1_0, 00:05:06
[..]
B>* 10.201.0.0/24 [200/0] is directly connected, customer2, 00:05:06
* via 2.2.2.3, eth2_0(vrf customer2), 00:05:06
B>* 10.201.1.0/24 [200/0] is directly connected, customer2, 00:05:06
* via 2.2.2.3, eth2_0(vrf customer2), 00:05:06
B>* 10.201.2.0/24 [200/0] is directly connected, customer2, 00:05:06
* via 2.2.2.3, eth2_0(vrf customer2), 00:05:06
[..]
how to interconnect traffic from a same VPN between sites.¶
Being able to interconnect between sites using L3VPN technology is now
possible. As explained in the introduction of that chapter, the traffic between
sites is encapsulated into an MPLS label, negotiated thanks to BGP. More
exactly, the ipv4-vpn address-family configured in BGP will obtain from
remote the label, and the underlay nexthop to use, to reach the remote. That
label will be the encapsulation between 2 VPNs, in one direction. The same
mechanism will apply in reverse direction. Adding to this, this label will be
conveyed by an other framework. Currently, LDP offers that framework by
conveying inner BGP labels in an outer MPLS label negoatiated by LDP.
More information on chapter (LDP configuration).
In order to activate L3VPN, use the following command under the main BGP
core instance. L3VPN address-family must be configured on the same VRF
where the LDP configuration is. Usually, the backbone is the main VR
instance.
vrf main
routing bgp
as 65500
neighbor 9.9.9.9 remote-as 65512
neighbor 9.9.9.9 update-source 3.3.3.3
neighbor 9.9.9.9 address-family ipv4-vpn enabled true
..
..
..
Consequently, having an L3VPN peering will trigger importation of L3VPN
entries. For instance, the presence of remote VPNs and associated prefixes
from peer 9.9.9.9 will trigger prefixes importation in the relevant VRs.
Those prefixes, if the remote VPNs match the local VPNs will be imported
in the associated VR. So as to permit that importation, the associated Cross-VRF
interfaces will be created between the main VR and the relevant VRs.
The following configuration illustrates the Cross-VRF interfaces.
vrf main
interface xvrf customer1
link-interface main
link-vrf customer1
..
..
interface xvrf customer2
link-interface main
link-vrf customer2
..
..
..
vrf customer1
interface xvrf main
link-interface customer1
link-vrf main
..
..
..
vrf customer2
interface xvrf main
link-interface customer2
link-vrf main
..
..
..
Also, in order for BGP to be able to export its own labels, BGP must be
configured so as to rely on its own labels, either automatically, or by
choosing its own. Below configuration illustrates the automatic label chosen
by VR customer1, while customer2 chooses to hardset its exportation
label to 300.
vrf customer1
routing bgp
as 65500
address-family ipv4-unicast
network 192.168.2.0/24
..
l3vpn export label auto
..
..
..
..
..
vrf customer2
routing bgp
as 65500
address-family ipv4-unicast
network 192.168.2.0/24
..
l3vpn export label 300
..
..
..
..
..
L3VPN setup using MPLS based framework¶
Above diagram illustrates a topology made up of an MPLS based backbone,
with LSR devices : rt3 and rt4, and LER devices. Each LER device
has a BGP core instance that has ipv4-vpn address-family enabled. Next
to each BGP instance, a BGP instance is created in each VR. Each VR
stands for a private network, either A or B. The color semantic explains
the relationship between the private networks on the various LER devices.
As can be seen with the arrows, each private network is geographically separate, but thanks to L3VPN, the private networks act as if there were only 2 specific private networks. The BGP configuration is depicted below. The LDP and OSPF configuration is out of scope of this chapter.
rt1
vrf main
routing bgp
router-id 5.5.5.5
as 65500
neighbor 9.9.9.9 remote-as 65500
neighbor 15.15.15.15 remote-as 65500
neighbor 9.9.9.9 update-source 5.5.5.5
neighbor 15.15.15.15 update-source 5.5.5.5
neighbor 9.9.9.9 address-family ipv4-vpn enabled true
neighbor 15.15.15.15 address-family ipv4-vpn enabled true
..
..
..
vrf customer1
routing bgp
router-id 1.1.1.1
as 65500
address-family ipv4-unicast
l3vpn export route-distinguisher 1:55
l3vpn export route-target 1:55
l3vpn export vpn true
l3vpn import route-target 1:55
l3vpn import vpn true
l3vpn export label auto
..
..
..
..
..
vrf customer2
routing bgp
router-id 2.2.2.2
as 65500
address-family ipv4-unicast
l3vpn export route-distinguisher 2:55
l3vpn export route-target 2:55
l3vpn export vpn true
l3vpn import route-target 2:55
l3vpn import vpn true
l3vpn export label auto
..
..
..
..
..
rt2
vrf main
routing bgp
router-id 9.9.9.9
as 65500
neighbor 5.5.5.5 remote-as 65500
neighbor 15.15.15.15 remote-as 65500
neighbor 5.5.5.5 update-source 9.9.9.9
neighbor 15.15.15.15 update-source 9.9.9.9
neighbor 5.5.5.5 address-family ipv4-vpn enabled true
neighbor 15.15.15.15 address-family ipv4-vpn enabled true
..
..
..
vrf customer1
routing bgp
router-id 1.1.1.10
as 65500
address-family ipv4-unicast
l3vpn export route-distinguisher 1:55
l3vpn export route-target 1:55
l3vpn export vpn true
l3vpn import route-target 1:55
l3vpn import vpn true
l3vpn export label auto
..
..
..
..
..
vrf customer2
routing bgp
router-id 2.2.2.2
as 65500
address-family ipv4-unicast
l3vpn export route-distinguisher 2:55
l3vpn export route-target 2:55
l3vpn export vpn true
l3vpn import route-target 2:55
l3vpn import vpn true
l3vpn export label auto
..
..
..
..
..
rt5
vrf main
routing bgp
router-id 15.15.15.15
as 65500
neighbor 5.5.5.5 remote-as 65500
neighbor 9.9.9.9 remote-as 65500
neighbor 5.5.5.5 update-source 15.15.15.15
neighbor 9.9.9.9 update-source 15.15.15.15
neighbor 5.5.5.5 address-family ipv4-vpn enabled true
neighbor 9.9.9.9 address-family ipv4-vpn enabled true
..
..
..
vrf customer1
routing bgp
router-id 1.1.1.20
as 65500
address-family ipv4-unicast
l3vpn export route-distinguisher 1:55
l3vpn export route-target 1:55
l3vpn export vpn true
l3vpn import route-target 1:55
l3vpn import vpn true
l3vpn export label auto
..
..
..
..
..
vrf customer2
routing bgp
router-id 2.2.2.20
as 65500
address-family ipv4-unicast
l3vpn export route-distinguisher 2:55
l3vpn export route-target 2:55
l3vpn export vpn true
l3vpn import route-target 2:55
l3vpn import vpn true
l3vpn export label 300
..
..
..
..
..
As can be seen, the network prefixes learnt by each BGP instance are
either learnt, by adding extra CE configuration linked to each instance, or
imported thanks to L3VPN technology. Below show dumps the vpnv4 entries:
rt1> show bgp ipv4 vpn
BGP table version is 32, local router ID is 5.5.5.5, vrf id 0
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 1:55
*> 10.101.0.0/24 1.1.1.2@2< 100 0 i
UN=1.1.1.2 EC{1:55} label=80 type=bgp, subtype=5
*> 10.101.1.0/24 1.1.1.2@2< 100 0 i
UN=1.1.1.2 EC{1:55} label=80 type=bgp, subtype=5
*> 10.101.2.0/24 1.1.1.2@2< 100 0 i
UN=1.1.1.2 EC{1:55} label=80 type=bgp, subtype=5
[..]
*>i10.101.11.0/24 9.9.9.9 100 0 i
UN=9.9.9.9 EC{1:55} label=80 type=bgp, subtype=0
*>i10.101.12.0/24 9.9.9.9 100 0 i
UN=9.9.9.9 EC{1:55} label=80 type=bgp, subtype=0
*>i10.101.13.0/24 9.9.9.9 100 0 i
UN=9.9.9.9 EC{1:55} label=80 type=bgp, subtype=0
[..]
*>i10.101.22.0/24 15.15.15.15 100 0 i
UN=15.15.15.15 EC{1:55} label=300 type=bgp, subtype=0
*>i10.101.23.0/24 15.15.15.15 100 0 i
UN=15.15.15.15 EC{1:55} label=300 type=bgp, subtype=0
*>i10.101.24.0/24 15.15.15.15 100 0 i
UN=15.15.15.15 EC{1:55} label=300 type=bgp, subtype=0
[..]
Route Distinguisher: 2:55
*> 10.201.0.0/24 2.2.2.3@1< 100 0 i
UN=2.2.2.3 EC{2:55} label=81 type=bgp, subtype=5
*> 10.201.1.0/24 2.2.2.3@1< 100 0 i
UN=2.2.2.3 EC{2:55} label=81 type=bgp, subtype=5
*> 10.201.2.0/24 2.2.2.3@1< 100 0 i
UN=2.2.2.3 EC{2:55} label=81 type=bgp, subtype=5
[..]
*>i10.201.11.0/24 9.9.9.9 100 0 i
UN=9.9.9.9 EC{2:55} label=81 type=bgp, subtype=0
*>i10.201.12.0/24 9.9.9.9 100 0 i
UN=9.9.9.9 EC{2:55} label=81 type=bgp, subtype=0
*>i10.201.13.0/24 9.9.9.9 100 0 i
UN=9.9.9.9 EC{2:55} label=81 type=bgp, subtype=0
[..]
As can be seen, all the entries have an underlay nexthop defined (UN)
that usually stands for the nexthop of the remote BGP global instance.
If local entries are imported, that nexthop stands will be sent to remote
BGP global instance. Along with the nexthop , an MPLS label will be
sent via BGP, and used on top of the MPLS backbone. MPLS stacking
will be performed. MPLS backbone will handle the LSP path.
It is worth to be noted too, that the first 3 visible entries are locally
exported entries. The nexthop to use 1.1.1.2 is located in the VR customer1.
The relationship between the VR name and the VR identifier displayed ( the
@2 value) can be done using the following command:
rt1> show bgp vrfs
Type Id routerId #PeersVfg #PeersEstb Name L3-VNI Rmac
DFLT 0 5.5.5.5 2 2 main 0 00:00:00:00:00:00
VRF 2 1.1.1.1 1 1 customer1 0 00:00:00:00:00:00
VRF 1 2.2.2.2 1 1 customer2 0 00:00:00:00:00:00
The label value 80 is the exported value sent to the remote BGP peers.
That value is received by remote BGP speaker 9.9.9.9 for instance, along
with the undelay network:
rt2> show bgp ipv4 vpn
BGP table version is 20, local router ID is 9.9.9.9, vrf id 0
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 1:55
*>i10.101.0.0/24 5.5.5.5 100 0 i
UN=5.5.5.5 EC{1:55} label=80 type=bgp, subtype=0
*>i10.101.1.0/24 5.5.5.5 100 0 i
UN=5.5.5.5 EC{1:55} label=80 type=bgp, subtype=0
*>i10.101.2.0/24 5.5.5.5 100 0 i
UN=5.5.5.5 EC{1:55} label=80 type=bgp, subtype=0
[..]
After having checked that L3VPN peering received the correct information,
it is possible to check against available entries in the bgp vrf customer1
instance like follows:
rt1> show bgp vrf customer1 ipv4
BGP table version is 32, local router ID is 1.1.1.1, vrf id 2
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*>i10.101.0.0/24 1.1.1.2 100 0 i
*>i10.101.1.0/24 1.1.1.2 100 0 i
*>i10.101.2.0/24 1.1.1.2 100 0 i
[..]
*> 10.101.11.0/24 9.9.9.9@0< 100 0 i
*> 10.101.12.0/24 9.9.9.9@0< 100 0 i
*> 10.101.13.0/24 9.9.9.9@0< 100 0 i
[..]
*> 10.101.22.0/24 15.15.15.15@0< 100 0 i
*> 10.101.23.0/24 15.15.15.15@0< 100 0 i
*> 10.101.24.0/24 15.15.15.15@0< 100 0 i
[..]
Like for the previous dump of vpn entries, it is worth to be noted that remote
vpn entries have their underlay nexthop visible, but annotated with @0<
indicating that this is a VR route leak going to the VPN. Only 1.1.1.2 ip
address is a locally reachable IP address. Actually, this is a CE of that
instance.
Routing output¶
Once those entries selected in the bgp RIB, nothing prevents the installation of those VR routes in the underlying system.
As remind, packets going from a VR to a remote VPN are first mpls
encapsulated once. Then, a second encapsulation takes place, as the backbone is
MPLS based. Then MPLS does the job to forward the packet to the nexthop marked
UN. Reversely, because the BGP vrf instance is at the LER place, the incoming
packets are only encapsulated with the negotiated BGP label. So, to go from the
backbone VR to the local VR, the packet is being popped its mpls label.
From above example, the following output can be extracted from the VR routing
table. As illustrated, the installed route entry performs a double MPLS
encapsulation (82/80). The inner value is the negotiated BGP value. The 82
value is an intermediate value that is being swapped by the negotiated MPLS value
on the backbone.
rt1> show ipv4-routes vrf customer1
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR,
> - selected route, * - FIB route
VRF customer1:
B>* 10.101.0.0/24 [200/0] via 1.1.1.2, eth1_0, 00:27:24
B>* 10.101.1.0/24 [200/0] via 1.1.1.2, eth1_0, 00:27:24
B>* 10.101.2.0/24 [200/0] via 1.1.1.2, eth1_0, 00:27:24
[..]
B>* 10.101.11.0/24 [200/0] is directly connected, main, label 82/80, 00:26:43
via 9.9.9.9(vrf main) (recursive), label 80, 00:26:43
* via 6.6.6.3, eth0_0(vrf main), label 17, 00:26:43
B>* 10.101.12.0/24 [200/0] is directly connected, main, label 82/80, 00:26:43
via 9.9.9.9(vrf main) (recursive), label 80, 00:26:43
* via 6.6.6.3, eth0_0(vrf main), label 17, 00:26:43
B>* 10.101.13.0/24 [200/0] is directly connected, main, label 82/80, 00:26:43
via 9.9.9.9(vrf main) (recursive), label 80, 00:26:43
* via 6.6.6.3, eth0_0(vrf main), label 17, 00:26:43
[..]
B>* 10.101.22.0/24 [200/0] is directly connected, main, label 84/300, 00:26:40
via 15.15.15.15(vrf main) (recursive), label 300, 00:26:40
* via 6.6.6.3, eth0_0(vrf main), label 22, 00:26:40
B>* 10.101.23.0/24 [200/0] is directly connected, main, label 84/300, 00:26:40
via 15.15.15.15(vrf main) (recursive), label 300, 00:26:40
* via 6.6.6.3, eth0_0(vrf main), label 22, 00:26:40
B>* 10.101.24.0/24 [200/0] is directly connected, main, label 84/300, 00:26:40
via 15.15.15.15(vrf main) (recursive), label 300, 00:26:40
* via 6.6.6.3, eth0_0(vrf main), label 22, 00:26:40
[..]
The next command dumps the LFIB of the backbone. As you can see, the 82 value
is replaced by the 17 value, and forwarded thanks to calculated LSP. Actually,
this entry stands for the path to rt2. For reverse direction, incoming packets
are being popped and redirected to the Cross-VRF interface leading to the VRF ( here
customer1).
rt1> show mpls-table
Inbound Outbound
Label Type Nexthop Label
-------- ------- --------------- --------
[..]
80 BGP customer1
81 BGP customer2
82 BGP 6.6.6.3 17
83 BGP 6.6.6.3 17
84 BGP 6.6.6.3 22
how to interconnect traffic from different VPNs between sites¶
By integrating examples of the 2 previous chapters, it becomes possible to perform interconnection between various VPNs between sites. Also, some VR route leaking across VPNs is done.
L3VPN setup using MPLS based framework with VR leaking.¶
The same topology as for previous chapter is chosen. Black arrows indicate
the traffic that will be authorised to pass between different VPNs. For
instance, network A and network B can access each other. Note that not
all data flow are illustrated in the drawing, but it is also possible to
do VR route leak between network A from rt1 and network B from rt1.
The L3VPN importation and exportation rules for a defined VPN apply for
that VPN, whatever its location is, ie local or remote.
Below configurations are BGP configuration changes. Usually, that kind of
configuration can be used, when some resources are shared with other VRs:
A video server located on that shared resource, or a management vr that is
only able to access to th other VRs.
rt1
vrf main
routing bgp
router-id 5.5.5.5
as 65500
neighbor 9.9.9.9 remote-as 65500
neighbor 15.15.15.15 remote-as 65500
neighbor 9.9.9.9 update-source 5.5.5.5
neighbor 15.15.15.15 update-source 5.5.5.5
neighbor 9.9.9.9 address-family ipv4-vpn enabled true
neighbor 15.15.15.15 address-family ipv4-vpn enabled true
..
..
..
vrf customer1
routing bgp
router-id 1.1.1.1
as 65500
address-family ipv4-unicast
maximum-path ebgp 4
maximum-path ibgp 4
l3vpn export route-distinguisher 1:55
l3vpn export route-target 1:55
l3vpn export vpn true
l3vpn import route-target 1:55 route-target 2:55
l3vpn import vpn true
l3vpn export label auto
..
..
..
..
..
vrf customer2
routing bgp
router-id 2.2.2.2
as 65500
address-family ipv4-unicast
l3vpn export route-distinguisher 2:55
l3vpn export route-target 2:55
l3vpn export vpn true
l3vpn import route-target 1:55 route-target 2:55
l3vpn import vpn true
l3vpn export label auto
..
..
..
..
rt2
vrf main
routing bgp
router-id 9.9.9.9
as 65500
neighbor 5.5.5.5 remote-as 65500
neighbor 15.15.15.15 remote-as 65500
neighbor 5.5.5.5 update-source 9.9.9.9
neighbor 15.15.15.15 update-source 9.9.9.9
neighbor 5.5.5.5 address-family ipv4-vpn enabled true
neighbor 15.15.15.15 address-family ipv4-vpn enabled true
..
..
..
vrf customer1
routing bgp
router-id 1.1.1.10
as 65500
address-family ipv4-unicast
l3vpn export route-distinguisher 1:55
l3vpn export route-target 1:55
l3vpn export vpn true
l3vpn import route-target 1:55 route-target 2:55
l3vpn import vpn true
l3vpn export label auto
..
..
..
..
..
vrf customer2
routing bgp
router-id 2.2.2.2
as 65500
address-family ipv4-unicast
l3vpn export route-distinguisher 2:55
l3vpn export route-target 2:55
l3vpn export vpn true
l3vpn import route-target 1:55 route-target 2:55
l3vpn import vpn true
l3vpn export label auto
..
..
..
..
..
rt5
vrf main
routing bgp
router-id 15.15.15.15
as 65500
neighbor 5.5.5.5 remote-as 65500
neighbor 9.9.9.9 remote-as 65500
neighbor 5.5.5.5 update-source 15.15.15.15
neighbor 9.9.9.9 update-source 15.15.15.15
neighbor 5.5.5.5 address-family ipv4-vpn enabled true
neighbor 9.9.9.9 address-family ipv4-vpn enabled true
..
..
vrf customer1
routing bgp
router-id 1.1.1.20
as 65500
address-family ipv4-unicast
l3vpn export route-distinguisher 1:55
l3vpn export route-target 1:55
l3vpn export vpn true
l3vpn import route-target 1:55 route-target 2:55
l3vpn import vpn true
l3vpn export label auto
..
..
..
..
..
vrf customer2
routing bgp
router-id 2.2.2.20
as 65500
address-family ipv4-unicast
l3vpn export route-distinguisher 2:55
l3vpn export route-target 2:55
l3vpn export vpn true
l3vpn import route-target 1:55 route-target 2:55
l3vpn import vpn true
l3vpn export label 300
..
..
..
..
..
The following show extract on rt1 dumps the routing entries of A and B
located on rt2. Two different MPLS labels chosen by BGP of rt2, are
received by rt1, for each network.
rt1> show bgp ipv4 vpn
BGP table version is 20, local router ID is 9.9.9.9, vrf id 0
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 2:55
[..]
*>i10.101.11.0/24 9.9.9.9 100 0 i
UN=9.9.9.9 EC{1:55} label=80 type=bgp, subtype=0
*>i10.101.12.0/24 9.9.9.9 100 0 i
UN=9.9.9.9 EC{1:55} label=80 type=bgp, subtype=0
*>i10.101.13.0/24 9.9.9.9 100 0 i
UN=9.9.9.9 EC{1:55} label=80 type=bgp, subtype=0
*>i10.201.11.0/24 9.9.9.9 100 0 i
UN=9.9.9.9 EC{2:55} label=81 type=bgp, subtype=0
*>i10.201.12.0/24 9.9.9.9 100 0 i
UN=9.9.9.9 EC{2:55} label=81 type=bgp, subtype=0
*>i10.201.13.0/24 9.9.9.9 100 0 i
UN=9.9.9.9 EC{2:55} label=81 type=bgp, subtype=0
When applied to the underlying system, the encapsulation for packets
leaving the customer1 VR is different, depending if the target prefix
belongs to A or B. 2 additional temporary labels are allocated and
are swapped as the show mpls labels indicate.
rt1> show ipv4-routes vrf customer1
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR,
> - selected route, * - FIB route
VRF customer1:
B>* 10.101.11.0/24 [200/0] is directly connected, main, label 83/80, 00:00:44
via 9.9.9.9(vrf main) (recursive), label 80, 00:00:44
* via 6.6.6.3, eth0_0(vrf main), label 18, 00:00:44
B>* 10.101.12.0/24 [200/0] is directly connected, main, label 83/80, 00:00:44
via 9.9.9.9(vrf main) (recursive), label 80, 00:00:44
* via 6.6.6.3, eth0_0(vrf main), label 18, 00:00:44
B>* 10.101.13.0/24 [200/0] is directly connected, main, label 83/80, 00:00:44
via 9.9.9.9(vrf main) (recursive), label 80, 00:00:44
* via 6.6.6.3, eth0_0(vrf main), label 18, 00:00:44
B>* 10.201.11.0/24 [200/0] is directly connected, main, label 82/81, 00:08:15
via 9.9.9.9(vrf main) (recursive), label 81, 00:08:15
* via 6.6.6.3, eth0_0(vrf main), label 19, 00:08:15
B>* 10.201.12.0/24 [200/0] is directly connected, main, label 82/81, 00:08:15
via 9.9.9.9(vrf main) (recursive), label 81, 00:08:15
* via 6.6.6.3, eth0_0(vrf main), label 19, 00:08:15
B>* 10.201.13.0/24 [200/0] is directly connected, main, label 82/81, 00:08:15
via 9.9.9.9(vrf main) (recursive), label 81, 00:08:15
* via 6.6.6.3, eth0_0(vrf main), label 19, 00:08:15
rt1> show mpls-table
Inbound Outbound
Label Type Nexthop Label
-------- ------- --------------- --------
80 BGP r1-cust1
81 BGP r1-cust2
82 BGP 6.6.6.3 18
83 BGP 6.6.6.3 18
84 BGP 6.6.6.3 18
85 BGP 6.6.6.3 18
An additional specificity of the setup is the possibility to import ECMP
entries coming from 2 separate locations; here, some 32 bit host routes are
retrieved. Some of the entries stand for a server with the same IP, but
geographically at 2 different places, namely rt2 and rt5. This kind of scenario
can be used for servers that require availability, or where load is split in two, to
avoid starvation of the resources of one of the machines.
rt1> show bgp ipv4 vpn
BGP table version is 20, local router ID is 9.9.9.9, vrf id 0
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*=i10.101.18.10/32 15.15.15.15 100 0 i
UN=15.15.15.15 EC{1:55} label=300 type=bgp, subtype=0
*>i 9.9.9.9 100 0 i
UN=9.9.9.9 EC{1:55} label=80 type=bgp, subtype=0
*=i10.101.19.10/32 15.15.15.15 100 0 i
UN=15.15.15.15 EC{1:55} label=300 type=bgp, subtype=0
*>i 9.9.9.9 100 0 i
UN=9.9.9.9 EC{1:55} label=80 type=bgp, subtype=0
*=i10.101.20.10/32 15.15.15.15 100 0 i
UN=15.15.15.15 EC{1:55} label=300 type=bgp, subtype=0
*>i 9.9.9.9 100 0 i
rt1> show ipv4-routes vrf customer1
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR,
> - selected route, * - FIB route
VRF customer1:
B>* 10.101.18.10/32 [20/0] is directly connected, main, label 19/80, 00:00:09
via 9.9.9.9(vrf main) (recursive), label 80, 00:00:09
* via 6.6.6.3, eth0_0(vrf main), label 20, 00:00:09
* is directly connected, main, label 22/300, 00:00:09
via 15.15.15.15(vrf main) (recursive), label 300, 00:00:09
* via 6.6.6.3, eth0_0(vrf main), label 21, 00:00:09
B>* 10.101.19.10/32 [20/0] is directly connected, main, label 19/80, 00:00:09
via 9.9.9.9(vrf main) (recursive), label 80, 00:00:09
* via 6.6.6.3, eth0_0(vrf main), label 20, 00:00:09
* is directly connected, main, label 22/300, 00:00:09
via 15.15.15.15(vrf main) (recursive), label 300, 00:00:09
* via 6.6.6.3, eth0_0(vrf main), label 21, 00:00:09
B>* 10.101.20.10/32 [20/0] is directly connected, main, label 19/80, 00:00:09
via 9.9.9.9(vrf main) (recursive), label 80, 00:00:09
* via 6.6.6.3, eth0_0(vrf main), label 20, 00:00:09
* is directly connected, main, label 22/300, 00:00:09
via 15.15.15.15(vrf main) (recursive), label 300, 00:00:09
* via 6.6.6.3, eth0_0(vrf main), label 21, 00:00:09