VPNs using BGP

Introduction

A VPN is a private network constructed within a public network infrastructure, such as the global Internet. In this chapter, the private network is an IP or layer 2 based network. BGP is the protocol that interconnects two or more VPNs across a public network. BGP has the necessary protocol extensions to piggy-backs the VPN information, and tells how traffic should be exchanged between VPNs.

See also

  • RFC 2764: A Framework for IP Based Virtual Private Networks

With the increasing usage of overlay information (widely used in data centers, but also deployed by operators, BGP evolves and is able to carry tunneling information. The following BGP services can help to interconnect VPNs: L3VPN and EVPN. Those two services rely on encapsulation techniques when traffic enters or leaves the VPN. The VPN generally sits on a given L3VRF, and the backbone is located on the default L3VRF.

Note

EVPN implements a layer 2 encapsulation technique, and can also convey layer 2 based networks. This technique is used by ISPs that want to provide network segmentation in VXLAN interfaces instead of using VLAN interfaces. In this case, the L3VRF instance is not used to handle that traffic, and default L3VRF is used instead.

The below chapters respectively introduce L3VPN and EVPN services. Both services share some similarities, and an additional chapter explains the concepts used to control and distribute routing information among VPNs.

L3VPN service

L3VPN is a VPN with an IP private network. In this chapter, the L3VPN BGP service exchanges MPLS labels which are used to encapsulate the IP traffic across the public network, and create the interconnection between VPNs. The encapsulation size is light compared to EVPN services. Using L3VPN requires to have an MPLS backbone or a GRE interface. The interconnection is achieved by creating route leaks.

See also

A basic configuration relies on a network defined in a given L3VRF, and the public network available on default L3VRF. Mainly, an MPLS backbone has to be available on the public network. Finally, BGP services can be set up above.

The below output illustrates the routing table for connecting customer1 L3VRF to a remote network 10.101.11.0/24. The resulting route is a L3VRF route leak and indicates which labels to push to IP packet to reach the remote network. The resulting traffic is an MPLS packet with two labels. In this scenario, the setup requires that an MPLS backbone be available between the devices where the VPNs are attached.

rt1> show ipv4-routes l3vrf customer1
 Codes: K - kernel route, C - connected, S - static, R - RIP,
        O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
        T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
        F - PBR,
        > - selected route, * - FIB route

 VRF customer1:
 [..]
 B>  10.101.11.0/24 [200/0] via 9.9.9.9(vrf vrf0) (recursive), label 80, 00:05:01
   *                          via 6.6.6.3, eth0_0(vrf main), label 20/80, 00:05:01

Conversely, when remote MPLS traffic is received, labels are popped. Resulting IP packet is routed into customer1 L3VRF. The below command dumps the pop action programmed by BGP. The nexthop attribute stands for the L3VRF interface and is the entry point to routing table of L3VRF instance.

rt1> show mpls-table
 Inbound Label  Type  Nexthop   Outbound Label
 -----------------------------------------------
 [..]
 80             BGP   customer1 -

The below figure depicts the MPLS encapsulation for forwarded packets going in and out of the L3VRF.

../../../../../_images/l3vpn-mpls-encapsulation.svg

L3VPN mpls encapsulation illustration

EVPN service

L2VPN is a VPN which interconnects private L2 networks together. In this chapter, the EVPN service is the specific BGP service implemented in Virtual Service Router. Ethernet-based traffic is encapsulated in a VXLAN header and sent to a remote network. The VNI value of VXLAN tunnels is exchanged by BGP to facilitate the interconnection between VPNs.

Configuration requires to bridge traffic to a VXLAN interface. Compared to the above L3VPN implementation, the encapsulation size is heavier, but no route leak is used.

See also

  • BGP MPLS-based Ethernet VPN, RFC 7432. MPLS encapsulation is not supported by Virtual Service Router.

  • A Network Virtualization Overlay Solution Using Ethernet VPN (EVPN), RFC 8365.

  • Integrated Routing and Bridging in Ethernet VPN, RFC 9135:

EVPN can be used to connect layer 2 domains. A basic configuration relies on a bridge with an L2 domain attached to one port, and a VXLAN interface used as a second port. The VXLAN encapsulation generally sits on a public network. Finally, BGP services can be set up above, to help for layer 2 interconnection.

To illustrate, BGP EVPN service is able to populate the layer 2 entries information received from the remote BGP session. The MAC address is conveyed by BGP signalisation. The below entry is created by BGP and populates the bridge FDB: The entry permits to switch outgoing traffic with destination mac address equal to the mac address of the entry; for the below example, the resulting outgoing traffic is a layer 2 frame which is encapsulated in the vxlan tunnel with VNI set to 2, and sent out to the remote endpoint 10.125.0.2.

rt1> show evpn mac vni 100
Number of MACs (local and remote) known for this VNI: 2
MAC               Type   Intf/Remote VTEP      VLAN  Seq #'s
[..]
be:21:5b:3e:20:7b remote 10.125.0.2                  0/0

EVPN can also be used to connect layer 3 networks. Routing information on the same L3VRF where the VXLAN interface is, can be routed via the VXLAN tunnel to reach a remote network. Here too, BGP is able to transmit that information to remote BGP clients. The below example gives an overview on how to reach a remote network 10.200.0.0/24 located behind the remote VTEP 10.125.0.2. The resulting output packet will be a VXLAN packet including the IP packet to forward; the layer 2 source mac addresses will be forged with the mac address of bridge1 interface and the layer 2 destination mac address will use the remote mac address of the remote bridge interface propagated by BGP.

rt1> show ipv4-route l3vrf customer1
 Codes: K - kernel route, C - connected, S - static, R - RIP,
        O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
        T - Table, A - Babel, D - SHARP, F - PBR, f - OpenFabric,
        > - selected route, * - FIB route, q - queued route, r - rejected route

 VRF customer1:
 [..]
 B>* 10.200.0.0/24 [20/0] via 10.125.0.2, bridge1 onlink, weight 1, 00:00:06

The below figure shows the VXLAN encapsulation for forwarded packets going in and out of the L3VRF.

../../../../../_images/evpn-vxlan-encapsulation.svg

EVPN vxlan encapsulation illustration

EVPN terminology

Route type 2, RT2 is a BGP EVPN MAC/IP advertisement route:

For a given L2 domain, this advertisement stands for a MAC entry availability in the remote VTEP. It may also contain an associated IP attached to that mac address. The output below of show bgp l2vpn evpn command illustrates what is a RT2 entry with the first letter of the line set to 2. Only the second entry includes an IP entry. For instance, for local VTEP, to reach host 10.50.0.1, an L2 frame should be forged with destination mac address de:ed:01:86:ce:15. This frame would be encapsulation with a VXLAN header with VNI set to 100 and destination IP set to 10.125.0.3.

 vsr> show bgp l2vpn evpn
 [..]
 Route Distinguisher: 10.125.0.3:2
 *>i[2]:[0]:[48]:[de:ed:01:14:d3:ee]
                     10.125.0.3                    100      0 i
                     RT:65500:100 ET:8
 *> [2]:[0]:[48]:[de:ed:01:86:ce:15]:[32]:[10.50.0.1]
                     10.125.0.1                         32768 i
                     ET:8 RT:65500:100
Route type 5, RT2 is a BGP EVPN IP prefix advertisement route:

For a given L3 routing domain, this prefix stands for an IP address in a given VPN. The below output of show bgp l2vpn evpn command illustrates what is a RT2 entry with the first letter of the line set to 5. The Rmac attribute is a BGP extended community and stands for the gateway mac address where this IP entry is reachable. For a remote endpoint, to reach the 10.100.0.0/24 sub-network, traffic should be encapsulated into VXLAN set with VNI set to 102, and destination address set to 10.125.0.1. The inner L2 destination MAC address is f2:d6:85:c3:5e:30.

 vsr> show bgp l2vpn evpn
 [..]
 Route Distinguisher: 65500:102
 *> [5]:[0]:[24]:[10.100.0.0]
                     10.125.0.1               0         32768 ?
                     ET:8 RT:65500:268435558 Rmac:f2:d6:85:c3:5e:30
 *> [5]:[0]:[32]:[10.100.0.1]
                     10.125.0.1               0         32768 ?
                     ET:8 RT:65500:268435558 Rmac:f2:d6:85:c3:5e:30
Route type 3, RT3 is a BGP EVPN inclusive multicast ethernet tag route:

For a given L2 VPN domain, a RT3 route may be sent to tell that BUM packets are authorized to be received and sent via the VXLAN interface.

Vxlan Network identifier, VNI:

VNI stands for the vxlan identifier contained in the vxlan header. Each vxlan tunnel has its own identifier, which is present in the VXLAN payload. A VXLAN interface can be used to carry L2 traffic, then we often refer to L2 VNI; when VXLAN interface is used to interconnect IP networks, we refer to L3 VNI.

Note

L3 VNI is limited to one per L3VRF, and needs to be manually referenced in the BGP configuration.

vsr running config# vrf main
vsr running vrf main# l3vrf customer1
vsr running l3vrf customer1#! table-id 10
vsr running l3vrf customer1# routing bgp
vsr running bgp# as 65500
vsr running bgp# l3vni 102
Virtual Termination Endpoint, VTEP:

VTEP stands for the vxlan interface endpoint. When creating a vxlan tunnel, only the source IP address of the tunnel, the destination port, and its VNI are given. BGP will populate the vxlan destination IP address of the tunnel thanks to the nexthop attribute of BGP updates. The forged tunnel is made of two endpoints called VTEPs.

Control and distribute routing information across BGP VPNs

Prefix identification is done by having a route distinguisher id (RD) in the prefix definition of the given VPN. Also, Prefixes distribution is filtered by BGP according to extended attributes named route-targets (RT), that are attached to the VPN prefixes.

Route Distinguisher, RD

This attribute is part of the prefixes originated from a given VPN. It is used by the PE routers to uniquely identify which VPN a packet belongs to. The RD value is encoded as a BGP extended community, the format is defined in RFC 4364. Generally, the AS number is used: for instance, 65500:12 can be seen as the VPN number 12 originated from AS 65500.

Route Target, RT

RTs are bgp extended communities that define which remote VPN to interconnect. One of the exported RTs sent to remote BGP endpoint with the BGP update and have to match with one of the imported RTs defined by the remote peer.

The below figure shows how 3 VPNs can interconnect with RTs values configured. Both customer1 and customer2 L3VRFs can access the sharedService L3VRF, and at the same there is no interconnection between customer1 and customer2.

../../../../../_images/route-target-explained.svg

It is not recommended to set the same values for both RD and RT. RD stands for a given VPN identifier of the local ISP. RT stands for the exchanged information with the remote BGP peer, and is set accordingly by operators.