VPNs using BGP

Introduction

A VPN is a private network constructed within a public network infrastructure, such as the global Internet. In this chapter, the private network is an IP or layer 2 based network. BGP is the protocol that interconnects two or more VPNs across a public network. BGP has the necessary protocol extensions to piggy-backs the VPN information, and tells how traffic should be exchanged between VPNs.

See also

  • RFC 2764: A Framework for IP Based Virtual Private Networks

With the increasing usage of overlay information (widely used in data centers, but also deployed by operators, BGP evolves and is able to carry tunneling information. The following BGP services can help to interconnect VPNs: L3VPN and EVPN. Those two services rely on encapsulation techniques when traffic enters or leaves the VPN. The VPN generally sits on a given L3VRF, and the backbone is located on the default L3VRF.

Note

EVPN implements a layer 2 encapsulation technique, and can also convey layer 2 based networks. This technique is used by ISPs that want to provide network segmentation in VXLAN interfaces instead of using VLAN interfaces. In this case, the L3VRF instance is not used to handle that traffic, and default L3VRF is used instead.

The below chapters respectively introduce L3VPN and EVPN services. Both services share some similarities, and an additional chapter explains the concepts used to control and distribute routing information among VPNs.

L3VPN service

L3VPN service is a VPN with an IP private network. The BGP L3VPN service tells traffic is exchanged with either MPLS labels or SRv6 encapsulation.

L3VPN MPLS service

In this chapter, the L3VPN BGP service exchanges MPLS labels which are used to encapsulate the IP traffic across the public network, and create the interconnection between VPNs. The encapsulation size is light compared to EVPN services. Using L3VPN requires to have an MPLS backbone or a GRE interface. The interconnection is achieved by creating route leaks.

See also

A basic configuration relies on a network defined in a given L3VRF, and the public network available on default L3VRF. Mainly, an MPLS backbone has to be available on the public network. Finally, BGP services can be set up above.

The below output illustrates the routing table for connecting customer1 L3VRF to a remote network 10.101.11.0/24. The resulting route is a L3VRF route leak and indicates which labels to push to IP packet to reach the remote network. The resulting traffic is an MPLS packet with two labels. In this scenario, the setup requires that an MPLS backbone be available between the devices where the VPNs are attached.

rt1> show ipv4-routes l3vrf customer1
 Codes: K - kernel route, C - connected, S - static, R - RIP,
        O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
        T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
        F - PBR,
        > - selected route, * - FIB route

 VRF customer1:
 [..]
 B>  10.101.11.0/24 [200/0] via 9.9.9.9(vrf vrf0) (recursive), label 80, 00:05:01
   *                          via 6.6.6.3, eth0_0(vrf main), label 20/80, 00:05:01

Conversely, when remote MPLS traffic is received, labels are popped. Resulting IP packet is routed into customer1 L3VRF. The below command dumps the pop action programmed by BGP. The nexthop attribute stands for the L3VRF interface and is the entry point to routing table of L3VRF instance.

rt1> show mpls-table
 Inbound Label  Type  Nexthop   Outbound Label
 -----------------------------------------------
 [..]
 80             BGP   customer1 -

The below figure depicts the MPLS encapsulation for forwarded packets going in and out of the L3VRF.

../../../../../_images/l3vpn-mpls-encapsulation.svg

L3VPN mpls encapsulation illustration

L3VPN SRV6 service

In this chapter, the L3VPN BGP service exchanges SRv6 SID values which are used to encapsulate the IP traffic across the public network. The public network must be an IPv6 network, and all L3VPN traffic is encapsulated in an IPv6 header which has a segment routing header (SRH) extension. Like for MPLS, the interconnection is achieved by creating route leaks, like it is done for L3VPN MPLS service.

See also

  • SRv6 packet format, see SRV6 SRH header.

  • SRv6 SID definition, see SRV6 address.

  • BGP Overlay Services Based on Segment Routing over IPv6, RFC 9252.

A basic configuration requires to define a locator on the router, so that the BGP service can pick up a SID value from that locator. The SID value must also be a routable IP address in the IPv6 core network. The below output illustrates the routing table for connecting green L3VRF to a remote 192.168.11.0/24 network. The resulting route is a L3VRF route leak and indicates that the 2001:db8:0:2:1:: SID value is used to encapsulate the inner IPv4 packet.

vsr> show ipv4-routes l3vrf green
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, A - Babel, D - SHARP, F - PBR, f - OpenFabric,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure

VRF green:
K>* 0.0.0.0/0 [255/8192] unreachable (ICMP unreachable), 16:17:26
C>* 192.168.10.0/24 is directly connected, eth1, 15:41:17
B>  192.168.11.0/24 [200/0] via 2001:db8:ffff::2 (vrf default) (recursive), label 16, seg6 2001:db8:0:2:1::, weight 1, 14:58:12
  *                           via fe80::ed1:12ff:feab:0, eth0 (vrf default), label 16, seg6 2001:db8:0:2:1::, weight 1, 14:58:12

Conversely, when the remote SRv6 traffic is received, the IPv6 header is decapsulated, and the resulting IP packet is routed to the table number 10, which stands for the table identifier of the green L3VRF. The green attribute stands for the L3VRF interface and is the entry point to routing table of L3VRF instance.

vsr> show ipv6-routes
Codes: K - kernel route, C - connected, S - static, R - RIPng,
       O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
       A - Babel, D - SHARP, F - PBR, f - OpenFabric,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure

[..]
B>* 2001:db8:0:1:1::/128 [20/0] is directly connected, green, seg6local End.DT4 table 10, weight 1, 16:18:13

EVPN service

L2VPN is a VPN which interconnects private L2 networks together. In this chapter, the EVPN service is the specific BGP service implemented in Virtual Service Router. Ethernet-based traffic is encapsulated in a VXLAN header and sent to a remote network. The VNI value of VXLAN tunnels is exchanged by BGP to facilitate the interconnection between VPNs.

Configuration requires to bridge traffic to a VXLAN interface. Compared to the above L3VPN implementation, the encapsulation size is heavier, but no route leak is used.

See also

  • BGP MPLS-based Ethernet VPN, RFC 7432. MPLS encapsulation is not supported by Virtual Service Router.

  • A Network Virtualization Overlay Solution Using Ethernet VPN (EVPN), RFC 8365.

  • Integrated Routing and Bridging in Ethernet VPN, RFC 9135:

EVPN can be used to connect layer 2 domains. A basic configuration relies on a bridge with an L2 domain attached to one port, and a VXLAN interface used as a second port. The VXLAN encapsulation generally sits on a public network. Finally, BGP services can be set up above, to help for layer 2 interconnection.

To illustrate, BGP EVPN service is able to populate the layer 2 entries information received from the remote BGP session. The MAC address is conveyed by BGP signalisation. The below entry is created by BGP and populates the bridge FDB: The entry permits to switch outgoing traffic with destination mac address equal to the mac address of the entry; for the below example, the resulting outgoing traffic is a layer 2 frame which is encapsulated in the vxlan tunnel with VNI set to 2, and sent out to the remote endpoint 10.125.0.2.

rt1> show evpn mac vni 100
Number of MACs (local and remote) known for this VNI: 2
MAC               Type   Intf/Remote VTEP      VLAN  Seq #'s
[..]
be:21:5b:3e:20:7b remote 10.125.0.2                  0/0

EVPN can also be used to connect layer 3 networks. Routing information on the same L3VRF where the VXLAN interface is, can be routed via the VXLAN tunnel to reach a remote network. Here too, BGP is able to transmit that information to remote BGP clients. The below example gives an overview on how to reach a remote network 10.200.0.0/24 located behind the remote VTEP 10.125.0.2. The resulting output packet will be a VXLAN packet including the IP packet to forward; the layer 2 source mac addresses will be forged with the mac address of bridge1 interface and the layer 2 destination mac address will use the remote mac address of the remote bridge interface propagated by BGP.

rt1> show ipv4-route l3vrf customer1
 Codes: K - kernel route, C - connected, S - static, R - RIP,
        O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
        T - Table, A - Babel, D - SHARP, F - PBR, f - OpenFabric,
        > - selected route, * - FIB route, q - queued route, r - rejected route

 VRF customer1:
 [..]
 B>* 10.200.0.0/24 [20/0] via 10.125.0.2, bridge1 onlink, weight 1, 00:00:06

The below figure shows the VXLAN encapsulation for forwarded packets going in and out of the L3VRF.

../../../../../_images/evpn-vxlan-encapsulation.svg

EVPN vxlan encapsulation illustration

EVPN terminology

Route type 2, RT2 is a BGP EVPN MAC/IP advertisement route:

For a given L2 domain, this advertisement stands for a MAC entry availability in the remote VTEP. It may also contain an associated IP attached to that mac address. The output below of show bgp l2vpn evpn command illustrates what is a RT2 entry with the first letter of the line set to 2. Only the second entry includes an IP entry. For instance, for local VTEP, to reach host 10.50.0.1, an L2 frame should be forged with destination mac address de:ed:01:86:ce:15. This frame would be encapsulation with a VXLAN header with VNI set to 100 and destination IP set to 10.125.0.3.

 vsr> show bgp l2vpn evpn
 [..]
 Route Distinguisher: 10.125.0.3:2
 *>i[2]:[0]:[48]:[de:ed:01:14:d3:ee]
                     10.125.0.3                    100      0 i
                     RT:65500:100 ET:8
 *> [2]:[0]:[48]:[de:ed:01:86:ce:15]:[32]:[10.50.0.1]
                     10.125.0.1                         32768 i
                     ET:8 RT:65500:100
Route type 5, RT2 is a BGP EVPN IP prefix advertisement route:

For a given L3 routing domain, this prefix stands for an IP address in a given VPN. The below output of show bgp l2vpn evpn command illustrates what is a RT2 entry with the first letter of the line set to 5. The Rmac attribute is a BGP extended community and stands for the gateway mac address where this IP entry is reachable. For a remote endpoint, to reach the 10.100.0.0/24 sub-network, traffic should be encapsulated into VXLAN set with VNI set to 102, and destination address set to 10.125.0.1. The inner L2 destination MAC address is f2:d6:85:c3:5e:30.

 vsr> show bgp l2vpn evpn
 [..]
 Route Distinguisher: 65500:102
 *> [5]:[0]:[24]:[10.100.0.0]
                     10.125.0.1               0         32768 ?
                     ET:8 RT:65500:268435558 Rmac:f2:d6:85:c3:5e:30
 *> [5]:[0]:[32]:[10.100.0.1]
                     10.125.0.1               0         32768 ?
                     ET:8 RT:65500:268435558 Rmac:f2:d6:85:c3:5e:30
Route type 3, RT3 is a BGP EVPN inclusive multicast ethernet tag route:

For a given L2 VPN domain, a RT3 route may be sent to tell that BUM packets are authorized to be received and sent via the VXLAN interface.

Vxlan Network identifier, VNI:

VNI stands for the vxlan identifier contained in the vxlan header. Each vxlan tunnel has its own identifier, which is present in the VXLAN payload. A VXLAN interface can be used to carry L2 traffic, then we often refer to L2 VNI; when VXLAN interface is used to interconnect IP networks, we refer to L3 VNI.

Note

L3 VNI is limited to one per L3VRF, and needs to be manually referenced in the BGP configuration.

vsr running config# vrf main
vsr running vrf main# l3vrf customer1
vsr running l3vrf customer1#! table-id 10
vsr running l3vrf customer1# routing bgp
vsr running bgp# as 65500
vsr running bgp# l3vni 102
Virtual Termination Endpoint, VTEP:

VTEP stands for the vxlan interface endpoint. When creating a vxlan tunnel, only the source IP address of the tunnel, the destination port, and its VNI are given. BGP will populate the vxlan destination IP address of the tunnel thanks to the nexthop attribute of BGP updates. The forged tunnel is made of two endpoints called VTEPs.

Control and distribute routing information across BGP VPNs

Prefix identification is done by having a route distinguisher id (RD) in the prefix definition of the given VPN. Also, Prefixes distribution is filtered by BGP according to extended attributes named route-targets (RT), that are attached to the VPN prefixes.

Route Distinguisher, RD

This attribute is part of the prefixes originated from a given VPN. It is used by the PE routers to uniquely identify which VPN a packet belongs to. The RD value is encoded as a BGP extended community, the format is defined in RFC 4364. Generally, the AS number is used: for instance, 65500:12 can be seen as the VPN number 12 originated from AS 65500.

Route Target, RT

RTs are bgp extended communities that define which remote VPN to interconnect. One of the exported RTs sent to remote BGP endpoint with the BGP update and have to match with one of the imported RTs defined by the remote peer.

The below figure shows how 3 VPNs can interconnect with RTs values configured. Both customer1 and customer2 L3VRFs can access the sharedService L3VRF, and at the same there is no interconnection between customer1 and customer2.

../../../../../_images/route-target-explained.svg

It is not recommended to set the same values for both RD and RT. RD stands for a given VPN identifier of the local ISP. RT stands for the exchanged information with the remote BGP peer, and is set accordingly by operators.