Note
High Availability IKE requires an IPsec Application License.
High Availability IKE¶
High Availability Internet Key Exchange (HA IKE) is an IKE extension that enables performing stateful synchronization of IKE between two HA nodes in active/backup mode.
HA IKE may be configured between two nodes forming an HA cluster: the IKE internal states (IKE SAs and CHILD SAs) and IPsec SAs sequence numbers are synchronized from the active node to the backup node.
If the HA state is switched between the two nodes, the new master node will be able to take over the IKE negotiations and IPsec dataplane traffic.
Overview¶
Configuring an HA IKE cluster¶
The HA state of a node can be statically configured using CLI/netconf or dynamically using the VRRP service.
HA IKE parameters are configured in the ha sub-context of ike.
Enter the ha sub-context on ha1:
ha1 running config# vrf main
ha1 running vrf main# ike
ha1 running ike# ha
ha1 running ha#!
Configure HA IKE parameters:
ha1 running ha#! local-address 10.150.0.1
ha1 running ha#! remote-address 10.150.0.2
ha1 running ha#! listen-ha-group ha-group1
ha1 running ha#
local-addressandremote-addressare the IPv4 or IPv6 addresses of the two HA nodes.listen-ha-groupis the high-availability group that controls the activity state of this HA node. See High-availability Groups for more information.
Display HA IKE parameters:
ha1 running ha# show config
ha
enabled true
listen-ha-group my-ha-group
local-address 10.150.0.1
remote-address 10.150.0.2
seqnum-sync
oseq-shift 65536
sync-period-time 10s
sync-period-packets 2
..
..
On ha2, the local-address and remote-address swapped:
ha2 running config# vrf main ike ha
ha2 running ha#! local-address 10.150.0.2
ha2 running ha#! remote-address 10.150.0.1
ha2 running ha#! listen-ha-group ha-group1
Specifying local-address and remote-address at the root of the ha context
only enables defining a single HA peer. It is recommended to use the more
flexible list of named peers instead. This list may contain 0, 1 or more peers:
ha1 running ha# peer ha2
ha1 running ha2# local-address 10.150.0.1
ha1 running ha2# remote-address 10.150.0.2
ha1 running ha2# ..
ha1 running ha# peer ha3
ha1 running ha3# local-address 10.150.0.1
ha1 running ha3# remote-address 10.150.0.3
ha1 running ha3# ..
The IKE HA service is started even if no peer is declared. If peers are declared later, the HA service will be able to synchronize its state without restarting the IKE service.
Note
The following parameters may be seamlessly modified without restarting the
IKE service: local-address, remote-address, the peer list and all
parameters in seqnum-sync.
However, modifying any of the other parameters in the ha context will
automatically restart the IKE service and hence tear down all established
IKE connections.
Displaying the state of HA IKE¶
To display the HA state of IKE, use the following command:
ha1> show ha-ike state
backup
To display the list of HA peers and their state, use the following command:
ha1> show ha-ike peer
Peer name Local address Remote address Role State
========= ============= ============== ==== =====
peer 10.150.0.1 10.150.0.2 client connected
Use case: L2 HA IKE cluster with VRRP¶
Layer 2 HA is a use case where the two HA nodes are directly connected on a local network, typically an ethernet network. The traffic that must be processed is steered to the master node at the layer 2 level, typically by an ethernet switch thanks to the VRRP protocol.
In this use case, two devices ha1 and ha2 are configured as a redundant security gateway, performing IKE negotiations with a remote security gateway SecGW.
The activity of each HA node is determined by the VRRP protocol (see VRRP command reference for details about VRRP): the VRRP protocol elects the master node depending on the link states and the priorities of nodes. It also controls the path of IP traffic by assigning VRRP virtual router MAC addresses (and optionally IP addresses) to the elected master node.
The two HA devices must be configured exactly the same, except for HA parameters (VRRP and HA IKE).
ha1 CLI configuration¶
Configure device hostname:
vsr running config# system hostname ha1
vsr running config# commit
Configuration committed.
Configure the HA group:
vsr running config# ha group ha-group1
ha1 running group ha-group1#! ..
ha1 running ha#! ..
Note
The HA group maintains the node high-availability state. It is controlled by
the VRRP protocol (via the notify-ha-group command) and monitored by HA IKE
(via the listen-ha-group command). Only one controller can be defined for
an HA group.
Move to VRF main configuration:
ha1 running config#! vrf main
ha1 running vrf main#!
Configure the network interfaces (adapt port ids to your hardware):
ha1 running vrf main#! interface physical eth1
ha1 running physical eth1#! port pci-b0s3
ha1 running physical eth1#! ipv4 address 10.22.0.1/24
ha1 running physical eth1#! ..
ha1 running interface#! physical eth2
ha1 running physical eth2#! port pci-b0s4
ha1 running physical eth2#! ipv4 address 10.23.0.1/24
ha1 running physical eth2#! ..
ha1 running interface#! physical eth3
ha1 running physical eth3#! port pci-b0s5
ha1 running physical eth3#! ipv4 address 10.150.0.1/24
ha1 running physical eth3#! ..
ha1 running interface#! loopback loopback0
ha1 running loopback loopback0#! ipv4 address 10.175.0.1/32
ha1 running loopback loopback0#! ..
ha1 running interface#! ..
Configure routes:
ha1 running vrf main#! routing
ha1 running routing#! static
ha1 running static#! ipv4-route 10.250.0.0/24 next-hop 10.200.0.1
ha1 running static#! ipv4-route 10.225.0.0/24 next-hop 10.200.0.1
ha1 running static#! ..
ha1 running routing#! ..
Configure VRRP:
ha1 running vrf main#! interface vrrp vrrp51
ha1 running vrrp vrrp52#! vrid 1
ha1 running vrrp vrrp51#! link-interface eth1
ha1 running vrrp vrrp51#! priority 100
ha1 running vrrp vrrp51#! advertisement-interval 1000
ha1 running vrrp vrrp51#! virtual-address 10.100.0.2/24
ha1 running vrrp vrrp51#! ..
ha1 running interface#! vrrp vrrp52
ha1 running vrrp vrrp52#! vrid 1
ha1 running vrrp vrrp52#! link-interface eth2
ha1 running vrrp vrrp52#! priority 100
ha1 running vrrp vrrp52#! advertisement-interval 1000
ha1 running vrrp vrrp52#! virtual-address 10.200.0.2/24
ha1 running vrrp vrrp52#! ..
ha1 running interface#! ..
ha1 running vrf main#! vrrp group group1
ha1 running group group1#! instance vrrp51
ha1 running group group1#! instance vrrp52
ha1 running group group1#! notify-ha-group ha-group1
ha1 running group group1# ..
ha1 running vrrp# ..
Show the configuration:
ha1 running vrf main# show config nodefault
vrf main
interface
vrrp vrrp51
link-interface eth1
vrid 1
virtual-address 10.100.0.2/24
..
vrrp vrrp52
link-interface eth2
vrid 1
virtual-address 10.200.0.2/24
..
..
vrrp
group group1
instance vrrp51
instance vrrp52
notify-ha-group ha-group1
..
..
..
Configure IKE:
ha1 running vrf main# ike
ha1 running ike# ike-policy-template ike1
ha1 running ike-policy-template ike1# ike-proposal 1 enc-alg aes128-cbc auth-alg hmac-sha1 dh-group modp1024
ha1 running ike-policy-template ike1# rekey-time 2h
ha1 running ike-policy-template ike1# ..
ha1 running ike# ipsec-policy-template ipsec1
ha1 running ipsec-policy-template ipsec1# esp-proposal 1 enc-alg aes128-cbc auth-alg hmac-sha1 esn true
ha1 running ipsec-policy-template ipsec1# rekey-time 1h
ha1 running ipsec-policy-template ipsec1# replay-window 1024
ha1 running ipsec-policy-template ipsec1# ..
ha1 running ike# vpn vpn-secgw
ha1 running vpn vpn-secgw#! ike-policy template ike1
ha1 running vpn vpn-secgw#! ipsec-policy template ipsec1
ha1 running vpn vpn-secgw# local-address 10.175.0.1
ha1 running vpn vpn-secgw# remote-address 10.225.0.1
ha1 running vpn vpn-secgw# security-policy site-to-secgw-site
ha1 running security-policy site-to-secgw-site# local-ts subnet 10.100.0.64/26
ha1 running security-policy site-to-secgw-site# remote-ts subnet 10.250.0.192/26
ha1 running security-policy site-to-secgw-site# ..
ha1 running vpn vpn-secgw# ..
ha1 running ike# pre-shared-key secgw
ha1 running pre-shared-key secgw#! id 10.225.0.1
ha1 running pre-shared-key secgw#! secret 0sBzAyaM5PTcnTHi/yRA1lARpAoRetSzP8
ha1 running pre-shared-key secgw# ..
ha1 running ike#
Show IKE configuration:
ha1 running ike# show config nodefault
ike
pre-shared-key secgw
id 10.225.0.1
secret 0sBzAyaM5PTcnTHi/yRA1lARpAoRetSzP8
..
global-options
dos-protection
..
sp-hash-ipv4
sp-hash-ipv6
..
ike-policy-template ike1
ike-proposal 1
enc-alg aes128-cbc
auth-alg hmac-sha1
dh-group modp1024
..
rekey-time 2h
..
ipsec-policy-template ipsec1
esp-proposal 1
enc-alg aes128-cbc
auth-alg hmac-sha1
esn true
..
replay-window 1024
..
vpn vpn-secgw
ike-policy
template ike1
..
ipsec-policy
template ipsec1
..
local-address 10.175.0.1
remote-address 10.225.0.1
security-policy site-to-secgw-site
local-ts subnet 10.100.0.64/26
remote-ts subnet 10.250.0.192/26
..
..
..
Configure HA IKE:
ha1 running ike# ha
ha1 running ha#! local-address 10.150.0.1
ha1 running ha#! remote-address 10.150.0.2
ha1 running ha#! listen-ha-group ha-group1
ha1 running ha# ..
ha1 running ike# commit
Configuration committed.
ha1 running ike#
Show HA IKE configuration:
ha1 running ike# show config nodefault ha
ha
listen-ha-group ha-group1
local-address 10.150.0.1
remote-address 10.150.0.2
seqnum-sync
..
..
ha2 CLI configuration¶
A similar configuration is used for ha2. The differences are the hostname, the physical interfaces addresses, VRRP parameters and IKE HA parameters.
The IKE parameters (except HA ones) must be strictly identical.
ha2 running config# show config nodefault
config
vrf main
interface
physical eth1
ipv4
address 10.22.0.2/24
..
..
physical eth2
ipv4
address 10.23.0.2/24
..
..
physical eth3
ipv4
address 10.150.0.2/24
..
..
loopback loopback0
ipv4
address 10.175.0.1/32
..
..
vrrp vrrp51
link-interface eth1
vrid 1
virtual-address 10.100.0.2/24
..
vrrp vrrp52
link-interface eth2
vrid 1
virtual-address 10.200.0.2/24
..
..
routing
static
ipv4-route 10.250.0.0/24
next-hop 10.200.0.1
..
ipv4-route 10.225.0.0/24
next-hop 10.200.0.1
..
..
..
vrrp
group group1
instance vrrp51
instance vrrp52
notify-ha-group ha-group1
..
..
ike
pre-shared-key secgw
id 10.225.0.1
secret 0sBzAyaM5PTcnTHi/yRA1lARpAoRetSzP8
..
global-options
dos-protection
..
sp-hash-ipv4
sp-hash-ipv6
..
ha
listen-ha-group ha-group1
local-address 10.150.0.2
remote-address 10.150.0.1
seqnum-sync
..
..
ike-policy-template ike1
ike-proposal 1
enc-alg aes128-cbc
auth-alg hmac-sha1
dh-group modp1024
..
rekey-time 2h
..
ipsec-policy-template ipsec1
esp-proposal 1
enc-alg aes128-cbc
auth-alg hmac-sha1
esn true
..
replay-window 1024
..
vpn vpn-secgw
ike-policy
template ike1
..
ipsec-policy
template ipsec1
..
local-address 10.175.0.1
remote-address 10.225.0.1
security-policy site-to-secgw-site
local-ts subnet 10.100.0.64/26
remote-ts subnet 10.250.0.192/26
..
..
..
..
system
hostname ha2
..
ha
group ha-group1
..
..
..
Use case: L3 HA IKE cluster with VRRP¶
Layer 3 HA is a use case where the HA nodes are not directly connected, they are on separate networks and can only contact each other via IP routing. The traffic that must be processed is steered to the master node at the layer 3 level, i.e. by routing, thanks to routing protocols. In our example, we use BGP.
The HA synchronization channel may be a simple routed TCP
connection, but should preferably go through a reliable and secured path,
possibly an IPsec-encrypted GRE tunnel or SVTI interface, as described
in Encrypting the synchronization messages, depending whether the network
between HA nodes is secured or not. In our example, we use a cross-VRF GRE
interface over an internal VLAN vlan_ha (without IPsec protection).
The VRRP protocol is used in unicast mode to elect the master and backup nodes
(see VRRP command reference for details about VRRP).
ha1 and ha2 exchange unicast point-to-point VRRP messages instead of
multicast ones. The VRRP protocol elects the master node depending on the
connectivity between nodes and their VRRP priorities. It also tracks the
link state of the HA synchronization channel, as well as the connectivity with
the routers to the trusted and untrusted networks, via the BFD protocol.
Finally, it installs a VRRP VIP on the master node (55.55.55.1), which is
the IKE and IPsec tunnel endpoint.
Only the master HA node owns the IPsec tunnel endpoint (55.55.55.1). The routes to the IPsec tunnel endpoint and to the remote networks discovered by IKE reverse route injection are announced only by the master node via BGP. However, in order to handle possible split brain situations (i.e. more than one node consider that they are master), one should make sure that preferences of the routes announced by different gateways match the order of the vrrp priorities. If necessary, route preferences may be influenced by using various BGP mechanisms such as AS-Path prepending or LOCAL_PREF.
To separate the trusted and untrusted networks, a VRF is created for each zone. This enables partitioning routing and makes sure crossing the untrusted network will require transiting via the redundant security gateway, that will provide IPsec protection.
In this example, a separate VRF and network is also created for the HA synchronization traffic, but it is not mandatory.
ha1 CLI configuration¶
Configure device hostname:
vsr running config# system hostname ha1
vsr running config# commit
Configuration committed.
In this example, ha1 only has one physical ethernet interface (eth1) and uses
VLANs in different VRFs to partition the trusted, untrusted and ha
networks.
Configure the physical network interface (adapt port ids to your hardware) in
VRF main:
ha1 running config# vrf main
ha1 running vrf main#! interface physical eth1
ha1 running physical eth1#! port pci-b0s4
ha1 running physical eth1# mtu 9000
ha1 running physical eth1# /
ha1 running config#
Then configure the VRF and VLAN for the trusted network:
ha1 running config# / vrf trusted interface
ha1 running interface# vlan vlan_trusted
ha1 running vlan vlan_trusted#! vlan-id 2
ha1 running vlan vlan_trusted#! link-interface eth1
ha1 running vlan vlan_trusted# link-vrf main
ha1 running vlan vlan_trusted# ipv4 address 10.2.0.1/30
ha1 running vlan vlan_trusted# /
ha1 running config#
Then configure the VRF and VLAN for the untrusted network:
ha1 running config# / vrf untrusted interface
ha1 running interface# vlan vlan_untrusted
ha1 running vlan vlan_untrusted#! vlan-id 307
ha1 running vlan vlan_untrusted#! link-interface eth1
ha1 running vlan vlan_untrusted# link-vrf main
ha1 running vlan vlan_untrusted# ipv4 address 10.55.0.2/30
ha1 running vlan vlan_untrusted# /
ha1 running config#
Then configure the VRF and VLAN for the HA synchronization network:
ha1 running config# / vrf ha interface
ha1 running interface# vlan vlan_ha
ha1 running vlan vlan_ha#! vlan-id 10
ha1 running vlan vlan_ha#! link-interface eth1
ha1 running vlan vlan_ha# link-vrf main
ha1 running vlan vlan_ha# ipv4 address 192.168.1.0/31
ha1 running vlan vlan_ha# /
ha1 running config#
In order to perform cross-VRF IPsec, the trusted traffic will be sent through
an SVTI cross-VRF interface in VRF trusted, with link-VRF untrusted:
ha1 running config# / vrf trusted interface
ha1 running interface# svti svti-trusted
ha1 running svti svti-trusted# svti-id 10
ha1 running svti svti-trusted# link-vrf untrusted
ha1 running svti svti-trusted# /
Now configure routing on the trusted network. The connectivity to the BGP peer
is tracked via the BFD protocol. The kernel routes of the trusted network are
redistributed via BGP. These are the routes to the remote trusted networks,
learnt by IKE reverse route injection. A route map TRUSTED-OUT is attached
to the peer, so that routes learnt by IKE reverse route injection are only
advertised to this peer when ha1 is master.
ha1 running config# / vrf trusted routing
ha1 running routing# bgp
ha1 running bgp# as 65111
ha1 running bgp# router-id 10.5.5.5
ha1 running bgp# ebgp-requires-policy false
ha1 running bgp# address-family ipv4-unicast redistribute kernel
ha1 running bgp# neighbor 10.2.0.2
ha1 running neighbor 10.2.0.2# remote-as 65123
ha1 running neighbor 10.2.0.2# address-family ipv4-unicast
ha1 running ipv4-unicast# as-override false
ha1 running ipv4-unicast# route-map out route-map-name TRUSTED-OUT
ha1 running running ipv4-unicast# ..
ha1 running running address-family# ..
ha1 running running neighbor 10.2.0.2# track bfd
ha1 running running neighbor 10.2.0.2# /
ha1 running config#
Now configure routing on the untrusted network. The connectivity to the BGP
peer is tracked via the BFD protocol. The IPsec tunnel endpoint
(55.55.55.1/32) is advertised by BGP. A route map UNTRUSTED-OUT is attached
to the peer, so that the route to the IPsec tunnel endpoint is only advertised
when ha1 is master.
ha1 running config# / vrf untrusted routing
ha1 running routing# bgp
ha1 running bgp# as 65111
ha1 running bgp# router-id 10.5.5.5
ha1 running bgp# ebgp-requires-policy false
ha1 running bgp# address-family ipv4-unicast network 55.55.55.1/32
ha1 running bgp# neighbor 10.55.0.1
ha1 running neighbor 10.55.0.1# remote-as 65123
ha1 running neighbor 10.55.0.1# address-family ipv4-unicast
ha1 running ipv4-unicast# as-override false
ha1 running ipv4-unicast# route-map out route-map-name UNTRUSTED-OUT
ha1 running running ipv4-unicast# ..
ha1 running running address-family# ..
ha1 running neighbor 10.55.0.1# track bfd
ha1 running neighbor 10.55.0.1# /
ha1 running config#
The ha network is an internal network that interconnects the HA nodes and
enables exchanging synchronization data. We chose to use a separate VRF and
to access it via a different VLAN. However, the IKE service runs in VRF
untrusted, hence the HA module needs to establish a TCP connection with the
HA peer in VRF untrusted. We therefore create a cross-VRF GRE interface
in VRF untrusted, with link-VRF ha. The HA synchronization messages will
be exchanged through this GRE tunnel, and the GRE- encapsulated packets
will travel in VRF ha:
ha1 running config# / vrf untrusted interface
ha1 running interface# gre gre-hasync
ha1 running gre gre-hasync# ttl 64
ha1 running gre gre-hasync# link-vrf ha
ha1 running gre gre-hasync# local 192.168.1.0
ha1 running gre gre-hasync# remote 192.168.1.1
ha1 running gre gre-hasync# key both 888
ha1 running gre gre-hasync# ipv4 address 192.168.2.0/31
ha1 running gre gre-hasync# /
ha1 running config#
Create a loopback interface whose address is the BGP and VRRP router-ids:
ha1 running config# / vrf untrusted interface
ha1 running interface# loopback loopback0
ha1 running loopback loopback0# ipv4 address 10.5.5.5/32
ha1 running loopback loopback0# /
ha1 running config#
Configure the route maps used by the BGP configurations so that the routes to the IPsec tunnel endpoint and to the remote networks discovered by IKE reverse route injection are announced only by the master:
ha1 running config# / routing
ha1 running routing# ipv4-prefix-list IPSEC_TEP
ha1 running ipv4-prefix-list IPSEC_TEP# seq 10 address 55.55.55.1/32 policy permit
ha1 running ipv4-prefix-list IPSEC_TEP# ..
ha1 running routing# ipv4-prefix-list RAN
ha1 running ipv4-prefix-list RAN# seq 10 address 10.7.0.0/24 policy permit le 32
ha1 running ipv4-prefix-list RAN# ..
ha1 running routing# route-map UNTRUSTED-OUT
ha1 running route-map UNTRUSTED-OUT# seq 10
ha1 running route-map seq 10# policy permit
ha1 running route-map seq 10# match ip address prefix-list IPSEC_TEP
ha1 running route-map seq 10# match ha-group name ha_ipsec
ha1 running route-map seq 10# ..
ha1 running route-map UNTRUSTED-OUT# ..
ha1 running routing# route-map TRUSTED-OUT
ha1 running route-map TRUSTED-OUT# seq 10
ha1 running route-map seq 10# policy permit
ha1 running route-map seq 10# match ip address prefix-list RAN
ha1 running route-map seq 10# match ha-group name ha_ipsec
ha1 running route-map seq 10# ..
ha1 running route-map TRUSTED-OUT# /
ha1 running config#
Create two trackers that monitor the connectivity with the routers to the trusted and untrusted networks:
ha1 running config# / tracker bfd pe-bfd-trusted
ha1 running bfd pe-bfd-trusted# source 10.2.0.1
ha1 running bfd pe-bfd-trusted# address 10.2.0.2
ha1 running bfd pe-bfd-trusted# vrf trusted
ha1 running bfd pe-bfd-trusted# ..
ha1 running config# / tracker bfd pe-bfd-untrusted
ha1 running bfd pe-bfd-untrusted# source 10.55.0.2
ha1 running bfd pe-bfd-untrusted# address 10.55.0.1
ha1 running bfd pe-bfd-untrusted# vrf untrusted
ha1 running bfd pe-bfd-untrusted# /
ha1 running config#
We can finally configure HA. First configure the HA group:
ha1 running config# / ha group ha_ipsec
ha1 running group ha_ipsec#! ..
ha1 running ha#! /
ha1 running config#!
Note
The HA group maintains the node high-availability state. It is controlled by
the VRRP protocol (via the notify-ha-group command) and monitored by HA IKE
(via the listen-ha-group command). Only one controller can be defined for
an HA group.
Configure VRRP: VRRP packets are sent over the gre-hasync interface in
unicast mode. VRRP tracks the link state of the gre-hasync interface, as
well as the connectivity with the routers to the trusted and untrusted networks,
via the BFD protocol. Finally, it installs a VRRP VIP on the master node
(55.55.55.1), which is the IKE and IPsec tunnel endpoint. It notifies the
services interested by the HA status via the HA group ha_ipsec (IKE and
routing):
ha1 running config# / vrf untrusted interface
ha1 running interface# vrrp vrrp_ha
ha1 running vrrp vrrp_ha# version 3
ha1 running vrrp vrrp_ha# link-interface gre-hasync
ha1 running vrrp vrrp_ha# use-vmac false
ha1 running vrrp vrrp_ha# vrid 10
ha1 running vrrp vrrp_ha# priority 200
ha1 running vrrp vrrp_ha# preempt-delay 30
ha1 running vrrp vrrp_ha# advertisement-interval 100
ha1 running vrrp vrrp_ha# track pe-bfd-trusted
ha1 running vrrp vrrp_ha# track pe-bfd-untrusted
ha1 running vrrp vrrp_ha# track-fast-path true
ha1 running vrrp vrrp_ha# unicast-peer 192.168.2.1
ha1 running vrrp vrrp_ha# virtual-address 55.55.55.1/32
ha1 running vrrp vrrp_ha# notify-ha-group ha_ipsec
ha1 running vrrp vrrp_ha# /
ha1 running config# / vrf untrusted vrrp
ha1 running vrrp# router-id 10.5.5.5
ha1 running vrrp# vrrp-startup-delay 30
ha1 running vrrp# /
ha1 running config#
Now that network connectivity and routing are configured, let us configure IKE.
Here, ha1 is configured as a VPN concentrator that accepts IKE connections
initiated by a remote peer. To support cross-VRF IPsec, the cross-VRF SVTI
interface svti-trusted is used (VRF trusted, link-VRF untrusted, svti-id 10).
The inject-routes option automatically adds routes to the remote traffic
selector via the SVTI interface (reverse route injection).
Configure IKE:
ha1 running config# / vrf untrusted ike
ha1 running ike# ike-policy-template nodeB
ha1 running ike-proposal-1# enc-alg aes256-cbc
ha1 running ike-proposal-1# auth-alg hmac-sha256
ha1 running ike-proposal-1# dh-group modp2048
ha1 running ike-proposal-1# ...
ha1 running ike-policy-template nodeB# unique-sa replace
ha1 running ike-policy-template nodeB# rekey-time 2800
ha1 running ike-policy-template nodeB# dpd-delay 10
ha1 running ike-policy-template nodeB# ..
ha1 running ike# ipsec-policy-template nodeB
ha1 running esp-proposal 1# esp-proposal 1
ha1 running esp-proposal 1# enc-alg aes256-cbc
ha1 running esp-proposal 1# auth-alg hmac-sha256
ha1 running esp-proposal 1# ..
ha1 running ipsec-policy-template nodeB# start-action none
ha1 running ipsec-policy-template nodeB# close-action none
ha1 running ipsec-policy-template nodeB# dpd-action clear
ha1 running ipsec-policy-template nodeB# rekey-time 2800
ha1 running ipsec-policy-template nodeB# ..
ha1 running ike# vpn nodeB
ha1 running vpn nodeB# ike-policy template nodeB
ha1 running vpn nodeB# ipsec-policy template nodeB
ha1 running vpn nodeB# local-address 55.55.55.1
ha1 running vpn nodeB# security-policy nodeB
ha1 running security-policy nodeB# local-ts subnet 10.77.77.0/24
ha1 running security-policy nodeB# remote-ts subnet 10.7.0.0/24
ha1 running security-policy nodeB# svti-id-in 10
ha1 running security-policy nodeB# svti-id-out 10
ha1 running security-policy nodeB# inject-routes
ha1 running inject-routes# next-hop svti-trusted
ha1 running inject-routes# vrf trusted
ha1 running inject-routes# ..
ha1 running security-policy nodeB# ..
ha1 running vpn nodeB# ..
ha1 running ike# pre-shared-key femto
ha1 running pre-shared-key femto# secret 0x12345678
ha1 running pre-shared-key femto# ..
ha1 running ike# /
ha1 running config#
Configure IKE HA:
ha1 running config#! / vrf untrusted ike
ha1 running ike#! ha
ha1 running ha#! listen-ha-group ha_ipsec
ha1 running ha#! local-address 192.168.2.0
ha1 running ha#! remote-address 192.168.2.1
ha1 running ha# /
ha1 running config#
Once everything is configured, commit the configuration:
ha1 running config#
ha1 running config# commit
Configuration committed.
ha2 CLI configuration¶
Configure device hostname:
vsr running config# system hostname ha2
vsr running config# commit
Configuration committed.
In this example, ha2 only has one physical ethernet interface (eth1) and uses
VLANs in different VRFs to partition the trusted, untrusted and ha
networks.
Configure the physical network interface (adapt port ids to your hardware) in
VRF main:
ha2 running config# vrf main
ha2 running vrf main#! interface physical eth1
ha2 running physical eth1#! port pci-b0s4
ha2 running physical eth1# mtu 9000
ha2 running physical eth1# /
ha2 running config#
Then configure the VRF and VLAN for the trusted network:
ha2 running config# / vrf trusted interface
ha2 running interface# vlan vlan_trusted
ha2 running vlan vlan_trusted#! vlan-id 2
ha2 running vlan vlan_trusted#! link-interface eth1
ha2 running vlan vlan_trusted# link-vrf main
ha2 running vlan vlan_trusted# ipv4 address 10.2.0.1/30
ha2 running vlan vlan_trusted# /
ha2 running config#
Then configure the VRF and VLAN for the untrusted network:
ha2 running config# / vrf untrusted interface
ha2 running interface# vlan vlan_untrusted
ha2 running vlan vlan_untrusted#! vlan-id 307
ha2 running vlan vlan_untrusted#! link-interface eth1
ha2 running vlan vlan_untrusted# link-vrf main
ha2 running vlan vlan_untrusted# ipv4 address 10.55.0.2/30
ha2 running vlan vlan_untrusted# /
ha2 running config#
Then configure the VRF and VLAN for the HA synchronization network:
ha2 running config# / vrf ha interface
ha2 running interface# vlan vlan_ha
ha2 running vlan vlan_ha#! vlan-id 10
ha2 running vlan vlan_ha#! link-interface eth1
ha2 running vlan vlan_ha# link-vrf main
ha2 running vlan vlan_ha# ipv4 address 192.168.1.1/31
ha2 running vlan vlan_ha# /
ha2 running config#
In order to perform cross-VRF IPsec, the trusted traffic will be sent through
an SVTI cross-VRF interface in VRF trusted, with link-VRF untrusted:
ha2 running config# / vrf trusted interface
ha2 running interface# svti svti-trusted
ha2 running svti svti-trusted# svti-id 10
ha2 running svti svti-trusted# link-vrf untrusted
ha2 running svti svti-trusted# /
Now configure routing on the trusted network. The connectivity to the BGP peer
is tracked via the BFD protocol. The kernel routes of the trusted network are
redistributed via BGP. These are the routes to the remote trusted networks,
learnt by IKE reverse route injection. A route map TRUSTED-OUT is attached
to the peer, so that routes learnt by IKE reverse route injection are only
advertised to this peer when ha2 is master.
ha2 running config# / vrf trusted routing
ha2 running routing# bgp
ha2 running bgp# as 65111
ha2 running bgp# router-id 10.6.6.6
ha2 running bgp# ebgp-requires-policy false
ha2 running bgp# address-family ipv4-unicast redistribute kernel
ha2 running bgp# neighbor 10.0.0.2
ha2 running neighbor 10.2.0.2# remote-as 65123
ha2 running neighbor 10.2.0.2# address-family ipv4-unicast
ha2 running ipv4-unicast# as-override false
ha2 running ipv4-unicast# route-map out route-map-name TRUSTED-OUT
ha2 running running ipv4-unicast# ..
ha2 running running address-family# ..
ha2 running running neighbor 10.2.0.2# track bfd
ha2 running running neighbor 10.2.0.2# /
ha2 running config#
Now configure routing on the untrusted network. The connectivity to the BGP
peer is tracked via the BFD protocol. The IPsec tunnel endpoint
(55.55.55.1/32) is advertised by BGP. A route map UNTRUSTED-OUT is attached
to the peer, so that the route to the IPsec tunnel endpoint is only advertised
when ha2 is master.
ha2 running config# / vrf untrusted routing
ha2 running routing# bgp
ha2 running bgp# as 65111
ha2 running bgp# router-id 10.6.6.6
ha2 running bgp# ebgp-requires-policy false
ha2 running bgp# address-family ipv4-unicast network 55.55.55.1/32
ha2 running bgp# neighbor 10.55.0.1
ha2 running neighbor 10.55.0.1# remote-as 65123
ha2 running neighbor 10.55.0.1# address-family ipv4-unicast
ha2 running ipv4-unicast# as-override false
ha2 running ipv4-unicast# route-map out route-map-name UNTRUSTED-OUT
ha2 running running ipv4-unicast# ..
ha2 running running address-family# ..
ha2 running neighbor 10.55.0.1# track bfd
ha2 running neighbor 10.55.0.1# /
ha2 running config#
The ha network is an internal network that interconnects the HA nodes and
enables exchanging synchronization data. We chose to use a separate VRF and
to access it via a different VLAN. However, the IKE service runs in VRF
untrusted and the HA module needs to establish a TCP connection with the
HA peer in VRF untrusted. We therefore create a cross-VRF GRE interface
in VRF untrusted, with link-VRF ha The HA synchronization messages will
be exchanged through this GRE tunnel, and the GRE- encapsulated packets
will travel in VRF ha:
ha2 running config# / vrf untrusted interface
ha2 running interface# gre gre-hasync
ha2 running gre gre-hasync# ttl 64
ha2 running gre gre-hasync# link-vrf ha
ha2 running gre gre-hasync# local 192.168.1.1
ha2 running gre gre-hasync# remote 192.168.1.0
ha2 running gre gre-hasync# key both 888
ha2 running gre gre-hasync# ipv4 address 192.168.2.1/31
ha2 running gre gre-hasync# /
ha2 running config#
Create a loopback interface whose address is the BGP and VRRP router-ids:
ha2 running config# / vrf untrusted interface
ha2 running interface# loopback loopback0
ha2 running loopback loopback0# ipv4 address 10.6.6.6/32
ha2 running loopback loopback0# /
ha2 running config#
Configure the route maps used by the BGP configurations so that the routes to the IPsec tunnel endpoint and to the remote networks discovered by IKE reverse route injection are announced only by the master. The cost of these routes is artificially increased by prepending the local AS to the AS-Path, so that the route via ha1 is preferred to the route to ha2 in case of split brain:
ha2 running config# / routing
ha2 running routing# ipv4-prefix-list IPSEC_TEP
ha2 running ipv4-prefix-list IPSEC_TEP# seq 10 address 55.55.55.1/32 policy permit
ha2 running ipv4-prefix-list IPSEC_TEP# ..
ha2 running routing# ipv4-prefix-list RAN
ha2 running ipv4-prefix-list RAN# seq 10 address 10.7.0.0/24 policy permit le 32
ha2 running ipv4-prefix-list RAN# ..
ha2 running routing# route-map UNTRUSTED-OUT
ha2 running route-map UNTRUSTED-OUT# seq 10
ha2 running route-map seq 10# policy permit
ha2 running route-map seq 10# match ip address prefix-list IPSEC_TEP
ha2 running route-map seq 10# match ha-group name ha_ipsec
ha2 running route-map seq 10# ..
ha2 running route-map UNTRUSTED-OUT# seq 20
ha2 running route-map seq 20# policy permit
ha2 running route-map seq 20# match ip address prefix-list IPSEC_TEP
ha2 running route-map seq 20# set as-path prepend asn 1 65111
ha2 running route-map seq 20# ..
ha2 running route-map UNTRUSTED-OUT# ..
ha2 running routing# route-map TRUSTED-OUT
ha2 running route-map TRUSTED-OUT# seq 10
ha2 running route-map seq 10# policy permit
ha2 running route-map seq 10# match ip address prefix-list RAN
ha2 running route-map seq 10# match ha-group name ha_ipsec
ha2 running route-map seq 10# ..
ha2 running route-map TRUSTED-OUT# seq 20
ha2 running route-map seq 20# policy permit
ha2 running route-map seq 20# match ip address prefix-list RAN
ha2 running route-map seq 20# set as-path prepend asn 1 65111
ha2 running route-map seq 20# ..
ha2 running route-map TRUSTED-OUT# /
ha2 running config#
Create two trackers that monitor the connectivity with the routers to the trusted and untrusted networks:
ha2 running config# / tracker bfd pe-bfd-trusted
ha2 running bfd pe-bfd-trusted# source 10.2.0.1
ha2 running bfd pe-bfd-trusted# address 10.2.0.2
ha2 running bfd pe-bfd-trusted# vrf trusted
ha2 running bfd pe-bfd-trusted# ..
ha2 running config# / tracker bfd pe-bfd-untrusted
ha2 running bfd pe-bfd-untrusted# source 10.55.0.2
ha2 running bfd pe-bfd-untrusted# address 10.55.0.1
ha2 running bfd pe-bfd-untrusted# vrf untrusted
ha2 running bfd pe-bfd-untrusted# /
ha2 running config#
We can finally configure HA. First configure the HA group:
ha2 running config# / ha group ha_ipsec
ha2 running group ha_ipsec#! ..
ha2 running ha#! /
ha2 running config#!
Note
The HA group maintains the node high-availability state. It is controlled by
the VRRP protocol (via the notify-ha-group command) and monitored by HA IKE
(via the listen-ha-group command). Only one controller can be defined for
an HA group.
Configure VRRP: VRRP packets are sent over the gre-hasync interface in
unicast mode. VRRP tracks the link state of the gre-hasync interface, as
well as the connectivity with the routers to the trusted and untrusted networks,
via the BFD protocol. Finally, it installs a VRRP VIP on the master node
(55.55.55.1), which is the IKE and IPsec tunnel endpoint. It notifies the
services interested by the HA status via the HA group ha_ipsec (IKE and
routing):
ha2 running config# / vrf untrusted interface
ha2 running interface# vrrp vrrp_ha
ha2 running vrrp vrrp_ha# version 3
ha2 running vrrp vrrp_ha# link-interface gre-hasync
ha2 running vrrp vrrp_ha# use-vmac false
ha2 running vrrp vrrp_ha# vrid 10
ha2 running vrrp vrrp_ha# priority 100
ha2 running vrrp vrrp_ha# preempt-delay 30
ha2 running vrrp vrrp_ha# advertisement-interval 100
ha2 running vrrp vrrp_ha# track pe-bfd-trusted
ha2 running vrrp vrrp_ha# track pe-bfd-untrusted
ha2 running vrrp vrrp_ha# track-fast-path true
ha2 running vrrp vrrp_ha# unicast-peer 192.168.2.0
ha2 running vrrp vrrp_ha# virtual-address 55.55.55.1/32
ha2 running vrrp vrrp_ha# notify-ha-group ha_ipsec
ha2 running vrrp vrrp_ha# /
ha2 running config# / vrf untrusted vrrp
ha2 running vrrp# router-id 10.6.6.6
ha2 running vrrp# vrrp-startup-delay 30
ha2 running vrrp# /
ha2 running config#
Now that network connectivity and routing are configured, let us configure IKE.
Here, ha2 is configured as a VPN concentrator that accepts IKE connections
initiated by a remote peer. To support cross-VRF IPsec, the cross-VRF SVTI
interface svti-trusted is used (VRF trusted, link-VRF untrusted, svti-id 10).
The inject-routes option automatically adds routes to the remote traffic
selector via the SVTI interface (reverse route injection).
Configure IKE:
ha2 running config# / vrf untrusted ike
ha2 running ike# ike-policy-template nodeB
ha2 running ike-proposal-1# enc-alg aes256-cbc
ha2 running ike-proposal-1# auth-alg hmac-sha25
ha2 running ike-proposal-1# dh-group modp2048
ha2 running ike-proposal-1# ...
ha2 running ike-policy-template nodeB# unique-sa replace
ha2 running ike-policy-template nodeB# rekey-time 2800
ha2 running ike-policy-template nodeB# dpd-delay 10
ha2 running ike-policy-template nodeB# ..
ha2 running ike# ipsec-policy-template nodeB
ha2 running esp-proposal 1# esp-proposal 1
ha2 running esp-proposal 1# enc-alg aes256-cbc
ha2 running esp-proposal 1# auth-alg hmac-sha256
ha2 running esp-proposal 1# ..
ha2 running ipsec-policy-template nodeB# start-action none
ha2 running ipsec-policy-template nodeB# close-action none
ha2 running ipsec-policy-template nodeB# dpd-action clear
ha2 running ipsec-policy-template nodeB# rekey-time 2800
ha2 running ipsec-policy-template nodeB# ..
ha2 running ike# vpn nodeB
ha2 running vpn nodeB# ike-policy template nodeB
ha2 running vpn nodeB# ipsec-policy template nodeB
ha2 running vpn nodeB# local-address 55.55.55.1
ha2 running vpn nodeB# security-policy nodeB
ha2 running security-policy nodeB# local-ts subnet 10.77.77.0/24
ha2 running security-policy nodeB# remote-ts subnet 10.7.0.0/24
ha2 running security-policy nodeB# svti-id-in 10
ha2 running security-policy nodeB# svti-id-out 10
ha2 running security-policy nodeB# inject-routes
ha2 running inject-routes# next-hop svti-trusted
ha2 running inject-routes# vrf trusted
ha2 running inject-routes# ..
ha2 running security-policy nodeB# ..
ha2 running vpn nodeB# ..
ha2 running ike# pre-shared-key femto
ha2 running pre-shared-key femto# secret 0x12345678
ha2 running pre-shared-key femto# ..
ha2 running ike# /
ha2 running config#
Configure IKE HA:
ha2 running config#! / vrf untrusted ike
ha2 running ike#! ha
ha2 running ha#! listen-ha-group ha_ipsec
ha2 running ha#! local-address 192.168.2.1
ha2 running ha#! remote-address 192.168.2.0
ha2 running ha# /
ha2 running config#
Once everything is configured, commit the configuration:
ha2 running config#
ha2 running config# commit
Configuration committed.
Use case: L3 HA IKE multipeer cluster with VRRP¶
Layer 3 HA multipeer is an extension of the L3 HA use case, where more than 2
HA nodes are members of the HA cluster. An example of use case is to install
2 HA nodes ha1 and ha2 in the same datacenter, that provide local
redundancy, and a third node ha3 in a remote datacenter, that provides
geographical redundancy. The 3 members of the cluster perform stateful
synchronization of IKE states. ha1 is the primary security gateway, i.e. the
highest priority HA node. By default, it is the master node. In case of
failure, the local node ha2 becomes master instead. If both nodes happen to
fail (for example due to a failure of the whole datacenter), the remote node
ha3 becomes the master node.
The configuration is overall the same as for the L3 HA use case with 2 nodes, just the configuration of the ha peers differs à little, as well as the HA synchronization channel.
The HA nodes are on separate networks and can only contact each other via IP routing. The traffic that must be processed is steered to the master node at the layer 3 level, i.e. by routing, thanks to routing protocols. In our example, we use BGP.
The HA synchronization channel may be a simple routed TCP
connection, but should preferably go through a reliable and secured path,
possibly an IPsec-encrypted GRE tunnel or SVTI interface, as described
in Encrypting the synchronization messages, depending whether the network
between HA nodes is secured or not. In our example, we use a cross-VRF SVTI
interface over an internal VLAN vlan_ha, with IPsec protection.
The VRRP protocol is used in unicast mode to elect the master and backup nodes (see VRRP command reference for details about VRRP). HA nodes exchange unicast point-to-point VRRP messages instead of multicast ones. The VRRP protocol elects the master node depending on the connectivity between nodes and their VRRP priorities. It also tracks the link state of the HA synchronization channel, as well as the connectivity with the routers to the trusted and untrusted networks, via the BFD protocol. Finally, it installs a VRRP VIP on the master node (55.55.55.1), which is the IKE and IPsec tunnel endpoint.
Only the master HA node owns the IPsec tunnel endpoint (55.55.55.1). The routes to the IPsec tunnel endpoint and to the remote networks discovered by IKE reverse route injection are announced only by the master node via BGP. However, in order to handle possible split brain situations (i.e. more than one node consider that they are master), one should make sure that preferences of the routes announced by different gateways match the order of the vrrp priorities. If necessary, route preferences may be influenced by using various BGP mechanisms such as AS-Path prepending or LOCAL_PREF.
To separate the trusted and untrusted networks, a VRF is created for each zone. This enables partitioning routing and makes sure crossing the untrusted network will require transiting via the redundant security gateway, that will provide IPsec protection.
In this example, a separate VRF and network is also created for the HA synchronization traffic, but it is not mandatory.
ha1 CLI configuration¶
Configure device hostname:
vsr running config# system hostname ha1
vsr running config# commit
Configuration committed.
In this example, ha1 only has one physical ethernet interface (eth1) and uses
VLANs in different VRFs to partition the trusted, untrusted and ha
networks.
Configure the physical network interface (adapt port ids to your hardware) in
VRF main:
ha1 running config# vrf main
ha1 running vrf main#! interface physical eth1
ha1 running physical eth1#! port pci-b0s4
ha1 running physical eth1# mtu 9000
ha1 running physical eth1# /
ha1 running config#
Then configure the VRF and VLAN for the trusted network:
ha1 running config# / vrf trusted interface
ha1 running interface# vlan vlan_trusted
ha1 running vlan vlan_trusted#! vlan-id 2
ha1 running vlan vlan_trusted#! link-interface eth1
ha1 running vlan vlan_trusted# link-vrf main
ha1 running vlan vlan_trusted# ipv4 address 10.2.0.1/29
ha1 running vlan vlan_trusted# /
ha1 running config#
Then configure the VRF and VLAN for the untrusted network:
ha1 running config# / vrf untrusted interface
ha1 running interface# vlan vlan_untrusted
ha1 running vlan vlan_untrusted#! vlan-id 307
ha1 running vlan vlan_untrusted#! link-interface eth1
ha1 running vlan vlan_untrusted# link-vrf main
ha1 running vlan vlan_untrusted# ipv4 address 10.55.0.1/29
ha1 running vlan vlan_untrusted# /
ha1 running config#
Then configure the VRF and VLAN for the HA synchronization network:
ha1 running config# / vrf ha interface
ha1 running interface# vlan vlan_ha
ha1 running vlan vlan_ha#! vlan-id 10
ha1 running vlan vlan_ha#! link-interface eth1
ha1 running vlan vlan_ha# link-vrf main
ha1 running vlan vlan_ha# ipv4 address 192.168.1.1/29
ha1 running vlan vlan_ha# /
ha1 running config#
In order to perform cross-VRF IPsec, the trusted traffic will be sent through
an SVTI cross-VRF interface in VRF trusted, with link-VRF untrusted:
ha1 running config# / vrf trusted interface
ha1 running interface# svti svti-trusted
ha1 running svti svti-trusted# svti-id 10
ha1 running svti svti-trusted# link-vrf untrusted
ha1 running svti svti-trusted# /
Now configure routing on the trusted network. The connectivity to the BGP peer
is tracked via the BFD protocol. The kernel routes of the trusted network are
redistributed via BGP. These are the routes to the remote trusted networks,
learnt by IKE reverse route injection. A route map TRUSTED-OUT is attached
to the peer, so that routes learnt by IKE reverse route injection are only
advertised to this peer when ha1 is master.
ha1 running config# / vrf trusted routing
ha1 running routing# bgp
ha1 running bgp# as 65111
ha1 running bgp# router-id 10.5.5.5
ha1 running bgp# ebgp-requires-policy false
ha1 running bgp# address-family ipv4-unicast redistribute kernel
ha1 running bgp# neighbor 10.2.0.10
ha1 running neighbor 10.2.0.10# remote-as 65123
ha1 running neighbor 10.2.0.10# address-family ipv4-unicast
ha1 running ipv4-unicast# route-map out route-map-name TRUSTED-OUT
ha1 running running ipv4-unicast# ..
ha1 running running address-family# ..
ha1 running running neighbor 10.2.0.10# track bfd
ha1 running running neighbor 10.2.0.10# /
ha1 running config#
Now configure routing on the untrusted network. The connectivity to the BGP
peer is tracked via the BFD protocol. The IPsec tunnel endpoint
(55.55.55.1/32) is advertised by BGP. A route map UNTRUSTED-OUT is attached
to the peer, so that the route to the IPsec tunnel endpoint is only advertised
when ha1 is master.
ha1 running config# / vrf untrusted routing
ha1 running routing# bgp
ha1 running bgp# as 65111
ha1 running bgp# router-id 10.5.5.5
ha1 running bgp# ebgp-requires-policy false
ha1 running bgp# address-family ipv4-unicast network 55.55.55.1/32
ha1 running bgp# neighbor 10.55.0.10
ha1 running neighbor 10.55.0.10# remote-as 65123
ha1 running neighbor 10.55.0.10# address-family ipv4-unicast
ha1 running ipv4-unicast# route-map out route-map-name UNTRUSTED-OUT
ha1 running running ipv4-unicast# ..
ha1 running running address-family# ..
ha1 running neighbor 10.55.0.10# track bfd
ha1 running neighbor 10.55.0.10# /
ha1 running config#
The ha network is an internal network that interconnects the HA nodes and
enables exchanging synchronization data. We chose to use a separate VRF and
to access it via a different VLAN. However, the IKE service runs in VRF
untrusted, hence the HA module needs to establish a TCP connection with
the HA peer in VRF untrusted. We also want to protect the synchronization
traffic with IPsec. To proceed, we create an cross-VRF SVTI interface in VRF
untrusted, with link-VRF ha. The HA synchronization messages are exchanged
between the HA nodes on the 192.168.2.0/29 network in vrf untrusted, and
IPsec-encapsulated in the 192.168.1.0/29 network in vrf ha:
ha1 running config# / vrf untrusted interface
ha1 running interface# svti svti-ha
ha1 running svti svti-ha# svti-id 33
ha1 running svti svti-ha# link-vrf ha
ha1 running svti svti-ha# ipv4 address 192.168.2.1/29
ha1 running svti svti-ha# /
ha1 running config#
Create a loopback interface whose address is the BGP and VRRP router-ids:
ha1 running config# / vrf untrusted interface
ha1 running interface# loopback loopback0
ha1 running loopback loopback0# ipv4 address 10.5.5.5/32
ha1 running loopback loopback0# /
ha1 running config#
Configure the route maps used by the BGP configurations so that the routes to the IPsec tunnel endpoint and to the remote networks discovered by IKE reverse route injection are announced only by the master:
ha1 running config# / routing
ha1 running routing# ipv4-prefix-list IPSEC_TEP
ha1 running ipv4-prefix-list IPSEC_TEP# seq 10 address 55.55.55.1/32 policy permit
ha1 running ipv4-prefix-list IPSEC_TEP# ..
ha1 running routing# ipv4-prefix-list RAN
ha1 running ipv4-prefix-list RAN# seq 10 address 10.7.0.0/24 policy permit le 32
ha1 running ipv4-prefix-list RAN# ..
ha1 running routing# route-map UNTRUSTED-OUT
ha1 running route-map UNTRUSTED-OUT# seq 10
ha1 running route-map seq 10# policy permit
ha1 running route-map seq 10# match ip address prefix-list IPSEC_TEP
ha1 running route-map seq 10# match ha-group name ha_ipsec
ha1 running route-map seq 10# ..
ha1 running route-map UNTRUSTED-OUT# ..
ha1 running routing# route-map TRUSTED-OUT
ha1 running route-map TRUSTED-OUT# seq 10
ha1 running route-map seq 10# policy permit
ha1 running route-map seq 10# match ip address prefix-list RAN
ha1 running route-map seq 10# match ha-group name ha_ipsec
ha1 running route-map seq 10# ..
ha1 running route-map TRUSTED-OUT# /
ha1 running config#
Create two trackers that monitor the connectivity with the routers to the trusted and untrusted networks:
ha1 running config# / tracker bfd pe-bfd-trusted
ha1 running bfd pe-bfd-trusted# source 10.2.0.1
ha1 running bfd pe-bfd-trusted# address 10.2.0.10
ha1 running bfd pe-bfd-trusted# vrf trusted
ha1 running bfd pe-bfd-trusted# ..
ha1 running config# / tracker bfd pe-bfd-untrusted
ha1 running bfd pe-bfd-untrusted# source 10.55.0.1
ha1 running bfd pe-bfd-untrusted# address 10.55.0.10
ha1 running bfd pe-bfd-untrusted# vrf untrusted
ha1 running bfd pe-bfd-untrusted# /
ha1 running config#
We can finally configure HA. First configure the HA group:
ha1 running config# / ha group ha-group1
ha1 running group ha-group1#! ..
ha1 running ha#! /
ha1 running config#!
Note
The HA group maintains the node high-availability state. It is controlled by
the VRRP protocol (via the notify-ha-group command) and monitored by HA IKE
(via the listen-ha-group command). Only one controller can be defined for
an HA group.
Configure VRRP: VRRP packets are sent over the svti-ha interface in
unicast mode. VRRP tracks the link state of the svti-ha interface, as
well as the connectivity with the routers to the trusted and untrusted networks,
via the BFD protocol. Finally, it installs a VRRP VIP on the master node
(55.55.55.1), which is the IKE and IPsec tunnel endpoint. It notifies the
services interested by the HA status via the HA group ha_ipsec (IKE and
routing):
ha1 running config# / vrf untrusted interface
ha1 running interface# vrrp vrrp_ha
ha1 running vrrp vrrp_ha# version 3
ha1 running vrrp vrrp_ha# link-interface svti-ha
ha1 running vrrp vrrp_ha# use-vmac false
ha1 running vrrp vrrp_ha# vrid 10
ha1 running vrrp vrrp_ha# priority 200
ha1 running vrrp vrrp_ha# preempt-delay 30
ha1 running vrrp vrrp_ha# advertisement-interval 100
ha1 running vrrp vrrp_ha# track pe-bfd-trusted
ha1 running vrrp vrrp_ha# track pe-bfd-untrusted
ha1 running vrrp vrrp_ha# track-fast-path true
ha1 running vrrp vrrp_ha# unicast-peer 192.168.2.2
ha1 running vrrp vrrp_ha# unicast-peer 192.168.2.3
ha1 running vrrp vrrp_ha# virtual-address 55.55.55.1/32
ha1 running vrrp vrrp_ha# notify-ha-group ha_ipsec
ha1 running vrrp vrrp_ha# /
ha1 running config# / vrf untrusted vrrp
ha1 running vrrp# router-id 10.5.5.5
ha1 running vrrp# vrrp-startup-delay 30
ha1 running vrrp# /
ha1 running config#
Now that network connectivity and routing are configured, let us configure IKE.
Here, ha1 is configured as a VPN concentrator that accepts IKE connections
initiated by a remote peer. To support cross-VRF IPsec, the cross-VRF SVTI
interface svti-trusted is used (VRF trusted, link-VRF untrusted, svti-id 10).
The inject-routes option automatically adds routes to the remote traffic
selector via the SVTI interface (reverse route injection).
Configure IKE:
ha1 running config# / vrf untrusted ike
ha1 running ike# ike-policy-template nodeB
ha1 running ike-proposal-1# enc-alg aes256-cbc
ha1 running ike-proposal-1# auth-alg hmac-sha256
ha1 running ike-proposal-1# dh-group modp2048
ha1 running ike-proposal-1# ...
ha1 running ike-policy-template nodeB# unique-sa replace
ha1 running ike-policy-template nodeB# rekey-time 2800
ha1 running ike-policy-template nodeB# dpd-delay 10
ha1 running ike-policy-template nodeB# ..
ha1 running ike# ipsec-policy-template nodeB
ha1 running esp-proposal 1# esp-proposal 1
ha1 running esp-proposal 1# enc-alg aes256-cbc
ha1 running esp-proposal 1# auth-alg hmac-sha256
ha1 running esp-proposal 1# ..
ha1 running ipsec-policy-template nodeB# start-action none
ha1 running ipsec-policy-template nodeB# close-action none
ha1 running ipsec-policy-template nodeB# dpd-action clear
ha1 running ipsec-policy-template nodeB# rekey-time 2800
ha1 running ipsec-policy-template nodeB# ..
ha1 running ike# vpn nodeB
ha1 running vpn nodeB# ike-policy template nodeB
ha1 running vpn nodeB# ipsec-policy template nodeB
ha1 running vpn nodeB# local-address 55.55.55.1
ha1 running vpn nodeB# security-policy nodeB
ha1 running security-policy nodeB# local-ts subnet 10.77.77.0/24
ha1 running security-policy nodeB# remote-ts subnet 10.7.0.0/24
ha1 running security-policy nodeB# svti-id-in 10
ha1 running security-policy nodeB# svti-id-out 10
ha1 running security-policy nodeB# inject-routes
ha1 running inject-routes# next-hop svti-trusted
ha1 running inject-routes# vrf trusted
ha1 running inject-routes# ..
ha1 running security-policy nodeB# ..
ha1 running vpn nodeB# ..
ha1 running ike# pre-shared-key femto
ha1 running pre-shared-key femto# secret 0x12345678
ha1 running pre-shared-key femto# ..
ha1 running ike# /
ha1 running config#
Configure IKE HA:
ha1 running config#! / vrf untrusted ike
ha1 running ike#! ha
ha1 running ha#! listen-ha-group ha_ipsec
ha1 running ha# peer peer1
ha1 running peer1# local-address 192.168.2.1
ha1 running peer1# remote-address 192.168.2.2
ha1 running peer1# ..
ha1 running ha# peer peer2
ha1 running peer2# local-address 192.168.2.1
ha1 running peer2# remote-address 192.168.2.3
ha1 running peer2# ..
ha1 running ha# /
ha1 running config#
Finally, configure IKE to encrypt the traffic on the HA network (transmitted
via the SVTI insterface svti-ha):
ha1 running config#
ha1 running config# / vrf ha ike
ha1 running ike# ike-policy-template ikepol-hasync
ha1 running ike-policy-template ikepol-hasync# ike-proposal 1
ha1 running ike-proposal 1#! enc-alg aes128-cbc
ha1 running ike-proposal 1#! auth-alg hmac-sha1
ha1 running ike-proposal 1#! prf-alg hmac-sha1
ha1 running ike-proposal 1#! dh-group modp1024
ha1 running ike-proposal 1# ..
ha1 running ike-policy-template ikepol-hasync# keying-tries 0
ha1 running ike-policy-template ikepol-hasync# unique-sa replace
ha1 running ike-policy-template ikepol-hasync# ..
ha1 running ike# ipsec-policy-template ipsecpol-hasync
ha1 running ipsec-policy-template ipsecpol-hasync# esp-proposal 1
ha1 running esp-proposal 1#! enc-alg aes128-cbc
ha1 running esp-proposal 1#! auth-alg hmac-sha1
ha1 running esp-proposal 1# dh-group modp1024
ha1 running esp-proposal 1# esn true
ha1 running esp-proposal 1# ..
ha1 running ipsec-policy-template ipsecpol-hasync# ..
ha1 running ike# vpn vpn-hasync12
ha1 running vpn vpn-hasync12#! ike-policy
ha1 running ike-policy#! template ikepol-hasync
ha1 running ike-policy#! ..
ha1 running vpn vpn-hasync12#! ipsec-policy
ha1 running ipsec-policy#! template ipsecpol-hasync
ha1 running ipsec-policy# ..
ha1 running vpn vpn-hasync12# local-id segw-1
ha1 running vpn vpn-hasync12# local-address 192.168.1.1
ha1 running vpn vpn-hasync12# remote-address 192.168.1.2
ha1 running vpn vpn-hasync12# security-policy svti-hasync12
ha1 running security-policy svti-hasync12# local-ts subnet 192.168.2.1
ha1 running security-policy svti-hasync12# remote-ts subnet 192.168.2.2
ha1 running security-policy svti-hasync12# svti-id-in 33
ha1 running security-policy svti-hasync12# svti-id-out 33
ha1 running security-policy svti-hasync12# ..
ha1 running vpn vpn-hasync12# ..
ha1 running ike# vpn vpn-hasync13
ha1 running vpn vpn-hasync13#! ike-policy
ha1 running ike-policy#! template ikepol-hasync
ha1 running ike-policy#! ..
ha1 running vpn vpn-hasync13#! ipsec-policy
ha1 running ipsec-policy#! ..
ha1 running vpn vpn-hasync13#! local-id segw-1
ha1 running vpn vpn-hasync13#! local-address 192.168.1.1
ha1 running vpn vpn-hasync13#! remote-address 192.168.1.3
ha1 running vpn vpn-hasync13#! security-policy svti-hasync13
ha1 running security-policy svti-hasync13#! local-ts subnet 192.168.2.1
ha1 running security-policy svti-hasync13#! remote-ts subnet 192.168.2.3
ha1 running security-policy svti-hasync13#! svti-id-in 33
ha1 running security-policy svti-hasync13#! svti-id-out 33
ha1 running security-policy svti-hasync13#! /
ha1 running config#
Once everything is configured, commit the configuration:
ha1 running config#
ha1 running config# commit
Configuration committed.
ha2 CLI configuration¶
Configure device hostname:
vsr running config# system hostname ha2
vsr running config# commit
Configuration committed.
In this example, ha2 only has one physical ethernet interface (eth1) and uses
VLANs in different VRFs to partition the trusted, untrusted and ha
networks.
Configure the physical network interface (adapt port ids to your hardware) in
VRF main:
ha2 running config# vrf main
ha2 running vrf main#! interface physical eth1
ha2 running physical eth1#! port pci-b0s4
ha2 running physical eth1# mtu 9000
ha2 running physical eth1# /
ha2 running config#
Then configure the VRF and VLAN for the trusted network:
ha2 running config# / vrf trusted interface
ha2 running interface# vlan vlan_trusted
ha2 running vlan vlan_trusted#! vlan-id 2
ha2 running vlan vlan_trusted#! link-interface eth1
ha2 running vlan vlan_trusted# link-vrf main
ha2 running vlan vlan_trusted# ipv4 address 10.2.0.2/29
ha2 running vlan vlan_trusted# /
ha2 running config#
Then configure the VRF and VLAN for the untrusted network:
ha2 running config# / vrf untrusted interface
ha2 running interface# vlan vlan_untrusted
ha2 running vlan vlan_untrusted#! vlan-id 307
ha2 running vlan vlan_untrusted#! link-interface eth1
ha2 running vlan vlan_untrusted# link-vrf main
ha2 running vlan vlan_untrusted# ipv4 address 10.55.0.2/29
ha2 running vlan vlan_untrusted# /
ha2 running config#
Then configure the VRF and VLAN for the HA synchronization network:
ha2 running config# / vrf ha interface
ha2 running interface# vlan vlan_ha
ha2 running vlan vlan_ha#! vlan-id 10
ha2 running vlan vlan_ha#! link-interface eth1
ha2 running vlan vlan_ha# link-vrf main
ha2 running vlan vlan_ha# ipv4 address 192.168.1.2/29
ha2 running vlan vlan_ha# /
ha2 running config#
In order to perform cross-VRF IPsec, the trusted traffic will be sent through
an SVTI cross-VRF interface in VRF trusted, with link-VRF untrusted:
ha2 running config# / vrf trusted interface
ha2 running interface# svti svti-trusted
ha2 running svti svti-trusted# svti-id 10
ha2 running svti svti-trusted# link-vrf untrusted
ha2 running svti svti-trusted# /
Now configure routing on the trusted network. The connectivity to the BGP peer
is tracked via the BFD protocol. The kernel routes of the trusted network are
redistributed via BGP. These are the routes to the remote trusted networks,
learnt by IKE reverse route injection. A route map TRUSTED-OUT is attached
to the peer, so that routes learnt by IKE reverse route injection are only
advertised to this peer when ha2 is master.
ha2 running config# / vrf trusted routing
ha2 running routing# bgp
ha2 running bgp# as 65111
ha2 running bgp# router-id 10.6.6.6
ha2 running bgp# ebgp-requires-policy false
ha2 running bgp# address-family ipv4-unicast redistribute kernel
ha2 running bgp# neighbor 10.0.0.2
ha2 running neighbor 10.2.0.10# remote-as 65123
ha2 running neighbor 10.2.0.10# address-family ipv4-unicast
ha2 running ipv4-unicast# route-map out route-map-name TRUSTED-OUT
ha2 running running ipv4-unicast# ..
ha2 running running address-family# ..
ha2 running running neighbor 10.2.0.10# track bfd
ha2 running running neighbor 10.2.0.10# /
ha2 running config#
Now configure routing on the untrusted network. The connectivity to the BGP
peer is tracked via the BFD protocol. The IPsec tunnel endpoint
(55.55.55.1/32) is advertised by BGP. A route map UNTRUSTED-OUT is attached
to the peer, so that the route to the IPsec tunnel endpoint is only advertised
when ha2 is master.
ha2 running config# / vrf untrusted routing
ha2 running routing# bgp
ha2 running bgp# as 65111
ha2 running bgp# router-id 10.6.6.6
ha2 running bgp# ebgp-requires-policy false
ha2 running bgp# address-family ipv4-unicast network 55.55.55.1/32
ha2 running bgp# neighbor 10.55.0.10
ha2 running neighbor 10.55.0.10# remote-as 65123
ha2 running neighbor 10.55.0.10# address-family ipv4-unicast
ha2 running ipv4-unicast# route-map out route-map-name UNTRUSTED-OUT
ha2 running running ipv4-unicast# ..
ha2 running running address-family# ..
ha2 running neighbor 10.55.0.10# track bfd
ha2 running neighbor 10.55.0.10# /
ha2 running config#
The ha network is an internal network that interconnects the HA nodes and
enables exchanging synchronization data. We chose to use a separate VRF and
to access it via a different VLAN. However, the IKE service runs in VRF
untrusted, hence the HA module needs to establish a TCP connection with
the HA peer in VRF untrusted. To proceed, we also want to protect the
synchronization traffic with IPsec. We create an cross-VRF SVTI interface in
VRF untrusted, with link-VRF ha. The HA synchronization messages are
exchanged between the HA nodes on the 192.168.2.0/29 network in vrf
untrusted, and IPsec-encapsulated in the 192.168.1.0/29 network in vrf
ha:
ha2 running config# / vrf untrusted interface
ha2 running interface# svti svti-ha
ha2 running svti svti-ha# svti-id 33
ha2 running svti svti-ha# link-vrf ha
ha2 running svti svti-ha# ipv4 address 192.168.2.2/29
ha2 running svti svti-ha# /
ha2 running config#
Create a loopback interface whose address is the BGP and VRRP router-ids:
ha2 running config# / vrf untrusted interface
ha2 running interface# loopback loopback0
ha2 running loopback loopback0# ipv4 address 10.6.6.6/32
ha2 running loopback loopback0# /
ha2 running config#
Configure the route maps used by the BGP configurations so that the routes to the IPsec tunnel endpoint and to the remote networks discovered by IKE reverse route injection are announced only by the master. The cost of these routes is artificially increased by prepending the local AS to the AS-Path, so that the route via ha1 is prefered to the route to ha2 in case of split brain:
ha2 running config# / routing
ha2 running routing# ipv4-prefix-list IPSEC_TEP
ha2 running ipv4-prefix-list IPSEC_TEP# seq 10 address 55.55.55.1/32 policy permit
ha2 running ipv4-prefix-list IPSEC_TEP# ..
ha2 running routing# ipv4-prefix-list RAN
ha2 running ipv4-prefix-list RAN# seq 10 address 10.7.0.0/24 policy permit le 32
ha2 running ipv4-prefix-list RAN# ..
ha2 running routing# route-map UNTRUSTED-OUT
ha2 running route-map UNTRUSTED-OUT# seq 10
ha2 running route-map seq 10# policy permit
ha2 running route-map seq 10# match ip address prefix-list IPSEC_TEP
ha2 running route-map seq 10# match ha-group name ha_ipsec
ha2 running route-map seq 20# policy permit
ha2 running route-map seq 20# match ip address prefix-list IPSEC_TEP
ha2 running route-map seq 20# set as-path prepend asn 1 65111
ha2 running route-map seq 20# ..
ha2 running route-map UNTRUSTED-OUT# ..
ha2 running routing# route-map TRUSTED-OUT
ha2 running route-map TRUSTED-OUT# seq 10
ha2 running route-map seq 10# policy permit
ha2 running route-map seq 10# match ip address prefix-list RAN
ha2 running route-map seq 10# match ha-group name ha_ipsec
ha2 running route-map seq 20# policy permit
ha2 running route-map seq 20# match ip address prefix-list RAN
ha2 running route-map seq 20# set as-path prepend asn 1 65111
ha2 running route-map seq 20# ..
ha2 running route-map TRUSTED-OUT# /
ha2 running config#
Create two trackers that monitor the connectivity with the routers to the trusted and untrusted networks:
ha2 running config# / tracker bfd pe-bfd-trusted
ha2 running bfd pe-bfd-trusted# source 10.2.0.2
ha2 running bfd pe-bfd-trusted# address 10.2.0.10
ha2 running bfd pe-bfd-trusted# vrf trusted
ha2 running bfd pe-bfd-trusted# ..
ha2 running config# / tracker bfd pe-bfd-untrusted
ha2 running bfd pe-bfd-untrusted# source 10.55.0.2
ha2 running bfd pe-bfd-untrusted# address 10.55.0.10
ha2 running bfd pe-bfd-untrusted# vrf untrusted
ha2 running bfd pe-bfd-untrusted# /
ha2 running config#
We can finally configure HA. First configure the HA group:
ha2 running config# / ha group ha-group1
ha2 running group ha-group1#! ..
ha2 running ha#! /
ha2 running config#!
Note
The HA group maintains the node high-availability state. It is controlled by
the VRRP protocol (via the notify-ha-group command) and monitored by HA IKE
(via the listen-ha-group command). Only one controller can be defined for
an HA group.
Configure VRRP: VRRP packets are sent over the svti-ha interface in
unicast mode. VRRP tracks the link state of the svti-ha interface, as
well as the connectivity with the routers to the trusted and untrusted networks,
via the BFD protocol. Finally, it installs a VRRP VIP on the master node
(55.55.55.1), which is the IKE and IPsec tunnel endpoint. It notifies the
services interested by the HA status via the HA group ha_ipsec (IKE and
routing):
ha2 running config# / vrf untrusted interface
ha2 running interface# vrrp vrrp_ha
ha2 running vrrp vrrp_ha# version 3
ha2 running vrrp vrrp_ha# link-interface svti-ha
ha2 running vrrp vrrp_ha# use-vmac false
ha2 running vrrp vrrp_ha# vrid 10
ha2 running vrrp vrrp_ha# priority 150
ha2 running vrrp vrrp_ha# preempt-delay 30
ha2 running vrrp vrrp_ha# advertisement-interval 100
ha2 running vrrp vrrp_ha# track pe-bfd-trusted
ha2 running vrrp vrrp_ha# track pe-bfd-untrusted
ha2 running vrrp vrrp_ha# track-fast-path true
ha2 running vrrp vrrp_ha# unicast-peer 192.168.2.1
ha2 running vrrp vrrp_ha# unicast-peer 192.168.2.3
ha2 running vrrp vrrp_ha# virtual-address 55.55.55.1/32
ha2 running vrrp vrrp_ha# notify-ha-group ha_ipsec
ha2 running vrrp vrrp_ha# /
ha2 running config# / vrf untrusted vrrp
ha2 running vrrp# router-id 10.6.6.6
ha2 running vrrp# vrrp-startup-delay 30
ha2 running vrrp# /
ha2 running config#
Now that network connectivity and routing are configured, let us configure IKE.
Here, ha2 is configured as a VPN concentrator that accepts IKE connections
initiated by a remote peer. To support cross-VRF IPsec, the cross-VRF SVTI
interface svti-trusted is used (VRF trusted, link-VRF untrusted, svti-id 10).
The inject-routes option automatically adds routes to the remote traffic
selector via the SVTI interface (reverse route injection).
Configure IKE:
ha2 running config# / vrf untrusted ike
ha2 running ike# ike-policy-template nodeB
ha2 running ike-proposal-1# enc-alg aes256-cbc
ha2 running ike-proposal-1# auth-alg hmac-sha256
ha2 running ike-proposal-1# dh-group modp2048
ha2 running ike-proposal-1# ...
ha2 running ike-policy-template nodeB# unique-sa replace
ha2 running ike-policy-template nodeB# rekey-time 2800
ha2 running ike-policy-template nodeB# dpd-delay 10
ha2 running ike-policy-template nodeB# ..
ha2 running ike# ipsec-policy-template nodeB
ha2 running esp-proposal 1# esp-proposal 1
ha2 running esp-proposal 1# enc-alg aes256-cbc
ha2 running esp-proposal 1# auth-alg hmac-sha256
ha2 running esp-proposal 1# ..
ha2 running ipsec-policy-template nodeB# start-action none
ha2 running ipsec-policy-template nodeB# close-action none
ha2 running ipsec-policy-template nodeB# dpd-action clear
ha2 running ipsec-policy-template nodeB# rekey-time 2800
ha2 running ipsec-policy-template nodeB# ..
ha2 running ike# vpn nodeB
ha2 running vpn nodeB# ike-policy template nodeB
ha2 running vpn nodeB# ipsec-policy template nodeB
ha2 running vpn nodeB# local-address 55.55.55.1
ha2 running vpn nodeB# security-policy nodeB
ha2 running security-policy nodeB# local-ts subnet 10.77.77.0/24
ha2 running security-policy nodeB# remote-ts subnet 10.7.0.0/24
ha2 running security-policy nodeB# svti-id-in 10
ha2 running security-policy nodeB# svti-id-out 10
ha2 running security-policy nodeB# inject-routes
ha2 running inject-routes# next-hop svti-trusted
ha2 running inject-routes# vrf trusted
ha2 running inject-routes# ..
ha2 running security-policy nodeB# ..
ha2 running vpn nodeB# ..
ha2 running ike# pre-shared-key femto
ha2 running pre-shared-key femto# secret 0x12345678
ha2 running pre-shared-key femto# ..
ha2 running ike# /
ha2 running config#
Configure IKE HA:
ha2 running config#! / vrf untrusted ike
ha2 running ike#! ha
ha2 running ha#! listen-ha-group ha_ipsec
ha2 running ha# peer peer1
ha2 running peer2# local-address 192.168.2.3
ha2 running peer2# remote-address 192.168.2.2
ha2 running peer2# ..
ha2 running ha# peer peer2
ha2 running peer3# local-address 192.168.2.3
ha2 running peer3# remote-address 192.168.2.1
ha2 running peer3# ..
ha2 running ha# /
ha2 running config#
Finally, configure IKE to encrypt the traffic on the HA network (transmitted
via the SVTI insterface svti-ha):
ha2 running config#
ha2 running config# / vrf ha ike
ha2 running ike# ike-policy-template ikepol-hasync
ha2 running ike-policy-template ikepol-hasync# ike-proposal 1
ha2 running ike-proposal 1#! enc-alg aes128-cbc
ha2 running ike-proposal 1#! auth-alg hmac-sha1
ha2 running ike-proposal 1#! prf-alg hmac-sha1
ha2 running ike-proposal 1#! dh-group modp1024
ha2 running ike-proposal 1# ..
ha2 running ike-policy-template ikepol-hasync# keying-tries 0
ha2 running ike-policy-template ikepol-hasync# unique-sa replace
ha2 running ike-policy-template ikepol-hasync# ..
ha2 running ike# ipsec-policy-template ipsecpol-hasync
ha2 running ipsec-policy-template ipsecpol-hasync# esp-proposal 1
ha2 running esp-proposal 1#! enc-alg aes128-cbc
ha2 running esp-proposal 1#! auth-alg hmac-sha1
ha2 running esp-proposal 1# dh-group modp1024
ha2 running esp-proposal 1# esn true
ha2 running esp-proposal 1# ..
ha2 running ipsec-policy-template ipsecpol-hasync# ..
ha2 running ike# vpn vpn-hasync21
ha2 running vpn vpn-hasync21#! ike-policy
ha2 running ike-policy#! template ikepol-hasync
ha2 running ike-policy#! ..
ha2 running vpn vpn-hasync21#! ipsec-policy
ha2 running ipsec-policy#! template ipsecpol-hasync
ha2 running ipsec-policy# ..
ha2 running vpn vpn-hasync21# local-id segw-2
ha2 running vpn vpn-hasync21# local-address 192.168.1.2
ha2 running vpn vpn-hasync21# remote-address 192.168.1.1
ha2 running vpn vpn-hasync21# security-policy svti-hasync21
ha2 running security-policy svti-hasync21# local-ts subnet 192.168.2.2
ha2 running security-policy svti-hasync21# remote-ts subnet 192.168.2.1
ha2 running security-policy svti-hasync21# svti-id-in 33
ha2 running security-policy svti-hasync21# svti-id-out 33
ha2 running security-policy svti-hasync21# ..
ha2 running vpn vpn-hasync21# ..
ha2 running ike# vpn vpn-hasync23
ha2 running vpn vpn-hasync23#! ike-policy
ha2 running ike-policy#! template ikepol-hasync
ha2 running ike-policy#! ..
ha2 running vpn vpn-hasync23#! ipsec-policy
ha2 running ipsec-policy#! ..
ha2 running vpn vpn-hasync23#! local-id segw-2
ha2 running vpn vpn-hasync23#! local-address 192.168.1.2
ha2 running vpn vpn-hasync23#! remote-address 192.168.1.3
ha2 running vpn vpn-hasync23#! security-policy svti-hasync23
ha2 running security-policy svti-hasync23#! local-ts subnet 192.168.2.2
ha2 running security-policy svti-hasync23#! remote-ts subnet 192.168.2.3
ha2 running security-policy svti-hasync23#! svti-id-in 33
ha2 running security-policy svti-hasync23#! svti-id-out 33
ha2 running security-policy svti-hasync23#! /
ha2 running config#
Once everything is configured, commit the configuration:
ha2 running config#
ha2 running config# commit
Configuration committed.
ha3 CLI configuration¶
Configure device hostname:
vsr running config# system hostname ha3
vsr running config# commit
Configuration committed.
In this example, ha3 only has one physical ethernet interface (eth1) and uses
VLANs in different VRFs to partition the trusted, untrusted and ha
networks.
Configure the physical network interface (adapt port ids to your hardware) in
VRF main:
ha3 running config# vrf main
ha3 running vrf main#! interface physical eth1
ha3 running physical eth1#! port pci-b0s4
ha3 running physical eth1# mtu 9000
ha3 running physical eth1# /
ha3 running config#
Then configure the VRF and VLAN for the trusted network:
ha3 running config# / vrf trusted interface
ha3 running interface# vlan vlan_trusted
ha3 running vlan vlan_trusted#! vlan-id 2
ha3 running vlan vlan_trusted#! link-interface eth1
ha3 running vlan vlan_trusted# link-vrf main
ha3 running vlan vlan_trusted# ipv4 address 10.2.0.3/29
ha3 running vlan vlan_trusted# /
ha3 running config#
Then configure the VRF and VLAN for the untrusted network:
ha3 running config# / vrf untrusted interface
ha3 running interface# vlan vlan_untrusted
ha3 running vlan vlan_untrusted#! vlan-id 307
ha3 running vlan vlan_untrusted#! link-interface eth1
ha3 running vlan vlan_untrusted# link-vrf main
ha3 running vlan vlan_untrusted# ipv4 address 10.55.0.3/29
ha3 running vlan vlan_untrusted# /
ha3 running config#
Then configure the VRF and VLAN for the HA synchronization network:
ha3 running config# / vrf ha interface
ha3 running interface# vlan vlan_ha
ha3 running vlan vlan_ha#! vlan-id 10
ha3 running vlan vlan_ha#! link-interface eth1
ha3 running vlan vlan_ha# link-vrf main
ha3 running vlan vlan_ha# ipv4 address 192.168.1.3/29
ha3 running vlan vlan_ha# /
ha3 running config#
In order to perform cross-VRF IPsec, the trusted traffic will be sent through
an SVTI cross-VRF interface in VRF trusted, with link-VRF untrusted:
ha3 running config# / vrf trusted interface
ha3 running interface# svti svti-trusted
ha3 running svti svti-trusted# svti-id 10
ha3 running svti svti-trusted# link-vrf untrusted
ha3 running svti svti-trusted# /
Now configure routing on the trusted network. The connectivity to the BGP peer
is tracked via the BFD protocol. The kernel routes of the trusted network are
redistributed via BGP. These are the routes to the remote trusted networks,
learnt by IKE reverse route injection. A route map TRUSTED-OUT is attached
to the peer, so that routes learnt by IKE reverse route injection are only
advertised to this peer when ha3 is master.
ha3 running config# / vrf trusted routing
ha3 running routing# bgp
ha3 running bgp# as 65111
ha3 running bgp# router-id 10.7.7.7
ha3 running bgp# ebgp-requires-policy false
ha3 running bgp# address-family ipv4-unicast redistribute kernel
ha3 running bgp# neighbor 10.0.0.10
ha3 running neighbor 10.2.0.10# remote-as 65123
ha3 running neighbor 10.2.0.10# address-family ipv4-unicast
ha3 running ipv4-unicast# route-map out route-map-name TRUSTED-OUT
ha3 running running ipv4-unicast# ..
ha3 running running address-family# ..
ha3 running running neighbor 10.2.0.10# track bfd
ha3 running running neighbor 10.2.0.10# /
ha3 running config#
Now configure routing on the untrusted network. The connectivity to the BGP
peer is tracked via the BFD protocol. The IPsec tunnel endpoint
(55.55.55.1/32) is advertised by BGP. A route map UNTRUSTED-OUT is attached
to the peer, so that the route to the IPsec tunnel endpoint is only advertised
when ha3 is master.
ha3 running config# / vrf untrusted routing
ha3 running routing# bgp
ha3 running bgp# as 65111
ha3 running bgp# router-id 10.7.7.7
ha3 running bgp# ebgp-requires-policy false
ha3 running bgp# address-family ipv4-unicast network 55.55.55.1/32
ha3 running bgp# neighbor 10.55.0.10
ha3 running neighbor 10.55.0.10# remote-as 65123
ha3 running neighbor 10.55.0.10# address-family ipv4-unicast
ha3 running ipv4-unicast# route-map out route-map-name UNTRUSTED-OUT
ha3 running running ipv4-unicast# ..
ha3 running running address-family# ..
ha3 running neighbor 10.55.0.10# track bfd
ha3 running neighbor 10.55.0.10# /
ha3 running config#
The ha network is an internal network that interconnects the HA nodes and
enables exchanging synchronization data. We chose to use a separate VRF and
to access it via a different VLAN. However, the IKE service runs in VRF
untrusted, hence the HA module needs to establish a TCP connection with the
HA peer in VRF untrusted. We also want to protect the synchronization
traffic with IPsec. To proceed, we create an cross-VRF SVTI interface in VRF
untrusted, with link-VRF ha. The HA synchronization messages will be
exchanged through this SVTI tunnel, and the IPsec-encapsulated packets will
travel in VRF ha:
ha3 running config# / vrf untrusted interface
ha3 running interface# svti svti-ha
ha3 running svti svti-ha# svti-id 33
ha3 running svti svti-ha# link-vrf ha
ha3 running svti svti-ha# ipv4 address 192.168.2.3/29
ha3 running svti svti-ha# /
ha3 running config#
Create a loopback interface whose address is the BGP and VRRP router-ids:
ha3 running config# / vrf untrusted interface
ha3 running interface# loopback loopback0
ha3 running loopback loopback0# ipv4 address 10.7.7.7/32
ha3 running loopback loopback0# /
ha3 running config#
Configure the route maps used by the BGP configurations so that the routes to the IPsec tunnel endpoint and to the remote networks discovered by IKE reverse route injection are announced only by the master. The cost of these routes is artificially increased by prepending the local AS twice to the AS-Path, so that the route via ha3 is costlier than routes to other security gateways in case of split brain:
ha3 running config# / routing
ha3 running routing# ipv4-prefix-list IPSEC_TEP
ha3 running ipv4-prefix-list IPSEC_TEP# seq 10 address 55.55.55.1/32 policy permit
ha3 running ipv4-prefix-list IPSEC_TEP# ..
ha3 running routing# ipv4-prefix-list RAN
ha3 running ipv4-prefix-list RAN# seq 10 address 10.7.0.0/24 policy permit le 32
ha3 running ipv4-prefix-list RAN# ..
ha3 running routing# route-map UNTRUSTED-OUT
ha3 running route-map UNTRUSTED-OUT# seq 10
ha3 running route-map seq 10# policy permit
ha3 running route-map seq 10# match ip address prefix-list IPSEC_TEP
ha3 running route-map seq 10# match ha-group name ha_ipsec
ha3 running route-map seq 20# policy permit
ha3 running route-map seq 20# match ip address prefix-list IPSEC_TEP
ha3 running route-map seq 20# set as-path prepend asn 1 65111
ha3 running route-map seq 20# set as-path prepend asn 2 65111
ha3 running route-map seq 20# ..
ha3 running route-map UNTRUSTED-OUT# ..
ha3 running routing# route-map TRUSTED-OUT
ha3 running route-map TRUSTED-OUT# seq 10
ha3 running route-map seq 10# policy permit
ha3 running route-map seq 10# match ip address prefix-list RAN
ha3 running route-map seq 10# match ha-group name ha_ipsec
ha3 running route-map seq 20# policy permit
ha3 running route-map seq 20# match ip address prefix-list RAN
ha3 running route-map seq 20# set as-path prepend asn 1 65111
ha3 running route-map seq 20# set as-path prepend asn 2 65111
ha3 running route-map seq 20# ..
ha3 running route-map TRUSTED-OUT# /
ha3 running config#
Create two trackers that monitor the connectivity with the routers to the trusted and untrusted networks:
ha3 running config# / tracker bfd pe-bfd-trusted
ha3 running bfd pe-bfd-trusted# source 10.2.0.3
ha3 running bfd pe-bfd-trusted# address 10.2.0.10
ha3 running bfd pe-bfd-trusted# vrf trusted
ha3 running bfd pe-bfd-trusted# ..
ha3 running config# / tracker bfd pe-bfd-untrusted
ha3 running bfd pe-bfd-untrusted# source 10.55.0.3
ha3 running bfd pe-bfd-untrusted# address 10.55.0.10
ha3 running bfd pe-bfd-untrusted# vrf untrusted
ha3 running bfd pe-bfd-untrusted# /
ha3 running config#
We can finally configure HA. First configure the HA group:
ha3 running config# / ha group ha-group1
ha3 running group ha-group1#! ..
ha3 running ha#! /
ha3 running config#!
Note
The HA group maintains the node high-availability state. It is controlled by
the VRRP protocol (via the notify-ha-group command) and monitored by HA IKE
(via the listen-ha-group command). Only one controller can be defined for
an HA group.
Configure VRRP: VRRP packets are sent over the svti-ha interface in
unicast mode. VRRP tracks the link state of the svti-ha interface, as
well as the connectivity with the routers to the trusted and untrusted networks,
via the BFD protocol. Finally, it installs a VRRP VIP on the master node
(55.55.55.1), which is the IKE and IPsec tunnel endpoint. It notifies the
services interested by the HA status via the HA group ha_ipsec (IKE and
routing):
ha3 running config# / vrf untrusted interface
ha3 running interface# vrrp vrrp_ha
ha3 running vrrp vrrp_ha# version 3
ha3 running vrrp vrrp_ha# link-interface svti-ha
ha3 running vrrp vrrp_ha# use-vmac false
ha3 running vrrp vrrp_ha# vrid 10
ha3 running vrrp vrrp_ha# priority 100
ha3 running vrrp vrrp_ha# preempt-delay 30
ha3 running vrrp vrrp_ha# advertisement-interval 100
ha3 running vrrp vrrp_ha# track pe-bfd-trusted
ha3 running vrrp vrrp_ha# track pe-bfd-untrusted
ha3 running vrrp vrrp_ha# track-fast-path true
ha3 running vrrp vrrp_ha# unicast-peer 192.168.2.1
ha3 running vrrp vrrp_ha# unicast-peer 192.168.2.2
ha3 running vrrp vrrp_ha# virtual-address 55.55.55.1/32
ha3 running vrrp vrrp_ha# notify-ha-group ha_ipsec
ha3 running vrrp vrrp_ha# /
ha3 running config# / vrf untrusted vrrp
ha3 running vrrp# router-id 10.7.7.7
ha3 running vrrp# vrrp-startup-delay 30
ha3 running vrrp# /
ha3 running config#
Now that network connectivity and routing are configured, let us configure IKE.
Here, ha3 is configured as a VPN concentrator that accepts IKE connections
initiated by a remote peer. To support cross-VRF IPsec, the cross-VRF SVTI
interface svti-trusted is used (VRF trusted, link-VRF untrusted, svti-id 10).
The inject-routes option automatically adds routes to the remote traffic
selector via the SVTI interface (reverse route injection).
Configure IKE:
ha3 running config# / vrf untrusted ike
ha3 running ike# ike-policy-template nodeB
ha3 running ike-proposal-1# enc-alg aes256-cbc
ha3 running ike-proposal-1# auth-alg hmac-sha256
ha3 running ike-proposal-1# dh-group modp2048
ha3 running ike-proposal-1# ...
ha3 running ike-policy-template nodeB# unique-sa replace
ha3 running ike-policy-template nodeB# rekey-time 2800
ha3 running ike-policy-template nodeB# dpd-delay 10
ha3 running ike-policy-template nodeB# ..
ha3 running ike# ipsec-policy-template nodeB
ha3 running esp-proposal 1# esp-proposal 1
ha3 running esp-proposal 1# enc-alg aes256-cbc
ha3 running esp-proposal 1# auth-alg hmac-sha256
ha3 running esp-proposal 1# ..
ha3 running ipsec-policy-template nodeB# start-action none
ha3 running ipsec-policy-template nodeB# close-action none
ha3 running ipsec-policy-template nodeB# dpd-action clear
ha3 running ipsec-policy-template nodeB# rekey-time 2800
ha3 running ipsec-policy-template nodeB# ..
ha3 running ike# vpn nodeB
ha3 running vpn nodeB# ike-policy template nodeB
ha3 running vpn nodeB# ipsec-policy template nodeB
ha3 running vpn nodeB# local-address 55.55.55.1
ha3 running vpn nodeB# security-policy nodeB
ha3 running security-policy nodeB# local-ts subnet 10.77.77.0/24
ha3 running security-policy nodeB# remote-ts subnet 10.7.0.0/24
ha3 running security-policy nodeB# svti-id-in 10
ha3 running security-policy nodeB# svti-id-out 10
ha3 running security-policy nodeB# inject-routes
ha3 running inject-routes# next-hop svti-trusted
ha3 running inject-routes# vrf trusted
ha3 running inject-routes# ..
ha3 running security-policy nodeB# ..
ha3 running vpn nodeB# ..
ha3 running ike# pre-shared-key femto
ha3 running pre-shared-key femto# secret 0x12345678
ha3 running pre-shared-key femto# ..
ha3 running ike# /
ha3 running config#
Configure IKE HA:
ha3 running config#! / vrf untrusted ike
ha3 running ike#! ha
ha3 running ha#! listen-ha-group ha_ipsec
ha3 running ha# peer peer1
ha3 running peer2# local-address 192.168.2.3
ha3 running peer2# remote-address 192.168.2.1
ha3 running peer2# ..
ha3 running ha# peer peer2
ha3 running peer3# local-address 192.168.2.3
ha3 running peer3# remote-address 192.168.2.2
ha3 running peer3# ..
ha3 running ha# /
ha3 running config#
Finally, configure IKE to encrypt the traffic on the HA network (transmitted
via the SVTI insterface svti-ha):
ha3 running config#
ha3 running config# / vrf ha ike
ha3 running ike# ike-policy-template ikepol-hasync
ha3 running ike-policy-template ikepol-hasync# ike-proposal 1
ha3 running ike-proposal 1#! enc-alg aes128-cbc
ha3 running ike-proposal 1#! auth-alg hmac-sha1
ha3 running ike-proposal 1#! prf-alg hmac-sha1
ha3 running ike-proposal 1#! dh-group modp1024
ha3 running ike-proposal 1# ..
ha3 running ike-policy-template ikepol-hasync# keying-tries 0
ha3 running ike-policy-template ikepol-hasync# unique-sa replace
ha3 running ike-policy-template ikepol-hasync# ..
ha3 running ike# ipsec-policy-template ipsecpol-hasync
ha3 running ipsec-policy-template ipsecpol-hasync# esp-proposal 1
ha3 running esp-proposal 1#! enc-alg aes128-cbc
ha3 running esp-proposal 1#! auth-alg hmac-sha1
ha3 running esp-proposal 1# dh-group modp1024
ha3 running esp-proposal 1# esn true
ha3 running esp-proposal 1# ..
ha3 running ipsec-policy-template ipsecpol-hasync# ..
ha3 running ike# vpn vpn-hasync31
ha3 running vpn vpn-hasync31#! ike-policy
ha3 running ike-policy#! template ikepol-hasync
ha3 running ike-policy#! ..
ha3 running vpn vpn-hasync31#! ipsec-policy
ha3 running ipsec-policy#! template ipsecpol-hasync
ha3 running ipsec-policy# ..
ha3 running vpn vpn-hasync31# local-id segw-2
ha3 running vpn vpn-hasync31# local-address 192.168.1.3
ha3 running vpn vpn-hasync31# remote-address 192.168.1.1
ha3 running vpn vpn-hasync31# security-policy svti-hasync31
ha3 running security-policy svti-hasync31# local-ts subnet 192.168.2.3
ha3 running security-policy svti-hasync31# remote-ts subnet 192.168.2.1
ha3 running security-policy svti-hasync31# svti-id-in 33
ha3 running security-policy svti-hasync31# svti-id-out 33
ha3 running security-policy svti-hasync31# ..
ha3 running vpn vpn-hasync31# ..
ha3 running ike# vpn vpn-hasync32
ha3 running vpn vpn-hasync32#! ike-policy
ha3 running ike-policy#! template ikepol-hasync
ha3 running ike-policy#! ..
ha3 running vpn vpn-hasync32#! ipsec-policy
ha3 running ipsec-policy#! ..
ha3 running vpn vpn-hasync32#! local-id segw-2
ha3 running vpn vpn-hasync32#! local-address 192.168.1.3
ha3 running vpn vpn-hasync32#! remote-address 192.168.1.2
ha3 running vpn vpn-hasync32#! security-policy svti-hasync32
ha3 running security-policy svti-hasync32#! local-ts subnet 192.168.2.3
ha3 running security-policy svti-hasync32#! remote-ts subnet 192.168.2.2
ha3 running security-policy svti-hasync32#! svti-id-in 33
ha3 running security-policy svti-hasync32#! svti-id-out 33
ha3 running security-policy svti-hasync32#! /
ha3 running config#
Once everything is configured, commit the configuration:
ha3 running config#
ha3 running config# commit
Configuration committed.
Encrypting the synchronization messages¶
The HA synchronization data exchanged between ha1 and ha2 contains sensitive information such as cryptographic keys. Now the network path used to convey this information (eth3 in the HA cluster figure) is not necessarily a trusted network.
It is possible to use IPsec to encrypt this traffic. In order to isolate the IKE and IPsec traffic used for securing the HA synchronization exchanges from the other IKE and IPsec traffic, the underlying interface (here eth3) is moved to a separate VRF hasync, and an IPsec-protected cross-VRF logical interface is created over this link. The HA synchronization is performed via the logical interface instead of the eth3 interface.
The logical interface may be for example an SVTI interface (that does a direct IPsec encapsulation of the HA exchanges) or a GRE interface (that first does a GRE encapsulation of the HA exchanges, then the GRE packets are IPsec encrypted).
Using a cross-vrf svti interface¶
A cross-vrf SVTI logical interface is used to convey the HA synchronization messages.
The SVTI interface is in the same VRF as the IKE service (here main) and its link-VRF is hasync. Then IKE is configured in VRF hasync to encrypt the data sent over the SVTI interface in tunnel mode.
The endpoints of the HA synchronization traffic (ha1 10.160.0.1, ha2 10.160.0.2) are configured on the SVTI interface. All data sent over the SVTI interface is encrypted in tunnel mode between the adresses of the underlying physical interface (ha1 10.150.0.1, ha2 10.150.0.2).
ha1 cli configuration¶
ha1 running config# show config nodefault
vrf main
(...)
interface
svti svti-hasync
ipv4
address 10.160.0.1/24
..
svti-id 888
link-vrf hasync
..
..
(...)
ike
ha
listen-ha-group g
local-address 10.160.0.1
remote-address 10.160.0.2
seqnum-sync
sync-period-time 1s
sync-period-packets 64
..
..
..
(...)
vrf hasync
ike
pre-shared-key psk-hasync
id primary@ha
id secondary@ha
secret 0xf0043c9a1e4c22177c4e01dd909a96b2
..
ike-policy-template ikepol-hasync
ike-proposal 1
enc-alg aes128-cbc
auth-alg hmac-sha1
prf-alg hmac-sha1
dh-group modp1024
..
keying-tries 0
unique-sa replace
..
ipsec-policy-template ipsecpol-hasync
esp-proposal 1
enc-alg aes128-cbc
auth-alg hmac-sha1
dh-group modp1024
esn true
..
..
vpn vpn-hasync
ike-policy
template ikepol-hasync
..
ipsec-policy
template ipsecpol-hasync
..
local-address 10.150.0.1
remote-address 10.150.0.2
local-id primary@ha
remote-id secondary@ha
security-policy svti-hasync
local-ts subnet 0.0.0.0/0
remote-ts subnet 0.0.0.0/0
svti-id-in 888
svti-id-out 888
..
..
..
interface
physical ntfp3
ipv4
address 10.150.0.1/24
..
port pci-b0s6
..
..
..
ha2 cli configuration¶
ha2 running config# show config nodefault
vrf main
(...)
interface
svti svti-hasync
ipv4
address 10.160.0.2/24
..
svti-id 888
link-vrf hasync
..
..
(...)
ike
ha
listen-ha-group g
local-address 10.160.0.2
remote-address 10.160.0.1
seqnum-sync
sync-period-time 1s
sync-period-packets 64
..
..
..
(...)
vrf hasync
ike
pre-shared-key psk-hasync
id primary@ha
id secondary@ha
secret 0xf0043c9a1e4c22177c4e01dd909a96b2
..
ike-policy-template ikepol-hasync
ike-proposal 1
enc-alg aes128-cbc
auth-alg hmac-sha1
prf-alg hmac-sha1
dh-group modp1024
..
keying-tries 0
unique-sa replace
..
ipsec-policy-template ipsecpol-hasync
esp-proposal 1
enc-alg aes128-cbc
auth-alg hmac-sha1
dh-group modp1024
esn true
..
..
vpn vpn-hasync
ike-policy
template ikepol-hasync
..
ipsec-policy
template ipsecpol-hasync
..
local-address 10.150.0.2
remote-address 10.150.0.1
local-id secondary@ha
remote-id primary@ha
security-policy svti-hasync
local-ts subnet 0.0.0.0/0
remote-ts subnet 0.0.0.0/0
svti-id-in 888
svti-id-out 888
..
..
..
interface
physical ntfp3
ipv4
address 10.150.0.2/24
..
port pci-b0s6
..
..
..
Using a cross-vrf gre interface¶
A cross-vrf GRE logical interface is used to convey the HA synchronization messages.
The GRE interface is in the same VRF as the IKE service (here main) and its link-VRF is hasync. Then IKE is configured in VRF hasync to encrypt the GRE traffic in transport mode.
The endpoints of the HA synchronization traffic (ha1 10.160.0.1, ha2 10.160.0.2) are configured on the GRE interface. Its encapsulation addresses are those of the underlying physical interface (ha1 10.150.0.1, ha2 10.150.0.2). The GRE-encapsulated packets are encrypted in transport mode.
ha1 cli configuration¶
ha1 running config# show config nodefault
vrf main
(...)
interface
gre gre-hasync
local 10.150.0.1
remote 10.150.0.2
ipv4
address 10.160.0.1/24
..
link-vrf hasync
..
..
(...)
ike
ha
listen-ha-group g
local-address 10.160.0.1
remote-address 10.160.0.2
seqnum-sync
sync-period-time 1s
sync-period-packets 64
..
..
..
(...)
vrf hasync
ike
pre-shared-key psk-hasync
id primary@ha
id secondary@ha
secret 0xf0043c9a1e4c22177c4e01dd909a96b2
..
ike-policy-template ikepol-hasync
ike-proposal 1
enc-alg aes128-cbc
auth-alg hmac-sha1
prf-alg hmac-sha1
dh-group modp1024
..
keying-tries 0
unique-sa replace
..
ipsec-policy-template ipsecpol-hasync
esp-proposal 1
enc-alg aes128-cbc
auth-alg hmac-sha1
dh-group modp1024
esn true
..
..
vpn vpn-hasync
ike-policy
template ikepol-hasync
..
ipsec-policy
template ipsecpol-hasync
..
local-address 10.150.0.1
remote-address 10.150.0.2
local-id primary@ha
remote-id secondary@ha
security-policy gre-hasync
local-ts protocol 47
remote-ts protocol 47
mode transport
..
..
..
interface
physical ntfp3
ipv4
address 10.150.0.1/24
..
port pci-b0s6
..
..
..
ha2 cli configuration¶
ha2 running config# show config nodefault
vrf main
(...)
interface
gre gre-hasync
local 10.150.0.2
remote 10.150.0.1
ipv4
address 10.160.0.2/24
..
link-vrf hasync
..
..
(...)
ike
ha
listen-ha-group g
local-address 10.160.0.2
remote-address 10.160.0.1
seqnum-sync
sync-period-time 1s
sync-period-packets 64
..
..
..
(...)
vrf hasync
ike
pre-shared-key psk-hasync
id primary@ha
id secondary@ha
secret 0xf0043c9a1e4c22177c4e01dd909a96b2
..
ike-policy-template ikepol-hasync
ike-proposal 1
enc-alg aes128-cbc
auth-alg hmac-sha1
prf-alg hmac-sha1
dh-group modp1024
..
keying-tries 0
unique-sa replace
..
ipsec-policy-template ipsecpol-hasync
esp-proposal 1
enc-alg aes128-cbc
auth-alg hmac-sha1
dh-group modp1024
esn true
..
..
vpn vpn-hasync
ike-policy
template ikepol-hasync
..
ipsec-policy
template ipsecpol-hasync
..
local-address 10.150.0.2
remote-address 10.150.0.1
local-id secondary@ha
remote-id primary@ha
security-policy gre-hasync
local-ts protocol 47
remote-ts protocol 47
mode transport
..
..
..
interface
physical ntfp3
ipv4
address 10.150.0.2/24
..
port pci-b0s6
..
..
..
Advanced options¶
Sequence number synchronization parameters¶
IPsec SAs sequence numbers and statistics are regularly synchronized from the active node to the backup node. In case of switchover, this enables the new master node to take over the IPsec dataplane processing with proper sequence numbers:
For an output SA, the output sequence number 1 on the backup node should be greater or equal to the last sequence number used by this SA on the master node. Otherwise, the remote IPsec peer is likely to drop some IPsec packets sent by the new master until the sequence numbers comply to its replay window state.
For an input SA, the input sequence number 2 on the backup node should be close to the highest sequence number received on the master node. Otherwise the new master node is vulnerable to accepting replayed packets sent by an attacker, because its replay window is too late.
- 1
i.e. the record of the highest SA sequence number of a sent packet protected with this SA
- 2
i.e. the record of the highest SA sequence number of a received packet protected with this SA
The pace at which sequence number synchronization is performed is configurable
in the ha seqnum-sync sub-context:
ha1 running vrf main# ike ha seqnum-sync
ha1 running seqnum-sync# sync-period-time 10s
ha1 running seqnum-sync# sync-period-packets 2
ha1 running seqnum-sync# oseq-shift 65536
ha1 running seqnum-sync# / vrf main
sync-period-timeis the minimum time between two sequence number updates. An update is sent to the backup node only if the sequence number changed since last update (default 10s, 0 disables the time-based periodic update).sync-period-packetsis the number of packets between two sequence number updates: if the input or output sequence number of an IPsec SA changes of at least that number since last synchronization, then an update is sent to the backup node (default 2, 0 disables the packet-based periodic update).oseq-shiftis a value added to IPsec SA output sequence numbers when they are synchronized to the backup node: since sequence numbers cannot be synchronized in real-time, the output sequence numbers communicated to the backup node are always late compared to the master node. Adding this value to the current output sequence number aims at ensuring that the output sequence number on the backup node always remains greater than on the master, even during two updates. It should be greater than the number of packets processed between two sequence number updates (default 65536).
DSCP of HA synchronization messages¶
HA synchronization messages are exchanged between the master and backup nodes via the TCP protocol on port 4510. This traffic is rather critical and needs to be safeguarded by QoS and control plane protection mechanisms.
An option enables setting the DSCP of these messages:
ha1 running config# / vrf main ike ha
ha1 running ha# dscp 56
ha1 running ha# commit
This example sets the DSCP of HA synchronization messages to 56.
HA-compatible virtual IP pools¶
IKE enables assigning a * VIP * and optional attributes during an IKE negotiation, i.e. an IKE initiator may request an additional IP address from the responder to use as inner IPsec tunnel address.
VIPs and attributes are exchanged using an IKEv2 Configuration Payload (or an IKEv1 Config Mode) exchange.
Additional parameters may be assigned during this exchange, such as a DNS server address, a NetBIOS server address or a DHCP server address.
To proceed, the responder maintains a pool of VIPs. (see IKE virtual IP pools).
If the IKE configuration makes use of VIP pools and HA IKE is enabled, then VIP leases must be synchronized between the master and the backup node.
This requires using specific HA-synchronized VIP pools. These pools are less flexible than standard VIP pools:
address pools can only be defined as subnets, not ranges of addresses.
there is no state information about these pools
When enabling HA IKE, be careful of using a virtual pool defined in the ha
context, because virtual pools defined directly in the ike context are not
synchronized between the master and backup node.
Define the pool. The address specification is mandatory, other attributes are
optional:
ha1 running vrf main# ike
ha1 running ike# ha
ha1 running ha# pool my-ha-pool
ha1 running pool my-ha-pool#! address 192.168.0.0/24
ha1 running pool my-ha-pool# dns 192.168.3.99
ha1 running pool my-ha-pool# ..
ha1 running ha#
addressis the set of addresses that can be assigned (IPv4 or IPv6 subnet).dnsis an optional list of DNS server addresses (IPv4 or IPv6).nbnsis an optional list of NetBIOS server addresses (IPv4 or IPv6).dhcpis an optional list of DHCP server addresses (IPv4 or IPv6).subnetis an optional list of subnets protected by the responder (IPv4 or IPv6). (attributes INTERNAL_IP4_SUBNET/INTERNAL_IP6_SUBNET in RFC 7296)netmaskis an optional value for the internal network’s netmask (IPv4). (attribute INTERNAL_IP4_NETMASK in RFC 7296)
Use it in a vpn:
ha1 running ike# vpn vpn-secgw
ha1 running vpn vpn-secgw# vip
ha1 running vpn vpn-secgw# vip-pool my-ha-pool
ha1 running vpn vpn-secgw# ..
ha1 running ike#
Display the IKE configuration:
ha1 running ike# show config nodefault
ike
vpn vpn-secgw
vip-pool my-ha-pool
(...)
ha
pool my-ha-pool
address 192.168.0.0/24
dns 192.168.3.99
..
(...)
(...)
Troubleshooting¶
“(unknown)” SAs after failover¶
If a failover occurs while an SA is being created or rekeyed it may result in an extra SA named “(unknown)” being permanently listed:
ha1# show ike vrf main ike-sa
VPN Local-Addr Local-ID Remote-Addr Remote-ID State Child-SAs
(unknown) 10.175.0.1 secgw.6wind.net 10.225.0.1 enodeb1.6wind.net connecting 0
Such SAs do not affect the system in any way other than cluttering the list and
can be cleared manually using the flush command
(see IKE Manually clearing SAs).
See also
The IKE command reference for details.