Note

High Availability IKE requires an IPsec Application License.

High Availability IKE

High Availability Internet Key Exchange (HA IKE) is an IKE extension that enables to perform stateful synchronization of IKE between two HA nodes in active/backup mode.

HA IKE may be configured between two nodes forming an HA cluster: the IKE internal states (IKE SAs and CHILD SAs) and IPsec SAs sequence numbers are synchronized from the active node to the backup node.

If the HA state is switched between the two nodes, the new master node will be able to take over the IKE negotiations and IPsec dataplane traffic.

Overview

The HA state of a node can be statically configured using CLI/netconf or dynamically using the VRRP service.

../../../_images/ha-ike-cluster.svg

HA IKE parameters are configured in the ha sub-context of ike.

Enter the ha sub-context on ha1:

ha1 running config# vrf main
ha1 running vrf main# ike
ha1 running ike# ha
ha1 running ha#!

Configure HA IKE parameters:

ha1 running ha#! local-address 10.150.0.1
ha1 running ha#! remote-address 10.150.0.2
ha1 running ha#! listen-ha-group ha-group1
ha1 running ha#
  • local-address and remote-address are the IPv4 or IPv6 addresses of the two HA nodes.

  • listen-ha-group is the high-availability group that controls the activity state of this HA node. See High-availability Groups for more information.

Display HA IKE parameters:

ha1 running ha# show config
ha
    enabled true
    listen-ha-group my-ha-group
    local-address 10.150.0.1
    remote-address 10.150.0.2
    seqnum-sync
        oseq-shift 65536
        sync-period-time 10s
        sync-period-packets 2
        ..
    ..

On ha2, the local-address and remote-address swapped:

ha2 running config# vrf main ike ha
ha2 running ha#! local-address 10.150.0.2
ha2 running ha#! remote-address 10.150.0.1
ha2 running ha#! listen-ha-group ha-group1

Note

The following parameters may be seamlessly modified without restarting the IKE service: local-address, remote-address, and all parameters in seqnum-sync.

However, modifying any of the other parameters in the ha context will automatically restart the IKE service and hence tear down all established IKE connections.

Use case: L2 HA IKE cluster with VRRP

Layer 2 HA is a use case where the two HA nodes are directly connected on a local network, typically an ethernet network. The traffic that must be processed is steered to the master node at the layer 2 level, typically by an ethernet switch thanks to the VRRP protocol.

../../../_images/ha-ike-vrrp.svg

In this use case, two devices ha1 and ha2 are configured as a redundant security gateway, performing IKE negotiations with a remote security gateway SecGW.

The activity of each HA node is determined by the VRRP protocol (see VRRP command reference for details about VRRP): the VRRP protocol elects the master node depending on the link states and the priorities of nodes. It also controls the path of IP traffic by assigning VRRP virtual router MAC addresses (and optionally IP addresses) to the elected master node.

The two HA devices must be configured exactly the same, except for HA parameters (VRRP and HA IKE).

ha1 CLI configuration

Configure device hostname:

vsr running config# system hostname ha1
vsr running config# commit
Configuration committed.

Configure the HA group:

vsr running config# ha group ha-group1
ha1 running group ha-group1#! ..
ha1 running ha#! ..

Note

The HA group maintains the node high-availability state. It is controlled by the VRRP protocol (via the notify-ha-group command) and monitored by HA IKE (via the listen-ha-group command). Only one controller can be defined for an HA group.

Move to VRF main configuration:

ha1 running config#! vrf main
ha1 running vrf main#!

Configure the network interfaces (adapt port ids to your hardware):

ha1 running vrf main#! interface physical eth1
ha1 running physical eth1#! port pci-b0s3
ha1 running physical eth1#! ipv4 address 10.22.0.1/24
ha1 running physical eth1#! ..
ha1 running interface#! physical eth2
ha1 running physical eth2#! port pci-b0s4
ha1 running physical eth2#! ipv4 address 10.23.0.1/24
ha1 running physical eth2#! ..
ha1 running interface#! physical eth3
ha1 running physical eth3#! port pci-b0s5
ha1 running physical eth3#! ipv4 address 10.150.0.1/24
ha1 running physical eth3#! ..
ha1 running interface#! loopback loopback0
ha1 running loopback loopback0#! ipv4 address 10.175.0.1/32
ha1 running loopback loopback0#! ..
ha1 running interface#! ..

Configure routes:

ha1 running vrf main#! routing
ha1 running routing#! static
ha1 running static#! ipv4-route 10.250.0.0/24 next-hop 10.200.0.1
ha1 running static#! ipv4-route 10.225.0.0/24 next-hop 10.200.0.1
ha1 running static#! ..
ha1 running routing#! ..

Configure VRRP:

ha1 running vrf main#! interface vrrp vrrp51
ha1 running vrrp vrrp52#! vrid 1
ha1 running vrrp vrrp51#! link-interface eth1
ha1 running vrrp vrrp51#! priority 100
ha1 running vrrp vrrp51#! advertisement-interval 1000
ha1 running vrrp vrrp51#! virtual-address 10.100.0.2/24
ha1 running vrrp vrrp51#! ..
ha1 running interface#! vrrp vrrp52
ha1 running vrrp vrrp52#! vrid 1
ha1 running vrrp vrrp52#! link-interface eth2
ha1 running vrrp vrrp52#! priority 100
ha1 running vrrp vrrp52#! advertisement-interval 1000
ha1 running vrrp vrrp52#! virtual-address 10.200.0.2/24
ha1 running vrrp vrrp52#! ..
ha1 running interface#! ..
ha1 running vrf main#! vrrp group group1
ha1 running group group1#! instance vrrp51
ha1 running group group1#! instance vrrp52
ha1 running group group1#! notify-ha-group ha-group1
ha1 running group group1# ..
ha1 running vrrp# ..

Show the configuration:

ha1 running vrf main# show config nodefault
vrf main
    interface
        vrrp vrrp51
            link-interface eth1
            vrid 1
            virtual-address 10.100.0.2/24
            ..
        vrrp vrrp52
            link-interface eth2
            vrid 1
            virtual-address 10.200.0.2/24
            ..
        ..
    vrrp
        group group1
            instance vrrp51
            instance vrrp52
            notify-ha-group ha-group1
            ..
        ..
    ..

Configure IKE:

ha1 running vrf main# ike
ha1 running ike# ike-policy-template ike1
ha1 running ike-policy-template ike1# ike-proposal 1 enc-alg aes128-cbc auth-alg hmac-sha1 dh-group modp1024
ha1 running ike-policy-template ike1# rekey-time 2h
ha1 running ike-policy-template ike1# ..
ha1 running ike# ipsec-policy-template ipsec1
ha1 running ipsec-policy-template ipsec1# esp-proposal 1 enc-alg aes128-cbc auth-alg hmac-sha1 esn true
ha1 running ipsec-policy-template ipsec1# rekey-time 1h
ha1 running ipsec-policy-template ipsec1# replay-window 1024
ha1 running ipsec-policy-template ipsec1# ..
ha1 running ike# vpn vpn-secgw
ha1 running vpn vpn-secgw#! ike-policy template ike1
ha1 running vpn vpn-secgw#! ipsec-policy template ipsec1
ha1 running vpn vpn-secgw# local-address 10.175.0.1
ha1 running vpn vpn-secgw# remote-address 10.225.0.1
ha1 running vpn vpn-secgw# security-policy site-to-secgw-site
ha1 running security-policy site-to-secgw-site# local-ts subnet 10.100.0.64/26
ha1 running security-policy site-to-secgw-site# remote-ts subnet 10.250.0.192/26
ha1 running security-policy site-to-secgw-site# ..
ha1 running vpn vpn-secgw# ..
ha1 running ike# pre-shared-key secgw
ha1 running pre-shared-key secgw#! id 10.225.0.1
ha1 running pre-shared-key secgw#! secret 0sBzAyaM5PTcnTHi/yRA1lARpAoRetSzP8
ha1 running pre-shared-key secgw# ..
ha1 running ike#

Show IKE configuration:

ha1 running ike# show config nodefault
ike
    pre-shared-key secgw
        id 10.225.0.1
        secret 0sBzAyaM5PTcnTHi/yRA1lARpAoRetSzP8
        ..
    global-options
        dos-protection
            ..
        sp-hash-ipv4
        sp-hash-ipv6
        ..
    ike-policy-template ike1
        ike-proposal 1
            enc-alg aes128-cbc
            auth-alg hmac-sha1
            dh-group modp1024
            ..
        rekey-time 2h
        ..
    ipsec-policy-template ipsec1
        esp-proposal 1
            enc-alg aes128-cbc
            auth-alg hmac-sha1
            esn true
            ..
        replay-window 1024
        ..
    vpn vpn-secgw
        ike-policy
            template ike1
            ..
        ipsec-policy
            template ipsec1
            ..
        local-address 10.175.0.1
        remote-address 10.225.0.1
        security-policy site-to-secgw-site
            local-ts subnet 10.100.0.64/26
            remote-ts subnet 10.250.0.192/26
            ..
        ..
    ..

Configure HA IKE:

ha1 running ike# ha
ha1 running ha#! local-address 10.150.0.1
ha1 running ha#! remote-address 10.150.0.2
ha1 running ha#! listen-ha-group ha-group1
ha1 running ha# ..
ha1 running ike# commit
Configuration committed.
ha1 running ike#

Show HA IKE configuration:

ha1 running ike# show config nodefault ha
ha
    listen-ha-group ha-group1
    local-address 10.150.0.1
    remote-address 10.150.0.2
    seqnum-sync
        ..
    ..

ha2 CLI configuration

A similar configuration is used for ha2. The differences are the hostname, the physical interfaces addresses, VRRP parameters and IKE HA parameters.

The IKE parameters (except HA ones) must be strictly identical.

ha2 running config# show config nodefault
config
    vrf main
        interface
            physical eth1
                ipv4
                    address 10.22.0.2/24
                    ..
                ..
            physical eth2
                ipv4
                    address 10.23.0.2/24
                    ..
                ..
            physical eth3
                ipv4
                    address 10.150.0.2/24
                    ..
                ..
            loopback loopback0
                ipv4
                    address 10.175.0.1/32
                    ..
                ..
            vrrp vrrp51
                link-interface eth1
                vrid 1
                virtual-address 10.100.0.2/24
                ..
            vrrp vrrp52
                link-interface eth2
                vrid 1
                virtual-address 10.200.0.2/24
                ..
            ..
        routing
            static
                ipv4-route 10.250.0.0/24
                    next-hop 10.200.0.1
                    ..
                ipv4-route 10.225.0.0/24
                    next-hop 10.200.0.1
                    ..
                ..
            ..
        vrrp
            group group1
                instance vrrp51
                instance vrrp52
                notify-ha-group ha-group1
                ..
            ..
        ike
            pre-shared-key secgw
                id 10.225.0.1
                secret 0sBzAyaM5PTcnTHi/yRA1lARpAoRetSzP8
                ..
            global-options
                dos-protection
                    ..
                sp-hash-ipv4
                sp-hash-ipv6
                ..
            ha
                listen-ha-group ha-group1
                local-address 10.150.0.2
                remote-address 10.150.0.1
                seqnum-sync
                    ..
                ..
            ike-policy-template ike1
                ike-proposal 1
                    enc-alg aes128-cbc
                    auth-alg hmac-sha1
                    dh-group modp1024
                    ..
                rekey-time 2h
                ..
            ipsec-policy-template ipsec1
                esp-proposal 1
                    enc-alg aes128-cbc
                    auth-alg hmac-sha1
                    esn true
                    ..
                replay-window 1024
                ..
            vpn vpn-secgw
                ike-policy
                    template ike1
                    ..
                ipsec-policy
                    template ipsec1
                    ..
                local-address 10.175.0.1
                remote-address 10.225.0.1
                security-policy site-to-secgw-site
                    local-ts subnet 10.100.0.64/26
                    remote-ts subnet 10.250.0.192/26
                    ..
                ..
            ..
        ..
    system
        hostname ha2
        ..
    ha
        group ha-group1
            ..
        ..
    ..

Use case: L3 HA IKE cluster with VRRP

Layer 3 HA is a use case where the two HA nodes are not directly connected, they are on separate networks and can only contact each other via IP routing. The traffic that must be processed is steered to the master node at the layer 3 level, i.e. by routing, thanks to routing protocols. In our example, we use BGP.

../../../_images/ha-ike-routing.svg

The HA synchronization channel hasync may be a simple routed TCP connection, but should preferably go through a reliable and secured path, possibly an IPsec-encrypted GRE tunnel or SVTI interface, as described in Encrypting the synchronization messages, depending whether the network between HA nodes is secured or not. In our example, we use a cross-VRF GRE interface over an internal VLAN (without IPsec protection).

The VRRP protocol is used in unicast mode to elect the master and backup nodes (see VRRP command reference for details about VRRP). ha1 and ha2 exchange unicast point-to-point VRRP messages instead of multicast ones. The VRRP protocol elects the master node depending on the the connectivity between nodes and their VRRP priorities. It also tracks the link state of the hasync channel, as well as the connectivity with the routers to the trusted and untrusted networks, via the BFD protocol. Finally, it installs a VRRP VIP on the master node (10.55.55.1), which is the IKE and IPsec tunnel end point.

Both HA nodes advertise the IKE tunnel end point (10.55.55.1) in the routing protocols, but with a different cost. However only the master node owns this address. In our example, we use BGP AS-PATH prepending to artificially lengthen the AS-PATH on the backup node, and hence increase the cost of the routes it advertises.

To separate the trusted and untrusted networks, a VRF is created for each zone. This enables to partition routing and makes sure crossing the untrusted network will require transiting via the redundant security gateway, that will provide IPsec protection.

ha1 CLI configuration

Configure device hostname:

vsr running config# system hostname ha1
vsr running config# commit
Configuration committed.

In this example, ha1 only has one physical ethernet interface (eth1) and uses VLANs in different VRFs to partition the trusted, untrusted and hasync networks.

../../../_images/ha-ike-routing-ha1.svg

Configure the physical network interface (adapt port ids to your hardware) in VRF main:

ha1 running config# vrf main
ha1 running vrf main#! interface physical eth1
ha1 running physical eth1#! port pci-b0s4
ha1 running physical eth1# mtu 9000
ha1 running physical eth1# /
ha1 running config#

Then configure the VRF and VLAN for the trusted network:

ha1 running config# / vrf trusted interface
ha1 running interface# vlan vlan_trusted
ha1 running vlan vlan_trusted#! vlan-id 2
ha1 running vlan vlan_trusted#! link-interface eth1
ha1 running vlan vlan_trusted# link-vrf main
ha1 running vlan vlan_trusted# ipv4 address 10.2.0.1/30
ha1 running vlan vlan_trusted# /
ha1 running config#

Then configure the VRF and VLAN for the untrusted network:

ha1 running config# / vrf untrusted interface
ha1 running interface# vlan vlan_untrusted
ha1 running vlan vlan_untrusted#! vlan-id 307
ha1 running vlan vlan_untrusted#! link-interface eth1
ha1 running vlan vlan_untrusted# link-vrf main
ha1 running vlan vlan_untrusted# ipv4 address 10.55.0.2/30
ha1 running vlan vlan_untrusted# /
ha1 running config#

Then configure the VRF and VLAN for the HA synchronization network:

ha1 running config# / vrf ha interface
ha1 running interface# vlan vlan_ha
ha1 running vlan vlan_ha#! vlan-id 10
ha1 running vlan vlan_ha#! link-interface eth1
ha1 running vlan vlan_ha# link-vrf main
ha1 running vlan vlan_ha# ipv4 address 192.168.1.0/31
ha1 running vlan vlan_ha# /
ha1 running config#

In order to perform cross-VRF IPsec, the trusted traffic will be sent through an SVTI cross-VRF interface in VRF trusted, with link-VRF untrusted:

ha1 running config# / vrf trusted interface
ha1 running interface# svti svti-trusted
ha1 running svti svti-trusted# svti-id 10
ha1 running svti svti-trusted# link-vrf untrusted
ha1 running svti svti-trusted# /

Now configure routing on the trusted network. The connectivity to the BGP peer is tracked via the BFD protocol. A route map TRUSTED-OUT is attached to the routes learnt via this peer, in order to adjust the cost of the learnt routes, depending on ha1 HA activity. The kernel routes of the trusted network are redistributed via BGP.

ha1 running config# / vrf trusted routing
ha1 running routing# bgp
ha1 running bgp# as 65111
ha1 running bgp# router-id 10.5.5.5
ha1 running bgp# ebgp-requires-policy false
ha1 running bgp# address-family ipv4-unicast redistribute kernel
ha1 running bgp# neighbor 10.2.0.2
ha1 running neighbor 10.2.0.2# remote-as 65123
ha1 running neighbor 10.2.0.2# address-family ipv4-unicast
ha1 running ipv4-unicast# as-override false
ha1 running ipv4-unicast# route-map out route-map-name TRUSTED-OUT
ha1 running running ipv4-unicast# ..
ha1 running running address-family# ..
ha1 running running neighbor 10.2.0.2# track bfd
ha1 running running neighbor 10.2.0.2# /
ha1 running config#

Now configure routing on the untrusted network. The connectivity to the BGP peer is tracked via the BFD protocol. A route map UNTRUSTED-OUT is attached to the routes learnt via this peer, in order to adjust the cost of the learnt routes, depending on ha1 HA activity. The IKE tunnel end point (10.55.55.1/32) is advertised by BGP.

ha1 running config# / vrf untrusted routing
ha1 running routing# bgp
ha1 running bgp# as 65111
ha1 running bgp# router-id 10.5.5.5
ha1 running bgp# ebgp-requires-policy false
ha1 running bgp# address-family ipv4-unicast network 10.55.55.1/32
ha1 running bgp# neighbor 10.55.0.1
ha1 running neighbor 10.55.0.1# remote-as 65123
ha1 running neighbor 10.55.0.1# address-family ipv4-unicast
ha1 running ipv4-unicast# as-override false
ha1 running ipv4-unicast# route-map out route-map-name UNTRUSTED-OUT
ha1 running running ipv4-unicast# ..
ha1 running running address-family# ..
ha1 running neighbor 10.55.0.1# track bfd
ha1 running neighbor 10.55.0.1# /
ha1 running config#

The hasync network is an internal network that interconnects the HA nodes and enables to exchange synchronization data. We chose to use a separate VRF and to access it via a different VLAN. However, the IKE service runs in VRF untrusted and the HA module needs to establish a TCP connection with the HA peer in VRF untrusted. We therefore create a cross-VRF GRE interface in VRF untrusted, with link-VRF ha:

ha1 running config# / vrf untrusted interface
ha1 running interface# gre gre-hasync
ha1 running gre gre-hasync# ttl 64
ha1 running gre gre-hasync# link-vrf ha
ha1 running gre gre-hasync# local 192.168.1.0
ha1 running gre gre-hasync# remote 192.168.1.1
ha1 running gre gre-hasync# key both 888
ha1 running gre gre-hasync# ipv4 address 192.168.2.0/31
ha1 running gre gre-hasync# /
ha1 running config#

Create a loopback interface:

ha1 running config# / vrf untrusted interface
ha1 running interface# loopback loopback0
ha1 running loopback loopback0# ipv4 address 10.5.5.5/32
ha1 running loopback loopback0# /
ha1 running config#

Configure the route maps used by the BGP configurations:

ha1 running config# / routing

ha1 running routing# ipv4-prefix-list IPSEC_TEP
ha1 running ipv4-prefix-list IPSEC_TEP# seq 10 address 10.55.55.1/32 policy permit
ha1 running ipv4-prefix-list IPSEC_TEP# ..

ha1 running routing# ipv4-prefix-list RAN
ha1 running ipv4-prefix-list RAN# seq 10 address 10.7.0.0/24 policy permit le 32
ha1 running ipv4-prefix-list RAN# ..

ha1 running routing# route-map UNTRUSTED-OUT
ha1 running route-map UNTRUSTED-OUT# seq 10
ha1 running route-map seq 10# policy permit
ha1 running route-map seq 10# match ip address prefix-list IPSEC_TEP
ha1 running route-map seq 10# match ha-group name ha_ipsec
ha1 running route-map seq 10# ..
ha1 running route-map UNTRUSTED-OUT# ..

ha1 running routing# route-map TRUSTED-OUT
ha1 running route-map TRUSTED-OUT# seq 10
ha1 running route-map seq 10# policy permit
ha1 running route-map seq 10# match ip address prefix-list RAN
ha1 running route-map seq 10# match ha-group name ha_ipsec
ha1 running route-map seq 10# ..
ha1 running route-map TRUSTED-OUT# /

ha1 running config#

Create two trackers that monitor the connectivity with the routers to the trusted and untrusted networks:

ha1 running config# / tracker bfd pe-bfd-trusted
ha1 running bfd pe-bfd-trusted# source 10.2.0.1
ha1 running bfd pe-bfd-trusted# address 10.2.0.2
ha1 running bfd pe-bfd-trusted# vrf trusted
ha1 running bfd pe-bfd-trusted# ..

ha1 running config# / tracker bfd pe-bfd-untrusted
ha1 running bfd pe-bfd-untrusted# source 10.55.0.2
ha1 running bfd pe-bfd-untrusted# address 10.55.0.1
ha1 running bfd pe-bfd-untrusted# vrf untrusted
ha1 running bfd pe-bfd-untrusted# /

ha1 running config#

We can finally configure HA. First configure the HA group:

ha1 running config# / ha group ha-group1
ha1 running group ha-group1#! ..
ha1 running ha#! /
ha1 running config#!

Note

The HA group maintains the node high-availability state. It is controlled by the VRRP protocol (via the notify-ha-group command) and monitored by HA IKE (via the listen-ha-group command). Only one controller can be defined for an HA group.

Configure VRRP in unicast mode. It also tracks the link state of the gre-hasync interface, as well as the connectivity with the routers to the trusted and untrusted networks, via the BFD protocol. Finally, it installs a VRRP VIP on the master node (10.55.55.1), which is the IKE and IPsec tunnel end point. It notifies the services interested by the HA status via the HA group ha_ipsec (IKE and routing):

ha1 running config# / vrf untrusted interface
ha1 running interface# vrrp vrrp_ha
ha1 running vrrp vrrp_ha# version 3
ha1 running vrrp vrrp_ha# link-interface gre-hasync
ha1 running vrrp vrrp_ha# use-vmac false
ha1 running vrrp vrrp_ha# vrid 10
ha1 running vrrp vrrp_ha# priority 200
ha1 running vrrp vrrp_ha# preempt-delay 30
ha1 running vrrp vrrp_ha# advertisement-interval 100
ha1 running vrrp vrrp_ha# track pe-bfd-trusted
ha1 running vrrp vrrp_ha# track pe-bfd-untrusted
ha1 running vrrp vrrp_ha# track-fast-path true
ha1 running vrrp vrrp_ha# unicast-peer 192.168.2.1
ha1 running vrrp vrrp_ha# virtual-address 10.55.55.1/32
ha1 running vrrp vrrp_ha# notify-ha-group ha_ipsec
ha1 running vrrp vrrp_ha# /

ha1 running config# / vrf untrusted vrrp
ha1 running vrrp# router-id 10.5.5.5
ha1 running vrrp# vrrp-startup-delay 30
ha1 running vrrp# /
ha1 running config#

Now that network connectivity and routing is configured, let us configure IKE. Here, ha1 is configured as a VPN concentrator that accepts IKE connections initiated by a remote peer. To support cross-VRF IPsec, the cross-VRF SVTI interface svti-trusted is used (VRF trusted, link-VRF untrusted, svti-id 10). The inject-routes option automatically adds routes to the remote traffic selector via the SVTI interface.

Configure IKE:

ha1 running config# / vrf untrusted ike
ha1 running ike# ike-policy-template nodeB
ha1 running ike-proposal-1# enc-alg aes256-cbc
ha1 running ike-proposal-1# auth-alg hmac-sha25
ha1 running ike-proposal-1# dh-group modp2048
ha1 running ike-proposal-1# ...
ha1 running ike-policy-template nodeB# unique-sa replace
ha1 running ike-policy-template nodeB# rekey-time 2800
ha1 running ike-policy-template nodeB# dpd-delay 10
ha1 running ike-policy-template nodeB# ..

ha1 running ike# ipsec-policy-template nodeB
ha1 running esp-proposal 1# esp-proposal 1
ha1 running esp-proposal 1# enc-alg aes256-cbc
ha1 running esp-proposal 1# auth-alg hmac-sha256
ha1 running esp-proposal 1# ..
ha1 running ipsec-policy-template nodeB# start-action none
ha1 running ipsec-policy-template nodeB# close-action none
ha1 running ipsec-policy-template nodeB# dpd-action clear
ha1 running ipsec-policy-template nodeB# rekey-time 2800
ha1 running ipsec-policy-template nodeB# ..

ha1 running ike# vpn nodeB
ha1 running vpn nodeB# ike-policy template nodeB
ha1 running vpn nodeB# ipsec-policy template nodeB
ha1 running vpn nodeB# local-address 55.55.55.1
ha1 running vpn nodeB# security-policy nodeB
ha1 running security-policy nodeB# local-ts subnet 10.77.77.0/24
ha1 running security-policy nodeB# remote-ts subnet 10.7.0.0/24
ha1 running security-policy nodeB# svti-id-in 10
ha1 running security-policy nodeB# svti-id-out 10
ha1 running security-policy nodeB# inject-routes
ha1 running inject-routes# next-hop svti-trusted
ha1 running inject-routes# vrf trusted
ha1 running inject-routes# ..
ha1 running security-policy nodeB# ..
ha1 running vpn nodeB# ..

ha1 running ike# pre-shared-key femto
ha1 running pre-shared-key femto# secret 0x12345678
ha1 running pre-shared-key femto# ..

ha1 running ike# /
ha1 running config#

Configure IKE HA:

ha1 running config#! / vrf untrusted ike
ha1 running ike#! ha
ha1 running ha#! listen-ha-group ha_ipsec
ha1 running ha#! local-address 192.168.2.0
ha1 running ha#! remote-address 192.168.2.1
ha1 running ha# /
ha1 running config#

Once everything is configured, commit the configuration:

ha1 running config#
ha1 running config# commit
Configuration committed.

ha2 CLI configuration

Configure device hostname:

vsr running config# system hostname ha2
vsr running config# commit
Configuration committed.

In this example, ha2 only has one physical ethernet interface (eth1) and uses VLANs in different VRFs to partition the trusted, untrusted and hasync networks.

../../../_images/ha-ike-routing-ha2.svg

Configure the physical network interface (adapt port ids to your hardware) in VRF main:

ha2 running config# vrf main
ha2 running vrf main#! interface physical eth1
ha2 running physical eth1#! port pci-b0s4
ha2 running physical eth1# mtu 9000
ha2 running physical eth1# /
ha2 running config#

Then configure the VRF and VLAN for the trusted network:

ha2 running config# / vrf trusted interface
ha2 running interface# vlan vlan_trusted
ha2 running vlan vlan_trusted#! vlan-id 2
ha2 running vlan vlan_trusted#! link-interface eth1
ha2 running vlan vlan_trusted# link-vrf main
ha2 running vlan vlan_trusted# ipv4 address 10.2.0.1/30
ha2 running vlan vlan_trusted# /
ha2 running config#

Then configure the VRF and VLAN for the untrusted network:

ha2 running config# / vrf untrusted interface
ha2 running interface# vlan vlan_untrusted
ha2 running vlan vlan_untrusted#! vlan-id 307
ha2 running vlan vlan_untrusted#! link-interface eth1
ha2 running vlan vlan_untrusted# link-vrf main
ha2 running vlan vlan_untrusted# ipv4 address 10.55.0.2/30
ha2 running vlan vlan_untrusted# /
ha2 running config#

Then configure the VRF and VLAN for the HA synchronization network:

ha2 running config# / vrf ha interface
ha2 running interface# vlan vlan_ha
ha2 running vlan vlan_ha#! vlan-id 10
ha2 running vlan vlan_ha#! link-interface eth1
ha2 running vlan vlan_ha# link-vrf main
ha2 running vlan vlan_ha# ipv4 address 192.168.1.1/31
ha2 running vlan vlan_ha# /
ha2 running config#

In order to perform cross-VRF IPsec, the trusted traffic will be sent through an SVTI cross-VRF interface in VRF trusted, with link-VRF untrusted:

ha2 running config# / vrf trusted interface
ha2 running interface# svti svti-trusted
ha2 running svti svti-trusted# svti-id 10
ha2 running svti svti-trusted# link-vrf untrusted
ha2 running svti svti-trusted# /

Now configure routing on the trusted network. The connectivity to the BGP peer is tracked via the BFD protocol. A route map TRUSTED-OUT is attached to the routes learnt via this peer, in order to adjust the cost of the learnt routes, depending on ha2 HA activity. The kernel routes of the trusted network are redistributed via BGP.

ha2 running config# / vrf trusted routing
ha2 running routing# bgp
ha2 running bgp# as 65111
ha2 running bgp# router-id 10.6.6.6
ha2 running bgp# ebgp-requires-policy false
ha2 running bgp# address-family ipv4-unicast redistribute kernel
ha2 running bgp# neighbor 2.0.0.2
ha2 running neighbor 10.2.0.2# remote-as 65123
ha2 running neighbor 10.2.0.2# address-family ipv4-unicast
ha2 running ipv4-unicast# as-override false
ha2 running ipv4-unicast# route-map out route-map-name TRUSTED-OUT
ha2 running running ipv4-unicast# ..
ha2 running running address-family# ..
ha2 running running neighbor 10.2.0.2# track bfd
ha2 running running neighbor 10.2.0.2# /
ha2 running config#

Now configure routing on the untrusted network. The connectivity to the BGP peer is tracked via the BFD protocol. A route map UNTRUSTED-OUT is attached to the routes learnt via this peer, in order to adjust the cost of the learnt routes, depending on ha2 HA activity. The IKE tunnel end point (10.55.55.1/32) is advertised by BGP.

ha2 running config# / vrf untrusted routing
ha2 running routing# bgp
ha2 running bgp# as 65111
ha2 running bgp# router-id 10.6.6.6
ha2 running bgp# ebgp-requires-policy false
ha2 running bgp# address-family ipv4-unicast network 10.55.55.1/32
ha2 running bgp# neighbor 10.55.0.1
ha2 running neighbor 10.55.0.1# remote-as 65123
ha2 running neighbor 10.55.0.1# address-family ipv4-unicast
ha2 running ipv4-unicast# as-override false
ha2 running ipv4-unicast# route-map out route-map-name UNTRUSTED-OUT
ha2 running running ipv4-unicast# ..
ha2 running running address-family# ..
ha2 running neighbor 10.55.0.1# track bfd
ha2 running neighbor 10.55.0.1# /
ha2 running config#

The hasync network is an internal network that interconnects the HA nodes and enables to exchange synchronization data. We chose to use a separate VRF and to access it via a different VLAN. However, the IKE service runs in VRF untrusted and the HA module needs to establish a TCP connection with the HA peer in VRF untrusted. We therefore create a cross-VRF GRE interface in VRF untrusted, with link-VRF ha:

ha2 running config# / vrf untrusted interface
ha2 running interface# gre gre-hasync
ha2 running gre gre-hasync# ttl 64
ha2 running gre gre-hasync# link-vrf ha
ha2 running gre gre-hasync# local 192.168.1.1
ha2 running gre gre-hasync# remote 192.168.1.0
ha2 running gre gre-hasync# key both 888
ha2 running gre gre-hasync# ipv4 address 192.168.2.1/31
ha2 running gre gre-hasync# /
ha2 running config#

Create a loopback interface:

ha2 running config# / vrf untrusted interface
ha2 running interface# loopback loopback0
ha2 running loopback loopback0# ipv4 address 10.6.6.6/32
ha2 running loopback loopback0# /
ha2 running config#

Configure the route maps used by the BGP configurations:

ha2 running config# / routing

ha2 running routing# ipv4-prefix-list IPSEC_TEP
ha2 running ipv4-prefix-list IPSEC_TEP# seq 10 address 10.55.55.1/32 policy permit
ha2 running ipv4-prefix-list IPSEC_TEP# ..

ha2 running routing# ipv4-prefix-list RAN
ha2 running ipv4-prefix-list RAN# seq 10 address 10.7.0.0/24 policy permit le 32
ha2 running ipv4-prefix-list RAN# ..

ha2 running routing# route-map UNTRUSTED-OUT
ha2 running route-map UNTRUSTED-OUT# seq 10
ha2 running route-map seq 10# policy permit
ha2 running route-map seq 10# match ip address prefix-list IPSEC_TEP
ha2 running route-map seq 10# match ha-group name ha_ipsec
ha2 running route-map seq 10# ..
ha2 running route-map UNTRUSTED-OUT# ..

ha2 running routing# route-map TRUSTED-OUT
ha2 running route-map TRUSTED-OUT# seq 10
ha2 running route-map seq 10# policy permit
ha2 running route-map seq 10# match ip address prefix-list RAN
ha2 running route-map seq 10# match ha-group name ha_ipsec
ha2 running route-map seq 10# ..
ha2 running route-map TRUSTED-OUT# /

ha2 running config#

Create two trackers that monitor the connectivity with the routers to the trusted and untrusted networks:

ha2 running config# / tracker bfd pe-bfd-trusted
ha2 running bfd pe-bfd-trusted# source 10.2.0.1
ha2 running bfd pe-bfd-trusted# address 10.2.0.2
ha2 running bfd pe-bfd-trusted# vrf trusted
ha2 running bfd pe-bfd-trusted# ..

ha2 running config# / tracker bfd pe-bfd-untrusted
ha2 running bfd pe-bfd-untrusted# source 10.55.0.2
ha2 running bfd pe-bfd-untrusted# address 10.55.0.1
ha2 running bfd pe-bfd-untrusted# vrf untrusted
ha2 running bfd pe-bfd-untrusted# /

ha2 running config#

We can finally configure HA. First configure the HA group:

ha2 running config# / ha group ha-group1
ha2 running group ha-group1#! ..
ha2 running ha#! /
ha2 running config#!

Note

The HA group maintains the node high-availability state. It is controlled by the VRRP protocol (via the notify-ha-group command) and monitored by HA IKE (via the listen-ha-group command). Only one controller can be defined for an HA group.

Configure VRRP in unicast mode. It also tracks the link state of the gre-hasync interface, as well as the connectivity with the routers to the trusted and untrusted networks, via the BFD protocol. Finally, it installs a VRRP VIP on the master node (10.55.55.1), which is the IKE and IPsec tunnel end point. It notifies the services interested by the HA status via the HA group ha_ipsec (IKE and routing):

ha2 running config# / vrf untrusted interface
ha2 running interface# vrrp vrrp_ha
ha2 running vrrp vrrp_ha# version 3
ha2 running vrrp vrrp_ha# link-interface gre-hasync
ha2 running vrrp vrrp_ha# use-vmac false
ha2 running vrrp vrrp_ha# vrid 10
ha2 running vrrp vrrp_ha# priority 100
ha2 running vrrp vrrp_ha# preempt-delay 30
ha2 running vrrp vrrp_ha# advertisement-interval 100
ha2 running vrrp vrrp_ha# track pe-bfd-trusted
ha2 running vrrp vrrp_ha# track pe-bfd-untrusted
ha2 running vrrp vrrp_ha# track-fast-path true
ha2 running vrrp vrrp_ha# unicast-peer 192.168.2.0
ha2 running vrrp vrrp_ha# virtual-address 10.55.55.1/32
ha2 running vrrp vrrp_ha# notify-ha-group ha_ipsec
ha2 running vrrp vrrp_ha# /

ha2 running config# / vrf untrusted vrrp
ha2 running vrrp# router-id 10.6.6.6
ha2 running vrrp# vrrp-startup-delay 30
ha2 running vrrp# /
ha2 running config#

Now that network connectivity and routing is configured, let us configure IKE. Here, ha2 is configured as a VPN concentrator that accepts IKE connections initiated by a remote peer. To support cross-VRF IPsec, the cross-VRF SVTI interface svti-trusted is used (VRF trusted, link-VRF untrusted, svti-id 10). The inject-routes option automatically adds routes to the remote traffic selector via the SVTI interface.

Configure IKE:

ha2 running config# / vrf untrusted ike
ha2 running ike# ike-policy-template nodeB
ha2 running ike-proposal-1# enc-alg aes256-cbc
ha2 running ike-proposal-1# auth-alg hmac-sha25
ha2 running ike-proposal-1# dh-group modp2048
ha2 running ike-proposal-1# ...
ha2 running ike-policy-template nodeB# unique-sa replace
ha2 running ike-policy-template nodeB# rekey-time 2800
ha2 running ike-policy-template nodeB# dpd-delay 10
ha2 running ike-policy-template nodeB# ..

ha2 running ike# ipsec-policy-template nodeB
ha2 running esp-proposal 1# esp-proposal 1
ha2 running esp-proposal 1# enc-alg aes256-cbc
ha2 running esp-proposal 1# auth-alg hmac-sha256
ha2 running esp-proposal 1# ..
ha2 running ipsec-policy-template nodeB# start-action none
ha2 running ipsec-policy-template nodeB# close-action none
ha2 running ipsec-policy-template nodeB# dpd-action clear
ha2 running ipsec-policy-template nodeB# rekey-time 2800
ha2 running ipsec-policy-template nodeB# ..

ha2 running ike# vpn nodeB
ha2 running vpn nodeB# ike-policy template nodeB
ha2 running vpn nodeB# ipsec-policy template nodeB
ha2 running vpn nodeB# local-address 55.55.55.1
ha2 running vpn nodeB# security-policy nodeB
ha2 running security-policy nodeB# local-ts subnet 10.77.77.0/24
ha2 running security-policy nodeB# remote-ts subnet 10.7.0.0/24
ha2 running security-policy nodeB# svti-id-in 10
ha2 running security-policy nodeB# svti-id-out 10
ha2 running security-policy nodeB# inject-routes
ha2 running inject-routes# next-hop svti-trusted
ha2 running inject-routes# vrf trusted
ha2 running inject-routes# ..
ha2 running security-policy nodeB# ..
ha2 running vpn nodeB# ..

ha2 running ike# pre-shared-key femto
ha2 running pre-shared-key femto# secret 0x12345678
ha2 running pre-shared-key femto# ..

ha2 running ike# /
ha2 running config#

Configure IKE HA:

ha2 running config#! / vrf untrusted ike
ha2 running ike#! ha
ha2 running ha#! listen-ha-group ha_ipsec
ha2 running ha#! local-address 192.168.2.1
ha2 running ha#! remote-address 192.168.2.0
ha2 running ha# /
ha2 running config#

Once everything is configured, commit the configuration:

ha2 running config#
ha2 running config# commit
Configuration committed.

Encrypting the synchronization messages

The HA synchronization data exchanged between ha1 and ha2 contains sensitive information such as cryptographic keys. Now the network path used to convey this information (eth3 in the HA cluster figure) is not necessarily a trusted network.

It is possible to use IPsec to encrypt this traffic. In order to isolate the IKE and IPsec traffic used for securing the HA synchronization exchanges from the other IKE and IPsec traffic, the underlying interface (here eth3) is moved to a separate VRF hasync, and an IPsec-protected cross-VRF logical interface is created over this link. The HA synchronization is performed via the logical interface instead of the eth3 interface.

The logical interface may be for example an SVTI interface (that does a direct IPsec encapsulation of the HA exchanges) or a GRE interface (that first does a GRE encapsulation of the HA exchanges, then the GRE packets are IPsec encrypted).

../../../_images/ha-ike-encr-sync.svg

Using a cross-vrf svti interface

A cross-vrf SVTI logical interface is used to convey the HA synchronization messages.

The SVTI interface is in the same VRF as the IKE service (here main) and its link-VRF is hasync. Then IKE is configured in VRF hasync to encrypt the data sent over the SVTI interface in tunnel mode.

The endpoints of the HA synchronization traffic (ha1 10.160.0.1, ha2 10.160.0.2) are configured on the SVTI interface. All data sent over the SVTI interface is encrypted in tunnel mode between the adresses of the underlying physical interface (ha1 10.150.0.1, ha2 10.150.0.2).

ha1 cli configuration

ha1 running config# show config nodefault
vrf main
    (...)
    interface
        svti svti-hasync
            ipv4
                address 10.160.0.1/24
                ..
            svti-id 888
            link-vrf hasync
            ..
        ..
    (...)
    ike
        ha
            listen-ha-group g
            local-address 10.160.0.1
            remote-address 10.160.0.2
            seqnum-sync
                sync-period-time 1s
                sync-period-packets 64
                ..
            ..
        ..
    (...)
vrf hasync
    ike
        pre-shared-key psk-hasync
            id primary@ha
            id secondary@ha
            secret 0xf0043c9a1e4c22177c4e01dd909a96b2
            ..
        ike-policy-template ikepol-hasync
            ike-proposal 1
                enc-alg aes128-cbc
                auth-alg hmac-sha1
                prf-alg hmac-sha1
                dh-group modp1024
                ..
            keying-tries 0
            unique-sa replace
            ..
        ipsec-policy-template ipsecpol-hasync
            esp-proposal 1
                enc-alg aes128-cbc
                auth-alg hmac-sha1
                dh-group modp1024
                esn true
                ..
            ..
        vpn vpn-hasync
            ike-policy
                template ikepol-hasync
                ..
            ipsec-policy
                template ipsecpol-hasync
                ..
            local-address 10.150.0.1
            remote-address 10.150.0.2
            local-id primary@ha
            remote-id secondary@ha
            security-policy svti-hasync
                local-ts subnet 0.0.0.0/0
                remote-ts subnet 0.0.0.0/0
                svti-id-in 888
                svti-id-out 888
                ..
            ..
        ..
    interface
        physical ntfp3
            ipv4
                address 10.150.0.1/24
                ..
            port pci-b0s6
            ..
        ..
    ..

ha2 cli configuration

ha2 running config# show config nodefault
vrf main
    (...)
    interface
        svti svti-hasync
            ipv4
                address 10.160.0.2/24
                ..
            svti-id 888
            link-vrf hasync
            ..
        ..
    (...)
    ike
        ha
            listen-ha-group g
            local-address 10.160.0.2
            remote-address 10.160.0.1
            seqnum-sync
                sync-period-time 1s
                sync-period-packets 64
                ..
            ..
        ..
    (...)
vrf hasync
    ike
        pre-shared-key psk-hasync
            id primary@ha
            id secondary@ha
            secret 0xf0043c9a1e4c22177c4e01dd909a96b2
            ..
        ike-policy-template ikepol-hasync
            ike-proposal 1
                enc-alg aes128-cbc
                auth-alg hmac-sha1
                prf-alg hmac-sha1
                dh-group modp1024
                ..
            keying-tries 0
            unique-sa replace
            ..
        ipsec-policy-template ipsecpol-hasync
            esp-proposal 1
                enc-alg aes128-cbc
                auth-alg hmac-sha1
                dh-group modp1024
                esn true
                ..
            ..
        vpn vpn-hasync
            ike-policy
                template ikepol-hasync
                ..
            ipsec-policy
                template ipsecpol-hasync
                ..
            local-address 10.150.0.2
            remote-address 10.150.0.1
            local-id secondary@ha
            remote-id primary@ha
            security-policy svti-hasync
                local-ts subnet 0.0.0.0/0
                remote-ts subnet 0.0.0.0/0
                svti-id-in 888
                svti-id-out 888
                ..
            ..
        ..
    interface
        physical ntfp3
            ipv4
                address 10.150.0.2/24
                ..
            port pci-b0s6
            ..
        ..
    ..

Using a cross-vrf gre interface

A cross-vrf GRE logical interface is used to convey the HA synchronization messages.

The GRE interface is in the same VRF as the IKE service (here main) and its link-VRF is hasync. Then IKE is configured in VRF hasync to encrypt the GRE traffic in transport mode.

The endpoints of the HA synchronization traffic (ha1 10.160.0.1, ha2 10.160.0.2) are configured on the GRE interface. Its encapsulation addresses are those of the underlying physical interface (ha1 10.150.0.1, ha2 10.150.0.2). The GRE-encapsulated packets are encrypted in transport mode.

ha1 cli configuration

ha1 running config# show config nodefault
vrf main
    (...)
    interface
        gre gre-hasync
            local 10.150.0.1
            remote 10.150.0.2
            ipv4
                address 10.160.0.1/24
                ..
            link-vrf hasync
            ..
        ..
    (...)
    ike
        ha
            listen-ha-group g
            local-address 10.160.0.1
            remote-address 10.160.0.2
            seqnum-sync
                sync-period-time 1s
                sync-period-packets 64
                ..
            ..
        ..
    (...)
vrf hasync
    ike
        pre-shared-key psk-hasync
            id primary@ha
            id secondary@ha
            secret 0xf0043c9a1e4c22177c4e01dd909a96b2
            ..
        ike-policy-template ikepol-hasync
            ike-proposal 1
                enc-alg aes128-cbc
                auth-alg hmac-sha1
                prf-alg hmac-sha1
                dh-group modp1024
                ..
            keying-tries 0
            unique-sa replace
            ..
        ipsec-policy-template ipsecpol-hasync
            esp-proposal 1
                enc-alg aes128-cbc
                auth-alg hmac-sha1
                dh-group modp1024
                esn true
                ..
            ..
        vpn vpn-hasync
            ike-policy
                template ikepol-hasync
                ..
            ipsec-policy
                template ipsecpol-hasync
                ..
            local-address 10.150.0.1
            remote-address 10.150.0.2
            local-id primary@ha
            remote-id secondary@ha
            security-policy gre-hasync
                local-ts protocol 47
                remote-ts protocol 47
                mode transport
                ..
            ..
        ..
    interface
        physical ntfp3
            ipv4
                address 10.150.0.1/24
                ..
            port pci-b0s6
            ..
        ..
    ..

ha2 cli configuration

ha2 running config# show config nodefault
vrf main
    (...)
    interface
        gre gre-hasync
            local 10.150.0.2
            remote 10.150.0.1
            ipv4
                address 10.160.0.2/24
                ..
            link-vrf hasync
            ..
        ..
    (...)
    ike
        ha
            listen-ha-group g
            local-address 10.160.0.2
            remote-address 10.160.0.1
            seqnum-sync
                sync-period-time 1s
                sync-period-packets 64
                ..
            ..
        ..
    (...)
vrf hasync
    ike
        pre-shared-key psk-hasync
            id primary@ha
            id secondary@ha
            secret 0xf0043c9a1e4c22177c4e01dd909a96b2
            ..
        ike-policy-template ikepol-hasync
            ike-proposal 1
                enc-alg aes128-cbc
                auth-alg hmac-sha1
                prf-alg hmac-sha1
                dh-group modp1024
                ..
            keying-tries 0
            unique-sa replace
            ..
        ipsec-policy-template ipsecpol-hasync
            esp-proposal 1
                enc-alg aes128-cbc
                auth-alg hmac-sha1
                dh-group modp1024
                esn true
                ..
            ..
        vpn vpn-hasync
            ike-policy
                template ikepol-hasync
                ..
            ipsec-policy
                template ipsecpol-hasync
                ..
            local-address 10.150.0.2
            remote-address 10.150.0.1
            local-id secondary@ha
            remote-id primary@ha
            security-policy gre-hasync
                local-ts protocol 47
                remote-ts protocol 47
                mode transport
                ..
            ..
        ..
    interface
        physical ntfp3
            ipv4
                address 10.150.0.2/24
                ..
            port pci-b0s6
            ..
        ..
    ..

Advanced options

Sequence number synchronization parameters

IPsec SAs sequence numbers and statistics are regularly synchronized from the active node to the backup node. In case of switchover, this enables the new master node to take over the IPsec dataplane processing with proper sequence numbers:

For an output SA, the output sequence number 1 on the backup node should be greater or equal to the last sequence number used by this SA on the master node. Otherwise, the remote IPsec peer is likely to drop some IPsec packets sent by the new master until the sequence numbers comply to its replay window state.

For an input SA, the input sequence number 2 on the backup node should be close to the highest sequence number received on the master node. Otherwise the new master node is vulnerable to accepting replayed packets sent by an attacker, because its replay window is too late.

1

i.e. the record of the highest SA sequence number of a sent packet protected with this SA

2

i.e. the record of the highest SA sequence number of a received packet protected with this SA

The pace at which sequence number synchronization is performed is configurable in the ha seqnum-sync sub-context:

ha1 running vrf main# ike ha seqnum-sync
ha1 running seqnum-sync# sync-period-time 10s
ha1 running seqnum-sync# sync-period-packets 2
ha1 running seqnum-sync# oseq-shift 65536
ha1 running seqnum-sync# / vrf main
  • sync-period-time is the minimum time between two sequence number updates. An update is sent to the backup node only if the sequence number changed since last update (default 10s, 0 disables the time-based periodic update).

  • sync-period-packets is the number of packets between two sequence number updates: if the input or output sequence number of an IPsec SA changes of at least that number since last synchronization, then an update is sent to the backup node (default 2, 0 disables the packet-based periodic update).

  • oseq-shift is a value added to IPsec SA output sequence numbers when they are synchronized to the backup node: since sequence numbers cannot be synchronized in real-time, the output sequence numbers communicated to the backup node are always late compared to the master node. Adding this value to the current output sequence number aims at ensuring that the output sequence number on the backup node always remains greater than on the master, even during two updates. It should be greater than the number of packets processed between two sequence number updates (default 65536).

DSCP of HA synchronization messages

HA synchronization messages are exchanged between the master and backup nodes via the TCP protocol on port 4510. This traffic is rather critical and needs to be safeguarded by QoS and control plane protection mechanisms.

An option enables to set the DSCP of these messages:

ha1 running config# / vrf main ike ha
ha1 running ha# dscp 56
ha1 running ha# commit

This example sets the DSCP of HA synchronization messages to 56.

HA-compatible virtual IP pools

IKE enables to assign a * VIP * and optional attributes during an IKE negotiation, i.e. an IKE initiator may request an additional IP address from the responder to use as inner IPsec tunnel address.

VIPs and attributes are exchanged using an IKEv2 Configuration Payload (or an IKEv1 Config Mode) exchange.

Additional parameters may be assigned during this exchange, such as a DNS server address, a NetBIOS server address or a DHCP server address.

To proceed, the responder maintains a pool of VIPs. (see IKE virtual IP pools).

If the IKE configuration makes use of VIP pools and HA IKE is enabled, then VIP leases must be synchronized between the master and the backup node.

This requires using specific HA-synchronized VIP pools. These pools are less flexible than standard VIP pools:

  • address pools can only be defined as subnets, not ranges of addresses.

  • there is no state information about these pools

When enabling HA IKE, be careful of using a virtual pool defined in the ha context, because virtual pools defined directly in the ike context are not synchronized between the master and backup node.

Define the pool. The address specification is mandatory, other attributes are optional:

ha1 running vrf main# ike
ha1 running ike# ha
ha1 running ha# pool my-ha-pool
ha1 running pool my-ha-pool#! address 192.168.0.0/24
ha1 running pool my-ha-pool# dns 192.168.3.99
ha1 running pool my-ha-pool# ..
ha1 running ha#
  • address is the set of addresses that can be assigned (IPv4 or IPv6 subnet).

  • dns is an optional list of DNS server addresses (IPv4 or IPv6).

  • nbns is an optional list of NetBIOS server addresses (IPv4 or IPv6).

  • dhcp is an optional list of DHCP server addresses (IPv4 or IPv6).

  • subnet is an optional list of subnets protected by the responder (IPv4 or IPv6). (attributes INTERNAL_IP4_SUBNET/INTERNAL_IP6_SUBNET in RFC 7296)

  • netmask is an optional value for the internal network’s netmask (IPv4). (attribute INTERNAL_IP4_NETMASK in RFC 7296)

Use it in a vpn:

ha1 running ike# vpn vpn-secgw
ha1 running vpn vpn-secgw# vip
ha1 running vpn vpn-secgw# vip-pool my-ha-pool
ha1 running vpn vpn-secgw# ..
ha1 running ike#

Display the IKE configuration:

ha1 running ike# show config nodefault
ike
    vpn vpn-secgw
        vip-pool my-ha-pool
        (...)
    ha
        pool my-ha-pool
            address 192.168.0.0/24
            dns 192.168.3.99
            ..
        (...)
    (...)

Legacy SA sequence numbers and statistics synchronization

Older versions of the Virtual Service Router (Virtual Service Router before 3.8) used an external daemon to synchronize SA sequence numbers and statistics. These states are now synchronized by the IKE daemon HA module, like all other states.

It is still possible to run the legacy external daemon by setting a node-id and interface.

  • node-id is a unique identifier for the node in the HA cluster. It ranges from 1 to 15.

  • interface is the network interface on which discovery messages are exchanged.

Example:

ha1 running ha# show config
ha
    enabled true
    listen-ha-group my-ha-group
    node-id 1
    interface eth3
    local-address 10.150.0.1
    remote-address 10.150.0.2
    seqnum-sync
        oseq-shift 65536
        sync-period-time 10s
        sync-period-packets 2
        ..
    ..

The synchronization daemons on the master and backup nodes discover themselves via multicast messages on the specified interface (UDP multicast group 225.0.0.1 or ff02::fa00:1, port 9007), and identify each other with their unique node-id.

The synchronization messages themselves are exchanged via the TCP protocol on port 9006.

Hot migration from a Virtual Service Router before 3.8

To migrate from a Virtual Service Router before 3.8 to a Virtual Service Router 3.8 on a live system, follow the following procedure:

An old Virtual Service Router has mandatory node-id and interface parameters and runs the external synchronization daemon.

  • First upgrade the backup node.

  • Once the backup node is upgraded, switch the HA activity so that it becomes active and the former master becomes backup.

  • Upgrade the now backup node.

  • Now that both nodes are upgraded, possibly switch the HA activity back if required by your internal policy.

  • Remove the node-id and interface parameters from both nodes configurations.

Now the HA nodes use the same synchronization channel as other HA states (TCP port 4510).

Troubleshooting

“(unknown)” SAs after failover

If a failover occurs while an SA is being created or rekeyed it may result in an extra SA named “(unknown)” being permanently listed:

ha1# show ike vrf main ike-sa
VPN       Local-Addr Local-ID        Remote-Addr Remote-ID         State      Child-SAs
(unknown) 10.175.0.1 secgw.6wind.net 10.225.0.1  enodeb1.6wind.net connecting 0

Such SAs do not affect the system in any way other than cluttering the list and can be cleared manually using the flush command (see IKE Manually clearing SAs).

See also

The IKE command reference for details.