5. Validation

5.1. VRRP failover and HA swap

A first test will consist in forcing secgw1 - the VRRP Master - to become faulty by disabling one of its interfaces. Its VRRP state should move to fault and secgw2 should become master. Also, the IKE state should change accordingly and IKE sessions must transit to ESTABLISHED on secgw2 and PASSIVE on secgw1.

Disable a VRRP interface on secgw1:

secgw1> edit running
secgw1 running config# vrf main interface physical ntfp1 enabled false
secgw1 running config# commit
Configuration committed.

The VRRP state is changed to fault:

secgw1 running config# show state vrf main vrrp
vrrp
    enabled true
    router-id secgw1
    snmp-enabled false
    traps-enabled false
    vrrp-startup-delay 0
    drop-traffic-on-backup true
    group vrrp_group
        instance vrrp_lan
        instance vrrp_public
        notify-ha-group ha_for_ike
        state fault
        ..
    ..
secgw1 running config#

The VRRP state is changed to master on secgw2:

secgw2> show state vrf main vrrp
vrrp
    enabled true
    router-id secgw2
    snmp-enabled false
    traps-enabled false
    vrrp-startup-delay 0
    drop-traffic-on-backup true
    group vrrp_group
        instance vrrp_lan
        instance vrrp_public
        notify-ha-group ha_for_ike
        state master
        ..
    ..
secgw2>

The IKE state is changed to PASSIVE on secgw1:

secgw1 running config# show ike ike-sa details
vpn_hq: #2, PASSIVE, IKEv2, 7a0e17fba5af1ed4_i b7d2d02835fd0952_r
  local  'secgw.6wind.com' @ 66.66.66.66[500]
  remote 'user2@dev.6wind.com' @ 2.2.2.2[500] [172.31.0.2]
  aes128-cbc/hmac-sha512/hmac-sha512/modp2048
  access_to_lan: #2, reqid 2, INSTALLED, TUNNEL, esp:aes128-cbc/hmac-sha256/modp2048
    installed 781s ago, rekeying in 2521s, expires in 3179s
    in  c7b832c7, 0 bytes, 0 packets
    out c104ceb4, 0 bytes, 0 packets
    local  172.30.0.0/24
    remote 172.31.0.2/32
vpn_hq: #1, PASSIVE, IKEv2, 080d80b4b06a2b2c_i ae34351d8c1c30d0_r
  local  'secgw.6wind.com' @ 66.66.66.66[500]
  remote 'user1@dev.6wind.com' @ 1.1.1.1[500] [172.31.0.1]
  aes128-cbc/hmac-sha512/hmac-sha512/modp2048
  access_to_lan: #1, reqid 1, INSTALLED, TUNNEL, esp:aes128-cbc/hmac-sha256/modp2048
    installed 804s ago, rekeying in 2578s, expires in 3156s
    in  ca18ffbd, 336 bytes, 4 packets
    out c095e9c5, 336 bytes, 4 packets
    local  172.30.0.0/24
    remote 172.31.0.1/32
secgw1 running config#

The IKE state is changed to ESTABLISHED on secgw2:

secgw2> show ike ike-sa details
vpn_hq: #2, ESTABLISHED, IKEv2, 7a0e17fba5af1ed4_i b7d2d02835fd0952_r
  local  'secgw.6wind.com' @ 66.66.66.66[500]
  remote 'user2@dev.6wind.com' @ 2.2.2.2[500] [172.31.0.2]
  aes128-cbc/hmac-sha512/hmac-sha512/modp2048
  established 49s ago, rekeying in 13976s
  access_to_lan: #2, reqid 2, INSTALLED, TUNNEL, esp:aes128-cbc/hmac-sha256/modp2048
    installed 829s ago, rekeying in 2751s, expires in 3131s
    in  c7b832c7, 0 bytes, 0 packets
    out c104ceb4, 0 bytes, 0 packets
    local  172.30.0.0/24
    remote 172.31.0.2/32
vpn_hq: #1, ESTABLISHED, IKEv2, 080d80b4b06a2b2c_i ae34351d8c1c30d0_r
  local  'secgw.6wind.com' @ 66.66.66.66[500]
  remote 'user1@dev.6wind.com' @ 1.1.1.1[500] [172.31.0.1]
  aes128-cbc/hmac-sha512/hmac-sha512/modp2048
  established 49s ago, rekeying in 13662s
  access_to_lan: #1, reqid 1, INSTALLED, TUNNEL, esp:aes128-cbc/hmac-sha256/modp2048
    installed 842s ago, rekeying in 2556s, expires in 3118s
    in  ca18ffbd, 336 bytes, 4 packets
    out c095e9c5, 336 bytes, 4 packets
    local  172.30.0.0/24
    remote 172.31.0.1/32

secgw2>

5.2. VRRP and HA swap back to initial state

A second test will consist in launching a ping from road warrior 1 (it should be successful as it goes through secgw2), then bringing back the disabled interface on secgw1. secgw1 should hold for 60 seconds, then preempt its Master state; the IKE state should transit accordingly, and the ping should go through secgw1 –there could be a one or two-second interruption though.

Start ping from road warrior 1:

warrior1> show interface details name int_vlan1
7: int_vlan1@ntfp1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether de:ed:01:53:da:36 brd ff:ff:ff:ff:ff:ff promiscuity 0 minmtu 0 maxmtu 65535
    vlan protocol 802.1Q id 1 <REORDER_HDR> numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535
    alias ISP
    inet 1.1.1.1/24 scope global int_vlan1
       valid_lft forever preferred_lft forever
    inet 172.31.0.1/32 scope global int_vlan1
       valid_lft forever preferred_lft forever
    inet6 fe80::dced:1ff:fe53:da36/64 scope link
       valid_lft forever preferred_lft forever
warrior1> cmd ping 172.30.0.10 source 172.31.0.1
PING 172.30.0.10 (172.30.0.10) from 172.31.0.1 : 56(84) bytes of data.
64 bytes from 172.30.0.10: icmp_seq=1 ttl=63 time=1.28 ms
64 bytes from 172.30.0.10: icmp_seq=2 ttl=63 time=0.770 ms
64 bytes from 172.30.0.10: icmp_seq=3 ttl=63 time=0.641 ms
(...)

Check VRRP and IKE states on secgw1 (respectively backup and PASSIVE):

secgw1 running config# show ike ike-sa details
vpn_hq: #2, PASSIVE, IKEv2, 7a0e17fba5af1ed4_i b7d2d02835fd0952_r
  local  'secgw.6wind.com' @ 66.66.66.66[500]
  remote 'user2@dev.6wind.com' @ 2.2.2.2[500] [172.31.0.2]
  aes128-cbc/hmac-sha512/hmac-sha512/modp2048
  access_to_lan: #2, reqid 2, INSTALLED, TUNNEL, esp:aes128-cbc/hmac-sha256/modp2048
    installed 781s ago, rekeying in 2521s, expires in 3179s
    in  c7b832c7, 0 bytes, 0 packets
    out c104ceb4, 0 bytes, 0 packets
    local  172.30.0.0/24
    remote 172.31.0.2/32
vpn_hq: #1, PASSIVE, IKEv2, 080d80b4b06a2b2c_i ae34351d8c1c30d0_r
  local  'secgw.6wind.com' @ 66.66.66.66[500]
  remote 'user1@dev.6wind.com' @ 1.1.1.1[500] [172.31.0.1]
  aes128-cbc/hmac-sha512/hmac-sha512/modp2048
  access_to_lan: #1, reqid 1, INSTALLED, TUNNEL, esp:aes128-cbc/hmac-sha256/modp2048
    installed 804s ago, rekeying in 2578s, expires in 3156s
    in  ca18ffbd, 672 bytes, 8 packets
    out c095e9c5, 672 bytes, 8 packets
    local  172.30.0.0/24
    remote 172.31.0.1/32

secgw1 running config# show state vrf main vrrp
vrrp
    enabled true
    router-id secgw1
    snmp-enabled false
    traps-enabled false
    vrrp-startup-delay 0
    drop-traffic-on-backup true
    group vrrp_group
        instance vrrp_lan
        instance vrrp_public
        notify-ha-group ha_for_ike
        state fault
        ..
    ..
secgw1 running config#

The IPsec traffic goes through secgw2:

secgw2> cmd show-traffic ntfp1 filter esp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ntfp1, link-type EN10MB (Ethernet), capture size 262144 bytes
08:57:16.354446 de:ed:02:69:30:81 > 00:00:5e:00:01:02, ethertype IPv4 (0x0800), length 170: 1.1.1.1 > 66.66.66.66: ESP(spi=0xc146d5d6,seq=0x5), length 136
08:57:16.354710 de:ed:01:2e:23:19 > de:ed:02:69:30:81, ethertype IPv4 (0x0800), length 170: 66.66.66.66 > 1.1.1.1: ESP(spi=0xce1daf17,seq=0x5), length 136
08:57:17.378435 de:ed:02:69:30:81 > 00:00:5e:00:01:02, ethertype IPv4 (0x0800), length 170: 1.1.1.1 > 66.66.66.66: ESP(spi=0xc146d5d6,seq=0x6), length 136
08:57:17.378724 de:ed:01:2e:23:19 > de:ed:02:69:30:81, ethertype IPv4 (0x0800), length 170: 66.66.66.66 > 1.1.1.1: ESP(spi=0xce1daf17,seq=0x6), length 136
(...)

Enable the interface previously shut down on secgw1 and check that after a while traffic starts flowing through secgw1:

secgw1 running config# vrf main interface physical ntfp1 enabled true
secgw1 running config# commit
Configuration committed.
secgw1 running config# cmd show-traffic ntfp1 filter esp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ntfp1, link-type EN10MB (Ethernet), capture size 262144 bytes
08:59:52.002775 de:ed:02:69:30:81 > 00:00:5e:00:01:02, ethertype IPv4 (0x0800), length 170: 1.1.1.1 > 66.66.66.66: ESP(spi=0xc146d5d6,seq=0x63), length 136
08:59:52.002964 de:ed:01:6b:02:ab > de:ed:02:69:30:81, ethertype IPv4 (0x0800), length 170: 66.66.66.66 > 1.1.1.1: ESP(spi=0xce1daf17,seq=0x63), length 136
08:59:53.026740 de:ed:02:69:30:81 > 00:00:5e:00:01:02, ethertype IPv4 (0x0800), length 170: 1.1.1.1 > 66.66.66.66: ESP(spi=0xc146d5d6,seq=0x64), length 136
08:59:53.026982 de:ed:01:6b:02:ab > de:ed:02:69:30:81, ethertype IPv4 (0x0800), length 170: 66.66.66.66 > 1.1.1.1: ESP(spi=0xce1daf17,seq=0x64), length 136
08:59:54.050736 de:ed:02:69:30:81 > 00:00:5e:00:01:02, ethertype IPv4 (0x0800), length 170: 1.1.1.1 > 66.66.66.66: ESP(spi=0xc146d5d6,seq=0x65), length 136
08:59:54.050957 de:ed:01:6b:02:ab > de:ed:02:69:30:81, ethertype IPv4 (0x0800), length 170: 66.66.66.66 > 1.1.1.1: ESP(spi=0xce1daf17,seq=0x65), length 136
(...)
^C
100 packets captured
100 packets received by filter
0 packets dropped by kernel
secgw1 running config#

The VRRP state becomes master after some time, and the IKE state becomes ESTABLISHED:

secgw1 running config# show state vrf main vrrp
vrrp
    enabled true
    router-id secgw1
    snmp-enabled false
    traps-enabled false
    vrrp-startup-delay 0
    drop-traffic-on-backup true
    group vrrp_group
        instance vrrp_lan
        instance vrrp_public
        notify-ha-group ha_for_ike
        state backup
        ..
    ..
secgw1 running config# show state vrf main vrrp
vrrp
    enabled true
    router-id secgw1
    snmp-enabled false
    traps-enabled false
    vrrp-startup-delay 0
    drop-traffic-on-backup true
    group vrrp_group
        instance vrrp_lan
        instance vrrp_public
        notify-ha-group ha_for_ike
        state master
        ..
    ..
secgw1 running config# show ike ike-sa details
vpn_hq: #2, ESTABLISHED, IKEv2, 7a0e17fba5af1ed4_i b7d2d02835fd0952_r
  local  'secgw.6wind.com' @ 66.66.66.66[500]
  remote 'user2@dev.6wind.com' @ 2.2.2.2[500] [172.31.0.2]
  aes128-cbc/hmac-sha512/hmac-sha512/modp2048
  established 1s ago, rekeying in 12983s
  access_to_lan: #2, reqid 2, INSTALLED, TUNNEL, esp:aes128-cbc/hmac-sha256/modp2048
    installed 1055s ago, rekeying in 2247s, expires in 2905s
    in  c7b832c7, 0 bytes, 0 packets
    out c104ceb4, 0 bytes, 0 packets
    local  172.30.0.0/24
    remote 172.31.0.2/32
vpn_hq: #1, ESTABLISHED, IKEv2, 080d80b4b06a2b2c_i ae34351d8c1c30d0_r
  local  'secgw.6wind.com' @ 66.66.66.66[500]
  remote 'user1@dev.6wind.com' @ 1.1.1.1[500] [172.31.0.1]
  aes128-cbc/hmac-sha512/hmac-sha512/modp2048
  established 1s ago, rekeying in 12823s
  access_to_lan: #1, reqid 1, INSTALLED, TUNNEL, esp:aes128-cbc/hmac-sha256/modp2048
    installed 1078s ago, rekeying in 2304s, expires in 2882s
    in  ca18ffbd, 13356 bytes, 159 packets
    out c095e9c5, 13356 bytes, 159 packets
    local  172.30.0.0/24
    remote 172.31.0.1/32

secgw1 running config#

The ping was not discontinued on road warrior 1 during the swap:

(...)
64 bytes from 172.30.0.10: icmp_seq=153 ttl=63 time=0.996 ms
64 bytes from 172.30.0.10: icmp_seq=154 ttl=63 time=0.906 ms
64 bytes from 172.30.0.10: icmp_seq=155 ttl=63 time=0.880 ms
64 bytes from 172.30.0.10: icmp_seq=156 ttl=63 time=0.945 ms
64 bytes from 172.30.0.10: icmp_seq=157 ttl=63 time=0.889 ms
64 bytes from 172.30.0.10: icmp_seq=158 ttl=63 time=0.851 ms
^C64 bytes from 172.30.0.10: icmp_seq=159 ttl=63 time=1.10 ms

--- 172.30.0.10 ping statistics ---
159 packets transmitted, 159 received, 0% packet loss, time 158662ms
rtt min/avg/max/mdev = 0.701/0.939/1.609/0.146 ms
warrior1>