Usage

Configuration file

The first step is to configure the general networking settings of the system, for example assigning the addresses and bringing up the interfaces.

The second step is to create a Fast Path CG-NAT configuration file.

NAT444 case

The following is an example configuration for a NAT444 usecase.

iface = { eth0 }

cgpool mypool 32.96.118.32-32.96.118.70 port 1024-65535 block-size 256

cgmap myrule eth0 172.16.0.0/12 -> mypool max-blocks-per-cpe 4

group default {
  pass all
}

The NAT444 usecase is used by the ISPs to cope with the public IPv4 address exhaustion. The ISPs need to find a solution to continue giving to their customers public IP addresses, when there are no new addresses available.

The below figure shows a conceptual example of NAT444. The ISP assigns a pool of public IPv4 addresses to the CG-NAT router on the “public IPv4 network”.

On the “private customer network” side, the CG-NAT router is connected to each CPE on the network 172.16.0.0/12. Each CPE uses another IP private address to reach all the computer in its network.

For example, a computer in one CPE network sent a packet to some internet website with a source address of 10.0.0.1. Then, the CPE NAT router translates the source address to 172.16.0.1. At CG-NAT router, the source address is translated to the public IPv4 address: 32.96.118.32.

If the public website sends back a packet to 32.96.118.32, the CG-NAT router translates the destination address back into 172.16.0.1 and forwards the packet to the CPE NAT. This one translates the destination address to 10.0.0.1.

NAT64 case

The following is an example configuration for a NAT64 usecase.

iface = { eth0 }

cgpool mypool 32.96.118.32-32.96.118.70 port 1024-65535 block-size 256

cgmap myrule eth0 algo nat64 2001:db8::/96 -> mypool nat64-prefix 64:ff9b::/96 max-conntracks-per-cpe 2048 max-blocks-per-cpe 4

group default {
  pass all
}

The NAT64 usecase is used by the ISPs to move to an IPv6 network while simultaneously handling IPv4 address depletion. A public IPv4 address pool is shared among several IPv6 only clients. NAT64 translates incoming IPv6 packets into IPv4 and vice versa.

Stateful NAT64 is used in conjunction with DNS64. The IPv6-only clients are configured to use a DNS64 server. When this name server receives a query from a client for AAAA (IPv6 address) records for some domain name, it tries to find an answer. If it doesn’t find any records, it looks up A records for the same domain name.

If one A record is found, it does not return it to the client directly. Instead, the IPv4 address of the A record is turned into an IPv6 address using the scheme described in RFC 6052, before sending the updated AAAA record.

The client is now able to access the server using this IPv6 address. The packet is routed through the NAT64 router, which translates the IPv6 packet into an IPv4 packet by extracting the embedded IPv4 address.

The below figure shows a conceptual example of NAT64. The ISP assigns a pool of public IPv4 addresses to the CG-NAT router on the “public IP network”.

On the “private customer network” side, the CG-NAT router is connected to each CPE on the network 2001:DB8::0/96.

To setup a DNS64 server, you can use bind9 on a linux server. Since 9.8.0 version, bind supports DNS64 with the dns64 option.

You need to add the following lines into your DNS64 bind9 configuration file:

acl rfc1918 { 10/8; 192.168/16; 172.16/12; };

options {
        ...

        // configure a caching DNS, request are redirect on google DNS server
        forwarders {
                8.8.8.8;
                8.8.4.4;
        };

        // configure dns64 with prefix 64:ff9b::/96
        dns64 64:ff9b::/96 {
                clients {  2001:db8::/96; };
                mapped { !rfc1918; any; };
                exclude { 64:ff9b::/96; };
        };
}

The IPv6 prefix in the bind9 configuration file should be the same as the one in the CG-NAT rule. NAT64 prefixes supported are /32s, /40s, /48s, /56s, /64s, or /96s.

The clients option can be used to select which client the DNS64 function will apply to. The CPE subnet in the CG-NAT rule should have the same value.

The mapped option can be used to avoid mapping specific IPv4 address to IPv6. For example, it is a good idea not to embed any RFC 1918 addresses that name servers on the Internet might inadvertently return.

DNS64 only applies to domain names that don’t have AAAA records. It is however possible to configure DNS64 to ignore AAAA records that contains IPv6 addresses.

The exclude option in the bind9 configuration is a safeguard to ensure that no IPv6 address matching 64:ff9b::/96 will be returned to IPv6 clients. Without it the CG-NAT might translate IPv6 packets going to this IPv6 server into IPv4 packet with an incorrect IPv4 address.

CG-NAT parameter

CG-NAT rules are expressed with the following syntax:

cgmap        = "cgmap" rulename interface [algo nat64] cpe_subnet "->" poolname
         [ "nat64-prefix" nat64-destination-prefix ]
         [ "port-algo" ( "parity" | "random") ]
         [ "address-pooling" ( "paired" | "no-paired") ]
         [ "endpoint-mapping" ( "dependent" | "independent") ]
         [ "endpoint-filtering" ( "dependent" | "independent") ]
         [ "hairpinning" ("on" | "off") ]
         [ "active-block-timeout timeout_in_second_before_creating_new_block]
         [ "cpe-timeout" timeout_in_second_before_to_release_a_cpe ]
         [ "max-conntracks-per-cpe" maximal_number_of_conntracks_per_cpe ]
         [ "max-blocks-per-cpe" maximal_number_of_blocks_per_cpe ]

CG-NAT deterministic rules are expressed with the following syntax:

cgmap-deterministic  = "cgmap" rulename interface [algo nat64] cpe_subnet "->" poolname
         [ "nat64-prefix" nat64-destination-prefix ]
         [ "port-algo" ( "parity" | "random") ]
         [ "endpoint-mapping" ( "dependent" | "independent") ]
         [ "endpoint-filtering" ( "dependent" | "independent") ]
         [ "hairpinning" ("on" | "off") ]
         [ "max-conntracks-per-cpe" maximal_number_of_conntracks_per_cpe ]

rule name

For each cgnat rule, a name must be set. It is used as an identifier of the rule.

CPE Subnet

For each IP address matching the CPE subnet, the cgmap rule is applied. It means that each time a packet goes out the cgmap interface matching the CPE subnet, a new cpe object is created to nat all the flows coming from this IP address.

Public IPs

Each cgnat rule needs to be associated with a pool of ip addresses.

Pool addresses are expressed with the following syntax:

cgpool{-deterministic} poolname public_ip_addr [port first_port-last_port] "block-size" number_max_of_session_per_block

The public IPs can be direclty set in the cgpool with a subnet:

cgmap myrule eth0 172.16.0.0/12 -> 32.96.118.0/24 block-size 256

Public IPs can be also configured via a variable:

$public_ips = 32.96.118.0/24
cgpool mypool $public_ip block-size 256

It’s also possible to configure discontiguous public IPs sets with a variable:

$public_ips = {32.96.118.0/24, 32.96.119.0/24, 32.96.118.32 - 32.96.118.70}

address pooling

The address-pooling argument can have two values:

• paired (default value): It means that the same public IP address is used for all sessions originating from the same CPE.

• no-paired: It means that different public IP addresses can be used for different sessions originating from the same CPE.

It’s recommended to use paired address pooling for applications that require all sessions associated with one private IP address to be translated to the same public IP address for multiple sessions.

block allocation

Dynamic allocation

With a NAT rule, all the ports are allocated dynamically from the same pool of port for any IP address. With a CG-NAT rule, a block of ports is allocated when a packet coming from a new user match the cgmap rule. The number of ports per block can be configured with the option ‘block-size’.

For any new connections, a port is allocated from its block of ports. Port allocation is done until the block is exhausted. If a block is exhausted, a new block can be allocated for this user. It becomes the active block. There is only one active block per user.

The number of blocks per user can be configured with the option ‘max-blocks-per-cpe’ (default value is 1). It prevents a single user consuming all ports.

Since ports are allocated from the same block (until this one is empty), port prediction can potentialy happen. Thus, to randomize port allocation, even if there are still some ports available in the active block, a new active block is allocated if the current active block has been active more than ‘active-block-timeout’ seconds. As this option can increase the average numbers of blocks allocated per user, this one is disabled by default.

When all the ports are released from a non-active block, this one is released immediately. For the active block, the block is only released when the user subscription has expired. The user timeout can be configured with the option ‘cpe-timeout’ (default value is 120 seconds).

Deterministic allocation

ISPs have a legal logging requirement to be able to map a subscriber’s inside address with the address used on the public Internet (e.g., for abuse response). With dynamic allocation, any new block of port allocated/released need to be logged.

The port allocation algorithm for deterministic is predictable and sequential (i.e. the first block goes to address 1, the second block to address 2, etc.).

This allocation method enables to retrieve at any time a mapping between a public IP and a private one without any log.

A cgmap-deterministic rule needs to be associated to a cgpool-deterministic. Public IPs can not be shared between deterministic and non deterministic rule.

For cgpool-deterministic, the block-size option is not mandatory. In fact, it’s recommended to not use it. Because it’s automatically computed to this maximal value with the following formula: (port_range_of_the_pool * number_pool_ips) / number_of_cpes.

The CPE have always one block per protocol. As a consequence cgmap-deterministic doesn’t support the ‘max-blocks-per-cpe’ option.

endpoint-mapping option

The endpoint-mapping argument can have two values:

• dependent: The NAT reuses the port mapping for subsequent packets sent from the same internal IP address and port to the same external IP address and port.

• independent (default value): The NAT reuses the port mapping for subsequent packets sent from the same internal IP address and port (X:x) to any external IP address and port.

endpoint-filtering option

The endpoint-filtering argument can have two values:

• dependent: Inbound packets from external endpoints are filtered out if they don’t match an existing mapping.

• independent (default value): Inbound packets from external endpoints are only filtered out if their destination IP address and port don’t match an existing NAT external IP address and port mapping.

hairpinning option

Hairpinning feature allows two endpoints (user 1 and user 2) on the internal network to communicate by using their external IP addresses and ports.

When packets from the internal network has been sourced nated, the destination IP is checked against a NAT table to recognize if this one is addressed to a host in the internal network.

In this case, the destination IP address is translated and the packet is sent back to the internal network.

port-algo option

The port algo argument allows choosing between two methods to allocate NAT ports:

• parity (default value): This algo preserves the parity of the port, i.e., an even port will be mapped to an even port, and an odd port will be mapped to an odd port.

• random: This algo chooses a port randomly.

Control tool

Fast Path CG-NAT can be controlled through the fp-npfctl application.

Usage:

# fp-npfctl
Usage: fp-npfctl [vrf-exec ID] start | stop | flush | show
      fp-npfctl [vrf-exec ID] validate | reload [<rule-file>]
      fp-npfctl [vrf-exec <id>|all] cgnat-rule { <rule-name> | all } { block-show | conntrack-show | cpe-count | cpe-show | cpe-reset-stats | cpe-usage | cpe-remove | cpe-flush | conntrack-stats | block-stats | port-stats } [<address>|threshold-errors NB|<proto>|usage-min 1-100]
      fp-npfctl [vrf-exec <id>|all] cgnat-pool { <pool-name> | all } { ip-count | ip-show | ip-stats | ip-usage } [<address>|<proto>|usage-min 1-100]
      fp-npfctl [vrf-exec ID] cgnat-deterministic { get-public-block <cpe-ip-addr> | get-cpe-ip <pub-ip-addr> <port> | dump-mappings | dump-config }
      fp-npfctl pool-usage
      fp-npfctl htable-stats
      fp-npfctl [vrf-exec ID] option { show | reset | set }
      fp-npfctl stats { show | reset } [json]

The ‘start’ and ‘stop’ commands do not actually change (i.e. load or unload) the active configuration. Running ‘start’ will only enable the passing of packets through Fast Path CG-NAT, while ‘stop’ will disable such passing.

Therefore, configuration should first be loaded using the ‘reload’ command and then nating enabled with ‘start’.

Similarly, clearing of the active configuration is done by performing the ‘stop’ and ‘flush’ commands.

Any change to npf.conf requires reloading the CG-NAT rules by performing a ‘reload’ command in order to make the changes active.

Such behaviour allows users to efficiently disable and enable CG-NAT without actually changing the active configuration, as it may be unnecessary.

cgnat-rule commands

cpe-show

The command ‘cpe-show’ shows all the CPE of a CG-NAT rule. For each CPE, it displays the number of block users and its statistics.

# fp-npfctl cgnat-rule "myrule" cpe-show
10.110.32.1 -> 10.130.0.69
    631/1024 conntracks
    3/4 tcp blocks, 0/4 udp blocks, 0/4 icmp blocks, 0/4 gre blocks
    0 no port errors, 0 no block errors, 0 full public ip errors
10.110.20.243 -> 10.130.0.155
    573/1024 conntracks
    3/4 tcp blocks, 0/4 udp blocks, 0/4 icmp blocks, 0/4 gre blocks
    0 no port errors, 0 no block errors, 0 full public ip errors
10.110.5.35 -> 10.130.0.234
    704/1024 conntracks
    3/4 tcp blocks, 0/4 udp blocks, 0/4 icmp blocks, 0/4 gre blocks
    0 no port errors, 0 no block errors, 0 full public ip errors
10.110.29.9 -> 10.130.0.38
    404/1024 conntracks
    3/4 tcp blocks, 0/4 udp blocks, 0/4 icmp blocks, 0/4 gre blocks
    0 no port errors, 0 no block errors, 0 full public ip errors

The command can also display only one CPE.

# fp-npfctl cgnat-rule "myrule" cpe-show 10.110.0.1
10.110.0.1 -> 10.130.0.133
  128/1024 conntracks
  1/4 tcp blocks, 0/4 udp blocks, 0/4 icmp blocks, 0/4 gre blocks
  0 no port errors, 0 no block errors, 0 full public ip errors

The option “threshold-errors N” can be used to only display CPE having more than or equal to N errors.

# fp-npfctl cgnat-rule "myrule" cpe-show threshold-errors 2
10.110.0.37 -> 10.130.0.38
    4/4 tcp blocks, 0/4 udp blocks, 0/4 icmp blocks, 0/4 gre blocks
    45 no port errors, 0 no block errors, 0 full public ip errors
10.110.0.14 -> 10.130.0.15
    4/4 tcp blocks, 0/4 udp blocks, 0/4 icmp blocks, 0/4 gre blocks
    195 no port errors, 0 no block errors, 0 full public ip errors
10.110.0.53 -> 10.130.0.54
    4/4 tcp blocks, 0/4 udp blocks, 0/4 icmp blocks, 0/4 gre blocks
    5 no port errors, 0 no block errors, 0 full public ip errors

The option “usage-min P” can be used to only display CPE using at least P (between 1 and 100) % of their maximal numbers of ports (i.e. max-blocks-per-cpe * block-size) or P% of their maximal numbers of connection (i.e. max-conntracks-per-cpe).

# fp-npfctl cgnat-rule "myrule" cpe-show usage-min 55
10.110.34.2 -> 10.130.0.99
        252 conntracks
        4/4 tcp blocks, 0/4 udp blocks, 0/4 icmp blocks, 0/4 gre blocks
        0 no port errors, 0 no block errors, 0 full public ip errors
10.110.32.132 -> 10.130.0.203
        253 conntracks
        4/4 tcp blocks, 0/4 udp blocks, 0/4 icmp blocks, 0/4 gre blocks
        0 no port errors, 0 no block errors, 0 full public ip errors
10.110.37.179 -> 10.130.0.74
        251 conntracks
        4/4 tcp blocks, 0/4 udp blocks, 0/4 icmp blocks, 0/4 gre blocks
        0 no port errors, 0 no block errors, 0 full public ip errors

cpe-count

The command ‘cpe-count’ counts all the CPE of a CG-NAT rule. It supports the same options than the ‘cpe-show’ command (i.e. usage-min and threshold-errors).

# fp-npfctl cgnat-rule "myrule" cpe-count usage-min 55
120

cpe-reset-stats

This command can be used to reset the statistics of a specific CPE or for several CPEs by filtering with “threshold-errors” or “usage-min” option or of all CPEs.

# fp-npfctl cgnat-rule "myrule" cpe-reset-stats 10.110.0.1
# fp-npfctl cgnat-rule "myrule" cpe-reset-stats usage-min 55
# fp-npfctl cgnat-rule "myrule" cpe-reset-stats

cpe-usage

The “cpe-usage” command displays the number of ports used for a CPE.

# fp-npfctl cgnat-rule "myrule" cpe-usage 10.110.0.75
tcp port usage: 3/256
udp port usage: 0/256
icmp port usage: 0/256
gre port usage: 0/256

cpe-remove

The “cpe-remove” command removes a CPE and all its conntracks.

# fp-npfctl cgnat-rule "myrule" cpe-remove 10.110.0.75

cpe-flush

The “cpe-flush” flush all the CPEs of a CG-NAT rule.

# fp-npfctl cgnat-rule "myrule" cpe-flush 10.110.0.75

block-show

The “block-show” command displays all the blocks of a CPE.

# fp-npfctl cgnat-rule myrule block-show 10.110.35.5
       BLOCK(0x7f614038fe00): status active, proto 6 tports 56705 - 56768 parity=0, usage 40/64, type dynamic
       BLOCK(0x7f614027ec00): status inactive, proto 6 tports 54081 - 54144 parity=0, usage 60/64, type dynamic
       BLOCK(0x7f6140a31b40): status inactive, proto 6 tports 59649 - 59712 parity=0, usage 54/64, type dynamic

conntrack-show

The “conntrack-show” command displays all the conntracks of a CPE.

# fp-npfctl cgnat-rule "myrule" conntrack-show 10.110.0.1
CONN:
       vrfid 0 proto 17 flags 0x45 tsdiff 0 timeout 120
       forw 10.200.0.1:57748-> 10.125.0.1:1027 hash:5d41d319
       back 10.100.0.1:5357-> 10.200.0.1:57748 hash:d08d2053
       state established
       NAT: original address 10.100.0.1 proto 17 oport 5357 tport 1027
CE:
       vrfid 0 proto 17 flags 0x81 timeout 60
       10.200.0.1:0-> 10.125.0.1:1027 hash:a7dc0492
       NAT: original address 10.100.0.1 proto 17 oport 5357 tport 1027
CONN:
       vrfid 0 proto 17 flags 0x45 tsdiff 0 timeout 30
       forw 10.200.0.1:57747-> 10.125.0.1:1026 hash:6decb7b2
       back 10.100.0.1:5356-> 10.200.0.1:57747 hash:549296ec
       state new
       NAT: original address 10.100.0.1 proto 17 oport 5356 tport 1026

cgnat-pool commands

The cgnat-pool commands are here to monitor usage of the cgnat pools.

These commands are not supported for deterministic pools, because these ones are always used at 100%.

ip-show

The “ip-show” command displays the number of blocks used for one public IP or for all the public IPs of a CG-NAT rule.

# fp-npfctl cgnat-pool "mypool" ip-show
10.130.0.2
   126/1008 tcp blocks, 42/1008 udp blocks, 42/1008 icmp blocks, 42/1008 gre blocks
10.130.0.3
   600/1008 tcp blocks, 202/1008 udp blocks, 202/1008 icmp blocks, 202/1008 gre blocks

The “usage-min” filter can be used to display only the IPs used by at least the given rate.

# fp-npfctl cgnat-pool "mypool" ip-show usage-min 50
10.130.0.3
   600/1008 tcp blocks, 202/1008 udp blocks, 202/1008 icmp blocks, 202/1008 gre blocks

ip-count

The “ip-count” command counts the number of public IPs of a CG-NAT pool. The “usage-min” filter can be used to count only the IPs used by at least the given rate.

# fp-npfctl cgnat-pool mypool ip-count
241
# fp-npfctl cgnat-pool mypool ip-count usage-min 90
10

ip-usage

The “ip-usage” command displays the number of blocks used for one public IP or for all the public IPs of a CG-NAT rule.

# fp-npfctl cgnat-pool "mypool" ip-usage 10.130.0.2
  tcp block usage: 126/1008 (12.50%)
  udp block usage: 42/1008 (4.17%)
  icmp block usage: 42/1008 (4.17%)
  gre block usage: 42/1008 (4.17%)
#
# fp-npfctl cgnat-pool "mypool" ip-usage
  tcp block usage: 30000/242928 (12.35%)
  udp block usage: 10000/242928 (4.12%)
  icmp block usage: 10000/242928 (4.12%)
  gre block usage: 10000/242928 (4.12%)

cgnat-deterministic commands

get-public-block

The “get-public-block” command retrieves the public block used by a cpe.

# fp-npfctl cgnat-deterministic get-public-block 10.100.0.25
10.175.0.3 1601-1664

get-cpe-ip

The “get-cpe-ip” command retrieves the cpe ip address using a specifc public address and port.

# fp-npfctl cgnat-deterministic get-cpe-ip 10.175.0.3 4201
10.100.0.65

dump-mappings

This “dump-mappings” command dumps mappings between cpe address and public ones.

# fp-npfctl cgnat-deterministic dump-mappings
10.100.0.0: 10.175.0.3 1 64
10.100.0.1: 10.175.0.3 65 128
10.100.0.2: 10.175.0.3 129 192
10.100.0.3: 10.175.0.3 193 256
10.100.0.4: 10.175.0.3 257 320
10.100.0.5: 10.175.0.3 321 384
...

option

The “option” command displays the different parameters of Fast Path CG-NAT.

# fp-npfctl option show
nat64_update_tcp_mss = 1
nat64_force_frag4 = 0
nat64_force_frag6 = 0
nat64_drop_udp4_zero_checksum = 1
nat64_lowest_ipv6_mtu = 0
icmp_connection_timeout_closed = 0
icmp_connection_timeout_new = 30
icmp_connection_timeout_established = 60
udp_connection_timeout_closed = 0
udp_connection_timeout_new = 30
udp_connection_timeout_established = 120
gre_pptp_connection_timeout_closed = 0
gre_pptp_connection_timeout_new = 600
gre_pptp_connection_timeout_established = 18000
tcp_connection_timeout_closed = 10
tcp_connection_timeout_syn_sent = 30
tcp_connection_timeout_simsyn_sent = 30
tcp_connection_timeout_syn_received = 60
tcp_connection_timeout_established = 7440
tcp_connection_timeout_fin_sent = 120
tcp_connection_timeout_fin_received = 120
tcp_connection_timeout_close_wait = 60
tcp_connection_timeout_fin_wait = 120
tcp_connection_timeout_last_ack = 30
tcp_connection_timeout_time_wait = 120
tcp_check_window = 1
tcp_strict_order_rst = 1

The “option” command can also be used to change the value of a parameter.

# fp-npfctl option set tcp_connection_timeout_last_ack 1

The “option” command can also be used to reset the value of a parameter to the default value.

# fp-npfctl option reset tcp_connection_timeout_last_ack

pool-usage

The “pool-usage” command displays the percentage of entries used in each memory pools used to allocate the different Fast Path CG-NAT objects.

Example:

# fp-npfctl pool-usage
npf_table_pool : 0/9184 (0.00%)
npf_conn_pool : 1280499/30009160 (4.27%)
npf_cpe_pool : 20000/20000 (100.00%)
npf_block_pool : 50045/80000 (62.56%)
npf_nat_pool : 1280499/30009160 (4.27%)

htable-stats

The “htable-stats” command displays the average number of elements per hash table entry. It does not take into account empty entries.

Example:

# fp-npfctl htable-stats
npf_expect_conndb: 33554432 buckets (non empty = 0, ratio 0.00%), entries per bucket: min = 0, max = 0, average = 0.00
npf_conndb: 33554432 buckets (non empty = 7049590, ratio 21.01%), entries per bucket: min = 1, max = 6, average = 1.12
        6250694 buckets (88.67%) have 1 entry
        737386 buckets (10.46%) have 2 entries
        57811 buckets (0.82%) have 3 entries
        3519 buckets (0.05%) have 4 entries
        177 buckets (0.00%) have 5 entries
        3 buckets (0.00%) have 6 entries
npf_int_nat_htable: 33554432 buckets (non empty = 2832664, ratio 8.44%), entries per bucket: min = 1, max = 5, average = 1.04
        2709784 buckets (95.66%) have 1 entry
        119260 buckets (4.21%) have 2 entries
        3543 buckets (0.13%) have 3 entries
        76 buckets (0.00%) have 4 entries
        1 bucket (0.00%) have 5 entries
npf_ext_nat_htable: 33554432 buckets (non empty = 2782049, ratio 8.29%), entries per bucket: min = 1, max = 5, average = 1.04
        2663870 buckets (95.75%) have 1 entry
        114808 buckets (4.13%) have 2 entries
        3306 buckets (0.12%) have 3 entries
        63 buckets (0.00%) have 4 entries
        2 buckets (0.00%) have 5 entries

block-stats

The “block-stats” command displays the average number of blocks used per CPE.

Example:

# fp-npfctl cgnat-rule myrule block-stats tcp
block-usage: 10000 cpes (non empty = 10000, ratio 100.00%), blocks per cpe: min = 2, max = 3, average = 2.23
     7745 cpes (77.45%) have 2 blocks
     2255 cpes (22.55%) have 3 blocks

ip-stats

The “ip-stats” command displays the average number of blocks used per public IP.

# fp-npfctl cgnat-rule myrule ip-stats  tcp
block-usage: 241 ips (non empty = 241, ratio 100.00%), blocks per ip: min = 80, max = 86, average = 82.99
        1 ip (0.41%) have 117 blocks used
        6 ips (2.49%) have 120 blocks used
        107 ips (44.40%) have 123 blocks used
        127 ips (52.70%) have 126 blocks used

port-stats

The “port-stats” command displays the average number of ports used per CPE.

Example:

# fp-npfctl cgnat-rule myrule port-stats tcp
port-usage: 10000 cpes (non empty = 10000, ratio 100.00%), ports per cpe: min = 155, max = 157, average = 156.47
        341 cpes (3.41%) have 155 ports
        4628 cpes (46.28%) have 156 ports
        5031 cpes (50.31%) have 157 ports

conntrack-stats

The “conntrack-stats” command displays the average number of conntracks used per CPE.

Example:

# fp-npfctl cgnat-rule myrule conntrack-stats
conntrack-usage: 10000 cpes (non empty = 10000, ratio 100.00%), conntracks per cpe: min = 320, max = 352, average = 325.07
        8340 cpes (83.40%) have between 320 and 323 conntracks (average: 320.01)
        25 cpes (0.25%) have between 324 and 326 conntracks (average: 324.52)
        18 cpes (0.18%) have between 327 and 329 conntracks (average: 328.00)
        12 cpes (0.12%) have between 330 and 333 conntracks (average: 331.25)
        20 cpes (0.20%) have between 334 and 336 conntracks (average: 335.30)
        25 cpes (0.25%) have between 337 and 339 conntracks (average: 337.80)
        15 cpes (0.15%) have between 340 and 343 conntracks (average: 341.60)
        17 cpes (0.17%) have between 344 and 346 conntracks (average: 345.06)
        33 cpes (0.33%) have between 347 and 349 conntracks (average: 347.94)
        1495 cpes (14.95%) have between 350 and 352 conntracks (average: 351.95)

stats

The “stats” command displays the global statistics of Fast Path CG-NAT:

# fp-npfctl stats show
Packets passed:
       0 default pass
       255 ruleset pass
       1689 state pass
Packets blocked:
       0 default block
       0 ruleset block
State and NAT entries:
       255 state allocations
       0 state destructions
       255 NAT entry allocations
       0 NAT entry destructions
       0 NAT port allocation failures
CGNat entries:
       22 CPE allocations
       0 CPE destructions
       0 CPE allocation failures
       22 Block allocations
       0 Block destructions
       0 Block allocation failures
       0 No IP Public
       0 Full IP Public
NAT64 Stats:
       0 udp null checksum packet drops
Network buffers:
       0 non-contiguous cases
       0 contig alloc failures
Invalid packet state cases:
       0 cases in total
       0 TCP case RST
       0 TCP case I
       0 TCP case II
       0 TCP case III
Packet race cases:
       0 CPE association race
       0 CPE creation race
       0 NAT association race
       0 duplicate state race
Other:
       0 unexpected errors

The “stats” can also be displayed in json format:

# fp-npfctl stats show json
{
  "Packets passed": {
    "default pass": 0,
    "ruleset pass": 255,
    "state pass": 1785
  },
  "Packet race cases": {
    "duplicate state race": 0,
    "CPE association race": 0,
    "CPE creation race": 0,
    "NAT association race": 0
  },
  "Packets blocked": {
    "default block": 0,
    "ruleset block": 0
  },
  "State and NAT entries": {
    "NAT port allocation failures": 0,
    "state allocations": 255,
    "state destructions": 0,
    "NAT entry allocations": 255,
    "NAT entry destructions": 0
  },
  "CGNat entries": {
    "Block allocation failures": 0,
    "CPE allocations": 22,
    "CPE destructions": 0,
    "CPE allocation failures": 0,
    "Block allocations": 22,
    "Block destructions": 0,
    "Full IP Public": 0,
    "No IP Public": 0
  },
  "Network buffers": {
    "non-contiguous cases": 0,
    "contig alloc failures": 0
  },
  "Other": {
    "unexpected errors": 0
  },
  "Invalid packet state cases": {
    "cases in total": 0,
    "TCP case III": 0,
    "TCP case II": 0,
    "TCP case RST": 0,
    "TCP case I": 0
  }
}

It is also possible to reset the stats:

# fp-npfctl stats reset

Monitoring

port block (de)allocation

The “fp-cgnat-logd” can be used to track allocation/deallocation of port block for each CPE. The information are displayed using syslog.

The following fields are displayed for each port block allocation/deallocation for a CPE:

• IP address of the CPE

• name of the CG-NAT rule

• type of event: NEW BLOCK (a new port range is associated to this CPE) or DESTROY BLOCK (port range is released for this CPE).

• Public IP used by the CPE

• port range of the block

• l4 protocol (i.e. 6 for tcp, 17 for udp, 1 for icmp, 47 for gre)

• timestamp of the event

Example:

fp-cgnat-logd: CPE 10.110.0.1 (from cgnat "myrule"):
  NEW BLOCK (ip public 10.130.0.2, proto 6, port 1024 - 1087) at Tue Dec 19 16:07:53 2017
fp-cgnat-logd: CPE 10.110.0.1 (from cgnat "myrule"):
  DESTROY BLOCK (ip public 10.130.0.2, proto 6, port 1024 - 1087) at Tue Dec 19 16:08:10 2017

deterministic

The deterministic CG-NAT uses a predictable port allocation algorithm. This allocation method enables to retrieve at any time a mapping between a public IP and a private by using cgnat-deterministic commands of fp-npfctl.

However, these commands apply only on the current configuration loaded in the fast path. In order to be able to identify a subscriber based on observed external IPv4 address, port, and timestamp, an operator needs to know how the deterministic CG-NAT was configured.

The “fp-cgnat-logd” application can be used to track the deterministic CG-NAT configuration. The information is displayed using syslog.

The following format is used for deterministic CG-NAT configuration messages:

cgnat-deterministic-conf;[POOL_NAME|ALGO_NAME|[CPE_NETWORKS]+|[PUBLIC_IPS]+|BLOC_SIZE|PORT_START-PORT_END]+

Example:

cgnat-deterministic-conf;p1|sequential1.0|10.100.0.0/24|10.175.0.3|255|1-65535 at 2021-02-24T16:37:29Z

The “fp-cgnat-deterministic” application can be used to retrieve a mapping between a public and a private IP in function of the configuration message.

Example:

# CGNAT_CONF='cgnat-deterministic-conf;p1|sequential1.0|10.100.0.0/24|10.175.0.3|255|1-65535'
# fp-cgnat-deterministic get-cpe-ip 10.175.0.3 42011 $CGNAT_CONF
10.100.0.164
# fp-cgnat-deterministic get-public-block 10.100.0.164 $CGNAT_CONF
10.175.0.3 41821-42075

Runtime options for the fast path

You can dynamically set the following parameters when starting the fast path:

• Default port for logging agent --mod-opt=fp-cgnat:--notif-port=<port-number>.

  Default is 5001.

• Maximum number of CPE supported with option --mod-opt=fp-cgnat:--max-cpe=<cpe-number>.

  Default is 128.

• Maximum number of block supported with option --mod-opt=fp-cgnat:--max-block=<block-number>.

  Default is 512.

• Maximum number of ports per block supported with option --mod-opt=fp-cgnat:--max-block-size=<ports-number>.

  Default is 8192.