Note

CG-NAT requires a CG-NAT Application License.

CG-NAT

CG-NAT, also known as Large Scale NAT (LSN), is an extension of NAT for large scale networks and ISPs.

The key advantages of CG-NAT compared to NAT are:

• High Transparency: CG-NAT implements multiple advanced features like Endpoint-Independent Mapping, Endpoint-Independent Filtering, address pooling and port parity preservation. These features provide better experience to ‘nated’ users and allow scaling.

• Fairness and Resource Sharing: CG-NAT provides options to limit the number of connections per user. This ensures that resources are equitably shared between the different users.

• Optimized Logging system: CG-NAT can generate large amounts of logging data. CG-NAT implements a feature called port block allocation to limit the number of log entries by grouping per port range.

• Transition between IPv6-only and IPv4-only networks, with NAT64 in conjunction with DNS64 or DS-Lite.

Caution

Stateful IP packet filtering and CG-NAT are exclusive. If CG-NAT is enabled, stateful IP packet filtering must be disabled on ports bound to the fast path.

See also

The CG-NAT command reference and the CG-NAT fast path limits command reference for details.

Configuration

Pool

A CG-NAT pool contains a list of IPv4 addresses used to change the IPv4 or IPv6 source address and port of a packet.

The CG-NAT implements a feature called port block allocation. Each time a new user sent a packet through the CG-NAT router, a block of ports (i.e. a port range) from one of the IPv4 addresses in the pool is allocated to this one. The number of ports given to a user is configurable via the block-size option.

Here is an example of a CG-NAT pool:

vsr running config# vrf main
vsr running vrf main# cg-nat
vsr running cg-nat!# pool mypool
vsr running mypool!# address 192.0.2.33-192.0.2.49
vsr running mypool!# address 192.0.1.0/24
vsr running mypool!# address 192.0.3.1
vsr running mypool!# allocation-mode dynamic-block block-size 512

A pool needs to be associated to a CG-NAT rule, see the next section.

Rule

We have three types of NAT rules:

• dynamic: private addresses are dynamically mapped to public addresses: for each private address, a public address is dynamically allocated from a pool and associated to the private address.

• deterministic: each private address is always mapped to the same public address and is assigned a dedicated port pool at the time of configuration

• static: mapping between private addresses and public ones are static. Public addresses are configured directly in the rule.

static SNAT44

All packets routed through an output interface and matching the filtering criteria defined in a CG-NAT rule are source nated with an IP or a range of IP.

IP addresses are mapped one to one. For example, with the range 100.64.0.1-100.64.0.9 translated to 192.0.2.1-192.0.2.9:

• 100.64.0.1 is translated to 192.0.2.1

• 100.64.0.2 is translated to 192.0.2.2

• …

• 100.64.0.9 is translated to 192.0.2.9

Note

If one of the IP range is larger, it will be truncated to match the size of the other range.

Here is an example of a CG-NAT rule:

vsr running mypool!# ..
vsr running cg-nat#! rule 1
vsr running rule 1#! static-snat44
vsr running static-snat44#! match
vsr running match#! source ipv4-range 100.64.0.1-100.64.0.9
vsr running match#! outbound-interface eth1
vsr running match#! ..
vsr running static-snat44#! translate-to
vsr running translate-to#! ipv4-range 192.0.2.1-192.0.2.9
vsr running translate-to# commit

To display the applied configuration:

vsr running config# show state vrf main cg-nat
cg-nat
    enabled true
    rule 1
        static-snat44
            match
                source
                    ipv4-range 100.64.0.1-100.64.0.9
                    ..
                outbound-interface eth1
                ..
            translate-to
                ipv4-range 192.0.2.1-192.0.2.9
                ..
            ..
        ..
    logging
        enabled false
        ..
    ..

static DNAT44

All packets received on an input interface and matching the filtering criteria defined in a CG-NAT rule are destination nated with an ip or a range of IP.

Here is an example of a CG-NAT rule:

vsr running mypool!# ..
vsr running cg-nat#! rule 1
vsr running rule 1#! static-dnat44
vsr running static-dnat44#! match
vsr running match#! destination ipv4-range 192.0.2.1
vsr running match#! inbound-interface eth1
vsr running match#! ..
vsr running static-dnat44#! translate-to
vsr running translate-to#! ipv4-range 100.64.0.1
vsr running translate-to# commit

To display the applied configuration:

vsr running config# show state vrf main cg-nat
cg-nat
    enabled true
    rule 1
        static-dnat44
            match
                destination
                    ipv4-range 192.0.2.1
                    ..
                inbound-interface eth1
                ..
            translate-to
                ipv4-range 100.64.0.1
                ..
            ..
        ..
    logging
        enabled false
        ..
    ..

dynamic SNAT44

All packets routed through an output interface and matching the filtering criteria defined in a CG-NAT rule are source nated with an ip dynamically assigned from one CG-NAT pool.

Here is an example of a CG-NAT rule:

vsr running mypool!# ..
vsr running cg-nat#! rule 1
vsr running rule 1#! dynamic-snat44
vsr running dynamic-snat44#! match
vsr running match#! source ipv4-address 100.64.0.0/10
vsr running match#! outbound-interface eth1
vsr running match#! ..
vsr running dynamic-snat44#! translate-to
vsr running translate-to#! pool-name mypool
vsr running translate-to# commit

To display the applied configuration:

vsr running config# show state vrf main cg-nat
cg-nat
    enabled true
    pool mypool
        address 192.0.1.1-192.0.1.254
        address 192.0.2.33-192.0.2.49
        address 192.0.3.1
        allocation-mode dynamic-block
            block-size 512
            ..
        port-range
            1024
            65535
            ..
        ..
    rule 1
        dynamic-snat44
            match
                source
                    ipv4-address 100.64.0.0/10
                    ..
                outbound-interface eth1
                ..
            translate-to
                pool-name mypool
                max-conntracks-per-user 0
                max-blocks-per-user 1
                active-block-timeout 0
                user-timeout 120
                port-algo parity
                endpoint-mapping independent
                endpoint-filtering independent
                hairpinning false
                address-pooling paired
                ..
            ..
        ..
    logging
        enabled false
        ..
    ..

The same configuration can be made using this NETCONF XML configuration:

vsr running config# show config xml absolute vrf main cg-nat
<config xmlns="urn:6wind:vrouter">
  <vrf>
    <name>main</name>
    <cg-nat xmlns="urn:6wind:vrouter/cgnat">
      <enabled>true</enabled>
      <logging/>
      <pool>
        <name>mypool</name>
        <port-range/>
        <address>192.0.2.33-192.0.2.49</address>
        <address>192.0.1.0/24</address>
        <address>192.0.3.1</address>
        <allocation-mode>
          <dynamic-block>
            <block-size>512</block-size>
          </dynamic-block>
        </allocation-mode>
      </pool>
      <rule>
        <id>1</id>
        <dynamic-snat44>
          <match>
            <source>
              <ipv4-address>100.64.0.0/10</address>
            </source>
            <outbound-interface>eth1</outbound-interface>
          </match>
          <translate-to>
            <pool-name>mypool</pool-name>
          </translate-to>
        </dynamic-snat44>
      </rule>
    </cg-nat>
  </vrf>
</config>

dynamic port SNAT44

The dynamic-port-snat44 works like dynamic-snat44, except that IP and ports from the CG-NAT pool are directly associated without any port block. It means that all the ports of a public IP is shared with several users.

See the Understanding Port Allocation/dynamic port section for more information about this point.

Here is an example of a dynamic port CG-NAT rule:

vsr running mypool!# allocation-mode dynamic-port
vsr running mypool!# ..
vsr running cg-nat#! rule 1
vsr running rule 1#! dynamic-port-snat44
vsr running dynamic-port-snat44#! match
vsr running match#! source ipv4-address 100.64.0.0/10
vsr running match#! outbound-interface eth1
vsr running match#! ..
vsr running dynamic-port-snat44#! translate-to
vsr running translate-to#! pool-name mypool
vsr running translate-to# commit

deterministic SNAT44

The deterministic-snat44 works like dynamic-snat44, except that IP and ports from the CG-NAT pool are associated in a deterministic manner to the user at the time of configuration: the CG-NAT pool is partitioned in chunks of port blocks, so that each user has its own port block. See the Understanding Port Block Allocation/deterministic for more information about this point.

Here is an example of a deterministic CG-NAT rule:

vsr running mypool!# allocation-mode deterministic-block
vsr running mypool!# ..
vsr running cg-nat#! rule 1
vsr running rule 1#! deterministic-snat44
vsr running deterministic-snat44#! match
vsr running match#! source ipv4-address 100.64.0.0/10
vsr running match#! outbound-interface eth1
vsr running match#! ..
vsr running deterministic-snat44#! translate-to
vsr running translate-to#! pool-name mypool
vsr running translate-to# commit

dynamic SNAT64

In the case of NAT64, translation between IPv6-only addresses on the private side and IPv4 addresses on the public side is taken care of by mapping the IPv4 public addresses to a special IPv6 prefix that is configured in the translate-to section of the rule.

DNS64 must be configured to translate IPv4 addresses DNS replies to IPv6 ones using this prefix.

The rest of the configuration is similar to dynamic NAT44.

vsr running config# vrf main
vsr running vrf main# cg-nat
vsr running cg-nat#! rule 1
vsr running rule 1#! dynamic-snat64
vsr running dynamic-snat64#! match
vsr running match#! source ipv6-address fd00:100::/64
vsr running match#! outbound-interface eth2
vsr running match#! ..
vsr running dynamic-snat64#! translate-to
vsr running translate-to#! pool-name mypool
vsr running translate-to#! max-blocks-per-user 4
vsr running translate-to#! destination-prefix 64:ff9b::/96
vsr running translate-to# .. .. .. ..
vsr running vrf main# dns
vsr running dns# proxy
vsr running proxy# dns64 64:ff9b::/96
vsr running dns64 64:ff9b::/96# client fd00:100::/64
vsr running dns64 64:ff9b::/96# exclude 64:ff9b::/96
vsr running dns64 64:ff9b::/96# mapped not 10.0.0.0/8 192.168.0.0/16 172.16.0.0/12

The client option can be used to select which client the DNS64 function will apply to. The CPE subnet in the CG-NAT rule should have the same value.

The exclude option is a safeguard to ensure that no IPv6 address matching 64:ff9b::/96 will be returned to IPv6 clients. Without it the CG-NAT might translate IPv6 packets going to this IPv6 server into IPv4 packet with an incorrect IPv4 address.

The mapped option can be used to avoid mapping specific IPv4 address to IPv6. For example, it is a good idea not to embed any RFC 1918 addresses that name servers on the Internet might inadvertently return.

dynamic port SNAT64

The dynamic-port-snat64 works like dynamic-snat64, except that IP and ports from the CG-NAT pool are directly associated without any port block. It means that all the ports of a public IP is shared with several users.

See the Understanding Port Allocation/dynamic port section for more information about this point.

Here is an example of a dynamic port CG-NAT rule:

vsr running mypool!# allocation-mode dynamic-port
vsr running mypool!# ..
vsr running cg-nat#! rule 1
vsr running rule 1#! dynamic-port-snat64
vsr running dynamic-port-snat44#! match
vsr running match#! source ipv6-address fd00:100::/120
vsr running match#! outbound-interface eth1
vsr running match#! ..
vsr running dynamic-port-snat64#! translate-to
vsr running translate-to#! pool-name mypool
vsr running translate-to# destination-prefix 64:ff9b::/96
vsr running translate-to# commit

deterministic SNAT64

The deterministic-snat64 works like dynamic-snat64, except that IP and ports from the CG-NAT pool are associated in a deterministic manner to the user at the time of configuration: the CG-NAT pool is partitioned in chunk of port block, so that each user has its own port block. See the Understanding Port Allocation/deterministic block section for more information about this point.

Here is an example of a deterministic CG-NAT rule:

vsr running mypool!# allocation-mode deterministic-block
vsr running mypool!# ..
vsr running cg-nat#! rule 1
vsr running rule 1#! deterministic-snat64
vsr running deterministic-snat64#! match
vsr running match#! source ipv6-address fd00:100::/120
vsr running match#! outbound-interface eth1
vsr running match#! ..
vsr running deterministic-snat64#! translate-to
vsr running translate-to#! pool-name mypool
vsr running translate-to# destination-prefix 64:ff9b::/96
vsr running translate-to# commit

static SNAT64

All IPv6 packets routed through an output interface and matching the filtering criteria defined in a CG-NAT rule are source nated with an IPv4 address or an IPv4 address range.

IP addresses are mapped one to one. For example, with the range fd00:100::1-fd00:100::9 translated to 192.0.2.1-192.0.2.9:

• fd00:100::1 is translated to 192.0.2.1

• fd00:100::2 is translated to 192.0.2.2

• …

• fd00:100::9 is translated to 192.0.2.9

Note

If one of the IP range is larger, it will be truncated to match the size of the other range.

Here is an example of a CG-NAT rule:

vsr running mypool!# ..
vsr running cg-nat#! rule 1
vsr running rule 1#! static-snat64
vsr running static-snat64#! match
vsr running match#! source ipv6-range fd00:100::1-fd00:100::9
vsr running match#! outbound-interface eth1
vsr running match#! ..
vsr running static-snat64#! translate-to
vsr running translate-to#! ipv4-range 192.0.2.1-192.0.2.9
vsr running translate-to# commit

static DNAT46

All IPv4 packets received on an input interface and matching the filtering criteria defined in a CG-NAT rule are destination nated with an IPv6 address or an IPv6 address range.

Here is an example of a CG-NAT rule:

vsr running mypool!# ..
vsr running cg-nat#! rule 1
vsr running rule 1#! static-dnat64
vsr running static-dnat46#! match
vsr running match#! destination ipv4-range 192.0.2.1
vsr running match#! inbound-interface eth1
vsr running match#! ..
vsr running static-dnat46#! translate-to
vsr running translate-to#! ipv6-range fd00:100::1
vsr running translate-to# commit

Dual-Stack Lite

DS-Lite is a transitional technology to provide IPv4 connectivity over IPv6 networks. The primary use case for DS-Lite is to allow ISPs to continue supporting IPv4-only clients while transitioning to IPv6 as the new protocol. DS-Lite uses IPv4-in-IPv6 tunneling, these tunnels are called softwires.

The Dual-Stack Lite deployment model consists of 2 main components:

• B4: It resides in the CPEs, it encapsulates IPv4 packets within an IPv6 header and transmits them into the IPv6 access network. IPv6 packets originated by CPE are transported natively over the access network.

• AFTR: It resides in the ISPs core network, it decapsulates IPv6 packets to recover the original IPv4 packet and processes source NAT44 address and port translations.

Overlapping IPv4 address spaces used by CPEs are disambiguated through the source IPv6 address (B4 address) of the incoming packets.

In order to configure a DS-Lite AFTR element, you need to configure a ipip interface with the option dual-stack-lite-aftr enabled:

vsr running vrf main# interface ipip aftr
vsr running ipip aftr#! local fd00:100::1 remote :: link-interface eth0
vsr running ipip aftr# ipv4 address 192.0.0.1/29
vsr running ipip aftr# ds-lite-aftr true
vsr running ipip aftr# commit

DS-Lite is using CG-NAT SNAT44 rules. Here is an example of a CG-NAT rule:

vsr running mypool!# ..
vsr running cg-nat#! rule 1
vsr running rule 1#! dynamic-snat44
vsr running dynamic-snat44#! match
vsr running match#! source ipv4-address 192.168.0.0/16
vsr running match#! outbound-interface eth1
vsr running match#! ds-lite
vsr running ds-lite#! softwire-address fd00:100::/120
vsr running ds-lite#! .. ..
vsr running dynamic-snat44#! translate-to
vsr running translate-to#! pool-name mypool
vsr running translate-to# commit

It is possible to assign a different rules for each CPE network via the softwire-address option.

Note

• As decribed in the RFC 6333, The tunnel introduces overhead to the packet and decreases the effective MTU size after encapsulation. Since fragmentation and reassembly is not optimal, a solution to deal with this problem is for the ISP to increase the MTU size of all the links between the B4 element and the AFTR elements by at least 40 bytes. If the access network MTU size is fixed and cannot be changed, some alternative methods, like rewriting the TCP MSS might be considered but are currently not supported.

• The RFC 7785 describs a new option subscriber-mask that indicates the length of significant bits to be applied on the source IPv6 address to identify unambiguously a CPE. This option is currently not supported and one IPv6 address is considered as a CPE.

• ALGs are currently not supported with DS-Lite.

Understanding Port Allocation

dynamic port

Each time a new user sends a packet through the Virtual Service Router, an IP address in the pool is assigned to him. The public IPs are selected using a round-robin algorithm.

Here is an example to configure a pool with dynamic-port allocation-mode:

vsr running config# vrf main
vsr running vrf main# cg-nat
vsr running cg-nat!# pool mypool
vsr running mypool!# address 192.0.2.33-192.0.2.49
vsr running mypool!# address 192.0.1.0/24
vsr running mypool!# address 192.0.3.1
vsr running mypool!# allocation-mode dynamic-port port-algo random

For legal requirement, all connections must be logged to be able to provide the mapping between a user and a public IP address at a given point in time.

To enable connection logging:

vsr running config# vrf main cg-nat logging enabled true
vsr running config# vrf main cg-nat logging local true
vsr running config# vrf main cg-nat logging event connection
vsr running config# commit

As a user can consume all the ports of a public IP address is highly recommended to limit the maximum numbers of connections per users by using max-conntracks-per-user parameters of the cg-nat rule.

vsr running config# vrf main cg-nat rule 1 dynamic-port-snat44 translate-to max-conntracks-per-user 512

dynamic block

It is a legal requirement for an ISP to be able to provide the mapping between a user and a public IP address at a given point in time. With classic NAT, it means that each new user session has to be logged. This is obviously not scalable. Additionally with classic NAT, a user can consume all the ports of the public IP.

To reduce the amount of logs and equitably share the ports of the different public IPs, CG-NAT provides the Port Block Allocation (PBA) feature that consists in allocating blocks of ports to each user. As logging is done per block of ports, the amount of logs is reduced. And as the number and size of the blocks can be configured, the user port consumption is controlled. Here is how PBA works.

Each time a new user sends a packet through the Virtual Service Router, a block of ports is allocated to him from one of the IP addresses in the pool. The public IPs are selected using a round-robin algorithm. Each public IP is divided into blocks of ports, whose size and range are defined in the pool configuration. This prevents a single user from consuming all ports.

Here is an example to change the number of ports per block:

vsr running cg-nat# / vrf main cg-nat pool mypool allocation-mode dynamic-block block-size 256
vsr running cg-nat# commit

For the next session of the same user, a port is allocated from its block of ports, until the block is exhausted. In that case, a new block can be allocated for this user and it becomes the active block. There is only one active block per user. The maximum number of blocks per user is defined in the rule configuration.

Here is an example to change the maximum number of blocks per user:

vsr running cg-nat# / vrf main cg-nat rule 1 translate-to
max-blocks-per-user 2
vsr running cg-nat# commit

Since ports are allocated from the same block (until this one is empty), port prediction can potentially happen. To randomize port allocation, it is possible to allocate a new active block if the current active block has been active for too long, even if there are still some ports available in the active block. This feature is called active block timeout. As it can increase the average numbers of blocks allocated per user, it is disabled by default.

Here is an example to configure an active block timeout of 60 seconds:

vsr running cg-nat# / vrf main cg-nat rule 1 translate-to
active-block-timeout 60
vsr running cg-nat# commit

When all the ports are released from a non-active block, this one is released immediately. Regarding the active block, the block is only released when the user subscription times out. The default user timeout is 120 seconds.

Here is an example to change the user timeout:

vsr running cg-nat# / vrf main cg-nat rule 1 translate-to user-timeout 180
vsr running cg-nat# commit

deterministic block

ISPs have a legal requirement to be able to map a subscriber’s inside address with the address and port used on the public Internet (e.g., for abuse response). With dynamic allocation, any new block of port allocated/released need to be logged.

The port allocation algorithm for deterministic is predictable and sequential (i.e. the first block goes to address 1, the second block to address 2, etc.).

Thanks to this allocation method, it is possible to retrieve at any time a mapping between a public IP and a private one without any log. The following commands can be used for this purpose:

vsr> show cg-nat deterministic-public-block user-address 10.100.0.59
10.175.0.3 15046-15300
vsr> show cg-nat deterministic-user-address public-address 10.175.0.3 public-port 17001
10.100.0.66
vsr> show cg-nat deterministic-mappings | pager
10.100.0.0: 10.175.0.3 1-255
10.100.0.1: 10.175.0.3 256-510
10.100.0.2: 10.175.0.3 511-765
...

A deterministic rule needs to be associated to a pool with allocation-mode set to deterministic-block. Public IPs cannot be shared between deterministic and dynamic rules.

For deterministic rules, the block-size option is not mandatory. It is even recommended to not use it because it is automatically computed to this maximal value with the following formula: (port_range_of_the_pool * number_pool_ips) / number_of_users.

The users always have one block per protocol. As a consequence, deterministic rules don’t support the ‘max-blocks-per-user’ option.

Address Pooling

The address-pooling option defines if a public IP address is paired to a user:

• paired (default value): It means that the same public IP address is used for all sessions originating from the same user.

• no-paired: It means that different public IP addresses can be used for different sessions originating from the same user.

It’s recommended to use paired address pooling for applications that require all sessions associated with one private IP address to be translated to the same public IP address for multiple sessions.

Port algorithm

The port-algo defines which method is used to allocate ports:

• parity (default): This algorithm preserves the parity of the port, i.e. an even port will be mapped to an even port, and an odd port will be mapped to an odd port.

• random: This algorithm chooses a port randomly.

Here is an example to configure random port allocation for dynamic or deterministic rules.

vsr running translate-to# port-algo random
vsr running translate-to# commit

The port-algo can be configured on rule when block of ports are provided by the IP pool to each user. As each user has its own chunk of ports, a different algorithm can be used.

When ports are provided by the IP pool to each user. The algorithm to allocate a port must be configured on the pool. As the ports of an IP address is common to different users, the port allocation algorithm must be the same.

Port overloading

Port overloading option allows concurrent use of a single NAT translated port for multiple transport sessions, which allows a NAT to successfully process packets in a network that has a limited number of IP addresses. When enabled, these options can be configured:

• unique-destination: Overload a port only when the destination address is unique or destination address and port pair is unique.

• protocol: Enable port overloading for protocol TCP, UDP or both.

• factor: Specify how much times a port can be used. For example, with a port range of 64512 and a port factor of 2, the maximum port capacity will be 129024.

Here is an example to configure port overloading for dynamic port pool.

vsr running dynamic-port# port-overloading
vsr running port-overloading# unique-destination address-and-port protocol tcp port-factor 32
vsr running port-overloading# commit

Note

• Ports will be overloaded only when all ports available are at least used 1 times.

• Port overloading is not compatible with endpoint mapping, endpoint filtering and hairpinning features. If port-overloading protocol option is set to TCP, endpoint mapping, endpoint filtering and hairpinning features for UDP packets will continue to work, and vice versa.

• ALG compatible with port overloading are dns, ftp, pptp, rtsp and tftp. Other ALG will be ignored for rule with port-overloading but if port-overloading protocol option is set to TCP, ALG using UDP will continue to work, and vice versa.

• Port overloading for dynamic port allocation mode is configured on the pool.

Warning

Port overloading feature is disabled by default. To enable the feature, the maximal port-overloading factor allowed must be configured in system/fast-path/limits/cg-nat/max-port-overloading-factor.

Active Block Timeout

The active-block-timeout defines how the active block changes:

• 0 (default): The active block changes only on port allocation, if it is full.

• any other value: The active block also changes every time the timeout expires.

Note

As this option can increase the average number of blocks allocated per user, it is disabled by default.

Here is an example to configure active block timeout.

vsr running translate-to# active-block-timeout 60
vsr running translate-to# commit

Endpoint mapping

There are two endpoint mapping behaviors:

• independent (default): The Virtual Service Router reuses the same port mapping for subsequent packets sent from the same internal IP address and port to any external IP address and port.

• dependent: The Virtual Service Router reuses the same port mapping for subsequent packets sent from the same internal IP address and port to the same external IP address and port.

Here is an example to configure dependent endpoint mapping.

vsr running translate-to# endpoint-mapping dependent
vsr running translate-to# commit

Endpoint filtering

There are two endpoint filtering behaviors:

• independent (default): Inbound packets from external endpoints are only filtered out if their destination IP address and port don’t match an existing public IP address and port mapping.

• dependent: Inbound packets from external endpoints are filtered out if they don’t match an existing mapping (source and destination IPs and ports, protocol).

Here is an example to configure dependent endpoint filtering.

vsr running translate-to# endpoint-filtering dependent
vsr running translate-to# commit

Hairpinning

The hairpinning feature allows two endpoints (user 1 and user 2) on the private network to communicate together using their public IP addresses and ports.

By default, the hairpinning feature is disabled. To enable it, use the following command.

vsr running translate-to# hairpinning true
vsr running translate-to# commit

Summary

The following table summarizes the different parameters for CG-NAT configuration.

Warning

Changing some configuration parameters requires to flush users. This is automatically done when required (see the table below), and causes a service interruption.

Category	Parameter	Flush	Note
pool	address	no	New addresses can be added without any impact. If an address is removed, all users assigned to it are flushed.
	block-size	yes
	port-algo	yes
	port-overloading	yes	Any options change will flush users.
	port-range	yes
rule	match	no	Conntracks not matching the new criteria are not flushed. Conntracks matching the new criteria are flushed.
	pool-name	yes
	max-blocks-per-user	no	Extra blocks are not flushed for users having more blocks than the new max. They become inactive and will be flushed after all sessions are released.
	active-block-timeout	no	The new timeout starts after the current one expires.
	user-timeout	no	The new timeout starts after the current one expires.
	port-algo	no	For new blocks only.
	port-overloading	yes	Changing factor does not require a flush and will be applied for new blocks only. Any others options change will flush users.
	endpoint-mapping	no	For new mappings only.
	endpoint-filtering	no	For new mappings only.
	hairpinning	no	For new mappings only.
	address-pooling	no	For new users only.

Logging

events

Different types of event can be logged for CG-NAT.

connection

You can enable logs for CG-NAT to track each connection allocation/deallocation.

vsr running config# vrf main cg-nat logging enabled true
vsr running config# vrf main cg-nat logging local true
vsr running config# vrf main cg-nat logging event connection
vsr running config# commit

The logs can be displayed with the following command:

vsr running config# show log service cg-nat
Jun 14 12:17:35 dut-vm fp-cgnat[2484]: NEW CONN: fwd proto 6 10.100.0.1:4241 -> 10.200.0.1:33815, back proto 6 10.200.0.1:33815 --> 10.175.0.3:10752
Jun 14 12:19:04 dut-vm fp-cgnat[2484]: DESTROY CONN: fwd proto 6 10.100.0.1:4241 -> 10.200.0.1:33815, back proto 6 10.200.0.1:33915 -> 10.175.0.3:10753

The following fields are displayed for each connection allocation/deallocation:

• timestamp of the event

• type of event: NEW CONN (a new connection is created) or DESTROY CONN (a connection is released)

• the forward flow with the following fields: - l4 protocol (i.e. 6 for tcp, 17 for udp, 1 for icmp, 47 for gre) - ip source and port source - ip destination and port destination

• the backward flow with the same fields as the forward one

port blocks

You can enable logs for CG-NAT to track each port block allocation/deallocation for a user :

vsr running config# vrf main cg-nat logging enabled true
vsr running config# vrf main cg-nat logging local true
vsr running config# vrf main cg-nat logging event port-block
vsr running config# commit

The logs can be displayed with the following command:

vsr running config# show log service cg-nat
Jun 11 08:02:46 vsr fp-cgnat[4269]: CGNAT Log listen on 5001
Jun 11 08:03:09 vsr fp-cgnat[4269]: USER 100.64.0.1 (matching rule 1): NEW BLOCK (pool "mypool", ip public 192.0.2.33, proto 6, port 1 - 512)
Jun 11 08:07:30 vsr fp-cgnat[4269]: USER 100.64.0.1 (matching rule 1): DESTROY BLOCK (pool "mypool", ip public 192.0.2.33, proto 6, port 1 - 512)

The following fields are displayed for each port block allocation/deallocation:

• IP address of the user

• name of the CG-NAT rule matched by the user

• type of event: NEW BLOCK (a new port range is associated to this user) or DESTROY BLOCK (port range is released for this user).

• pool and ip used to nat the user

• port range of the block

• l4 protocol (i.e. 6 for tcp, 17 for udp, 1 for icmp, 47 for gre)

deterministic conf

The deterministic algorithm is predictable, so block allocation/deallocations don’t need to be logged. Anyway, to retrieve the mapping between user and public address, the deterministic algorithm needs to know the deterministic CG-NAT configuration.

To log the deterministic CG-NAT configuration:

vsr running config# vrf main cg-nat logging enabled true
vsr running config# vrf main cg-nat logging local true
vsr running config# vrf main cg-nat logging event deterministic-conf
vsr running config# commit

In consequence, the deterministic CG-NAT configuration is logged.

vsr running config# show log service cg-nat
Mar 02 14:32:06 dut-vm fp-cgnat[3526]: CGNAT Log listen on 5001
Mar 02 14:32:06 dut-vm fp-cgnat[3526]: cgnat-deterministic-conf;none
Mar 02 14:35:42 dut-vm fp-cgnat[3526]: cgnat-deterministic-conf;mypool|sequential1.0|10.100.0.0/24|10.175.0.3|255|1-65535

The deterministic CG-NAT configuration can be used to retrieve a user address behind a public address/port with the fp-cgnat-deterministic application.

# fp-cgnat-deterministic get-cpe-ip 10.175.0.3 17001 \
  'cgnat-deterministic-conf;mypool|sequential1.0|10.100.0.0/24|10.175.0.3|255|1-65535'
10.100.0.66

The ‘show cg-nat deterministic-public-block/deterministic-user-address/deterministic-mappings’ commands can also be used to retrieve a mapping for a previous deterministic CG-NAT configuration by using the date-and-time parameter. This one is able to retrieve the configuration applied at this time and get the information requested with fp-cgnat-deterministic application.

vsr> show cg-nat deterministic-user-address public-address 10.175.0.3 public-port 17001 date-and-time '2021-03-03 11:21:00'
10.100.0.66

remote syslog servers

For performance reason and permanent storage, it is highly recommended to log to one or more remote syslog servers. Logging in local should be only used for debug purpose.

We support octet-counting and non-transparent-framing to separate log. For more details, see RFC 6587.

vsr running config# vrf main cg-nat logging enabled true
vsr running config# vrf main cg-nat logging local false
vsr running config# vrf main cg-nat logging event conntrack
vsr running config# vrf main cg-nat logging framing-method non-transparent-framing
vsr running config# vrf main cg-nat logging rsyslog-server 10.125.0.2
vsr running config# .. logging rsyslog-server 10.125.0.3
vsr running config# commit

Note

The remote syslog server must be configured to accept TCP connection.

When several rsyslog servers are defined, the logs are dispatched in function of the user’s ip address.

Some statistics and connection status of rsyslog-server can be retrieved with the following command:

vsr> show state vrf main cg-nat logging
logging
    enabled true
    local false
    event conntrack
    framing-method non-transparent-framing
    rsyslog-server 10.125.0.2
        port 514
        vrf main
        statistics
            transmit 0
            transmit-error 0
            build-error 0
            ..
        connection-status not-connected
        ..
    rsyslog-server 10.125.0.3
        port 514
        vrf main
        statistics
            transmit 0
            transmit-error 0
            build-error 0
            ..
        connection-status connecting
        ..
     ..
 ..

Troubleshooting

Invalid packet state statistics

To display the CG-NAT statistics, use the following command.

vsr> show cg-nat statistics
...
Invalid packet state cases:
...
   0 TCP case RST
...
   0 TCP case I
   0 TCP case II
   0 TCP case III
...

If the TCP case I, II or III statistics are incremented, you may disable TCP window checks as follows.

vsr> edit running
vsr running config# vrf main network-stack fast-path conntrack
vsr running conntrack# behavior tcp-window-check enabled false
vsr running conntrack# commit

If the TCP case RST statistic is incremented, you may disable TCP RST strict ordering as follows.

vsr> edit running
vsr running config# vrf main network-stack fast-path conntrack
vsr running conntrack# behavior tcp-rst-strict-order enabled false
vsr running conntrack# commit

Note

Disabling these features improves performance to the detriment of TCP robustness.

State/NAT/USER/Block Allocation Failures

vsr> show cg-nat statistics
...
State and NAT entries:
...
   0 state allocation failures
...
   0 NAT entry allocation failures
   0 NAT port allocation failures
CGNat entries:
...
   0 USER allocation failures
...
   0 Block allocation failures
...

If one of these statistics is incremented, it means that one of the memory pools of the Virtual Service Router is full. Memory pool usage can be dumped using the following command.

vsr> show cg-nat mempool-usage
cgnat_user_pool : 2000/10000 (20.00%)
cgnat_block_pool : 8000/80000 (10.00%)
table_pool : 0/1056 (0.00%)
conn_pool : 1056736/1056736 (100.00%)
nat_pool : 1056736/1056736 (100.00%)

Another way to dump the memory pool is to look into the state using the following command.

vsr> show state / system fast-path limits cg-nat
cg-nat
    max-nat 1056736
    max-users 10000
    max-blocks 80000
    min-block-size 8
    max-block-size 8192
    max-pools 32
    max-rules 64
    max-block-translation-ips 10
    max-port-translation-ips 10
    max-port-overloading-factor 1
    cur-rules 1
    cur-block-translation-ips 5
    cur-port-translation-ips 0
    cur-nat 1056736
    cur-users 2000
    cur-blocks 8000
    ..
vsr> show state / system fast-path limits fp-max-conntracks
fp-max-conntracks 1056736
vsr> show state / system fast-path limits fp-cur-conntracks
fp-cur-conntracks 1056736

In the example above, the connection and NAT memory pools are full. Their size must be increased as follows.

vsr running config# / system fast-path limits cg-nat max-nat 2000000
vsr running config# / system fast-path limits fp-max-conntracks 2000000
vsr running cg-nat# commit

Refer to the capability tuning section.

No IP Public errors

vsr> show cg-nat statistics
...
CGNat entries:
...
   0 No IP Public
...

If this statistic is incremented, it means there are no blocks available in any public IP. This can be checked using the following command.

vsr> show cg-nat pool-usage pool-name mypool
tcp block usage: 4095/4095 (100.0%)
udp block usage: 4095/4095 (100.0%)
icmp block usage: 4095/4095 (100.0%)
gre block usage: 4095/4095 (100.0%)

To solve this issue, add a new public IP to the pool using the following command.

vsr> edit running
vsr running config# vrf main cg-nat pool mypool
vsr running pool mypool#  address 32.96.120.0/24
vsr running pool mypool# commit

NAT port allocation failures

There are two main reasons for port allocation failures:

• A user has consumed all its port blocks. The maximum number of blocks per user can be increased in the rule using the max-blocks-per-user command.

• No blocks are available on the public IP allocated to the user. In this case, the Full IP Public statistic is also incremented.

To list users with allocation failures to understand how many users are impacted, use the following command.

vsr> show cg-nat user rule-id 1 threshold-errors 1
100.64.0.1 -> 32.96.119.108
   2/2 tcp blocks, 0/2 udp blocks, 0/2 icmp blocks, 0/2 gre blocks
   63 no port errors, 0 no block errors, 0 full public ip errors

To understand why a specific user has many connections, use the following command.

vsr> show cg-nat conntracks rule-id 1 user-address 100.64.0.1
CONN:
   state:time_wait alg:none inactive_since:47s timeout:120s
   origin: tcp 100.64.0.1:1024 -> 32.96.118.2:6001
   reply : tcp 32.96.118.2:6001 -> 32.96.119.108:1216
   NAT source: 100.64.0.1:1024 -> 32.96.119.108:1216
CONN:
   state:time_wait alg:none inactive_since:56s timeout:120s
   origin: tcp 100.64.0.1:65024 -> 32.96.118.2:6000
   reply : tcp 32.96.118.2:6000 -> 32.96.119.108:1024
   NAT source: 100.64.0.1:65024 -> 32.96.119.108:1024
...

Note

show cg-nat commands with rule-id argument cannot be used for static rule.

Maximum number of blocks reached

If the maximum number of blocks is reached, it probably means that you have not allocated enough blocks per user. You can collect some statistics to get average/percentile block and translation usage of all users with the following commands.

vsr> show cg-nat block-statistics  rule-id 1
block-usage:
   1 user (with > 1 block = 1, ratio 100.00%)
   blocks per user: min = 2, max = 2, average = 2.00
   1 user (100.00%) have 2 blocks
vsr> show cg-nat translation-statistics  rule-id 1
translation-usage:
   1 user (with > 1 translation = 1, ratio 100.00%)
   translations per user: min = 128, max = 128, average = 128.00
   1 user (100.00%) have 128 translations

You can also check the block usage and translation usage of a specific user to get more details with the following commands.

vsr> show cg-nat blocks rule-id 1 user-address 100.64.0.1
BLOCK: status active, proto 1 tports 1024 - 1031 parity=1, usage 1/8
vsr> show cg-nat translation-usage rule-id 1 user-address 100.64.0.1
tcp translation usage: 0/16
udp translation usage: 0/16
icmp translation usage: 1/16
gre translation usage: 0/16

Then, you can decide to increase the number of blocks per user or the block size. Refer to the Changing parameters section.

Note

show cg-nat commands with rule-id argument cannot be used for static rule.

Full IP Public errors

vsr> show cg-nat statistics
...
CGNat entries:
...
   0 Full IP Public
...

The paired address pooling feature ensures the assignment of the same public IP address to all sessions originating from the same internal user, as described in RFC 4787 Req 2.

It means that when a user has started to use one public IP address, all its port blocks will be allocated from this same IP. Adding a new public IP to the pool won’t solve the issue, as the user cannot allocate a block from a new public IP.

A possible way to recover such situation is to add new IP address to the pool, and then flush all the current connections of all users, as follows.

vsr running config# / vrf main cg-nat pool mypool
vsr running pool mypool#  address 32.96.120.0/24
vsr running pool mypool# commit
Configuration committed.

vsr running pool mypool# flush cg-nat user rule-id 1

Dimensioning

Different limits can be configured into fast path limits to dimension cg-nat. These limits can be adjusted to adapt to the amount of memory available in the system.

vsr running config# system fast-path limits cg-nat
max-block-size  max-block-translation-ips max-blocks max-nat max-pools
max-port-overloading-factor  max-port-translation-ips max-rules
max-users  min-block-size

These limits must properly tuned to have enough objects to load cg-nat configuration and handling network taffic.

Warning

Changing these values triggers a restart of the fast-path (hence, a service interruption). Therefore you should carefully think about your dimensioning before launching your Virtual Service Router into production.

Modifying these limits will automatically restart the fast path and interrupt packet processing. To check that the fast path is back up and running, use the following commands.

vsr running config# show state system fast-path
fast-path
   enabled stopping
   ..
vsr running config#  show state system fast-path
fast-path
   enabled starting
   ..
vsr running config# # show state system fast-path
fast-path
   enabled true
   ...

Users, blocks, nats, conntracks

The maximum numbers for nat entries, users, conntracks (sessions), blocks and block sizes are defined in the configuration. These limits can be adjusted to adapt to the amount of memory available in the system.

The following table shows a list of different limit combinations and the corresponding memory requirement. This is empirical and may have to be tuned according to your use case.

Max conntracks	Max nat entries	Max use	Max blocks	Required memory
1M	1M	10K	80K	5 GB
2M	2M	20K	80K	6 GB
4M	4M	20K	80K	8 GB
8M	8M	20K	80K	12 GB
16M	16M	20K	80K	24 GB
30M	30M	20K	80K	32 GB

Rule

The maximal number of rules can be configured into fast path limits:

vsr running vrf config# / system fast-path limits cg-nat max-rules 10

To check how many rules are loaded into fast-path:

show state  / system fast-path limits cg-nat cur-rules
cur-rules 1

Warning

If the number of rules in the CG-NAT is bigger than the fast path limits, the configuration will not be loaded. Please make sure to configure max-rules to be large enough for loading your configuration.

Pool

The maximal number of pools can be configured into fast path limits:

vsr running vrf config# / system fast-path limits cg-nat max-pools 1

To check how many pools are loaded into fast-path:

show state  / system fast-path limits cg-nat cur-pools
cur-pools 3

Warning

If the number of pools in the CG-NAT configuration is bigger than the fast path limits, the configuration will not be loaded. Please make sure to configure max-pools to be large enough for loading your configuration.

IPs pool

The maximal number of ip for a pool can also be configured into fast path limits:

vsr running vrf config# / system fast-path limits cg-nat max-block-translation-ips 280
vsr running vrf config# / system fast-path limits cg-nat max-port-translation-ips 0

The max-block-translation-ips limit is for IP objects of dynamic-block or deterministic-block pool. And the max-port-translation-ips limit is for IP objects of dynamic-port pool.

Warning

If the number of IPs in the CG-NAT configuration is bigger than the fast path limits, the configuration will not be loaded. Please make sure to configure IPs to be large enough for loading your configuration.

Block-size

The minimal and maximal number of blocks can be configured into fast path limits:

vsr running vrf config# / system fast-path limits cg-nat max-block-size 4096
vsr running vrf config# / system fast-path limits cg-nat min-block-size 16

Block objects are pre-allocated at fast path initialization. Tuning these parameters can reduced the memory needed for fast path.

Warning

If a pool is configured with a block-size lower than min-block-size or bigger than max-block-size, the block-size is automatically updated to fit within these limits.

Warning

If a pool has a port range (i.e. port max - port min + 1) lower than min-block-size, the port range is automatically updated to respect this limit.