Usage¶

Configuration
Control tool
- option
- table
- pool-usage
- htable-stats
- stats
Runtime options for Fast Path CG-Firewall

Configuration ¶

The general system network configuration should already be done, e.g. assigning IP addresses and bringing up the interfaces.

Then a Fast Path CG-Firewall configuration file must be created, which will be loaded using fp-npfctl.

This section describes how to write a configuration file.

Example ¶

The following example gives a few configuration elements. It contains two groups for two network interfaces and a default group:

iface = {eth0, eth1}

$ext_if = { inet4(eth0) }
$int_if = { inet4(eth1) }

$services_tcp = { ftp, http, https, smtp, domain, 6000 }
$services_udp = { domain, ntp, 6000 }
$localnet = { 10.1.1.0/24 }
$server = { 10.1.1.3 }

$pub_ip1 = 203.0.113.10
$pub_ip2 = 203.0.113.11

alg "ftp"

map $ext_if static $server <-> $pub_ip2
map $ext_if dynamic $localnet -> $pub_ip1

group "external" on $ext_if {
        pass stateful out final all

        pass stateful in final proto tcp to $pub_ip2 port $services_tcp
        pass stateful in final proto udp to $pub_ip2 port $services_udp
}

group "internal" on $int_if {
        block in all

        # Ingress filtering as per BCP 38 / RFC 2827.
        pass in final from $localnet
        pass out final all
}

group default {
        block all
}

When a configuration is loaded and Fast Path CG-Firewall is enabled, it will by default process all packets from any interfaces in reception and emission. For performance reasons, it can be useful to limit Fast Path CG-Firewall to specific interfaces. The active interface list can be provided with the following syntax:

# single interface
iface = interface

# or as a set:
iface = { interface, ... }

Example:

iface = { eth0, eth1 }

Variables ¶

Variables are general purpose keywords which can be used in the ruleset to make it more flexible and easier to manage. Most commonly, variables are used to define one of the following: IP addresses, networks, ports or interfaces. A variable can contain multiple elements.

A variable is defined and referenced using the dollar ($) sign. Variables are defined by assigning a value to them:

$var = 10.0.0.1
$list = { 10.0.0.1, 10.0.0.2 }

$str = "string"
$num = 42
$port_range = 1-1024
$ip_addr = 10.0.0.1
$net = 10.0.0.0/24

# The ipv4 addresses of eth0
$ext_net4 = inet4(eth0)

# The ipv6 addresses of eth0
$ext_net6 = inet6(eth0)

Interfaces ¶

An interface can be directly referenced by its name. A variable can also be used instead of repeating references to said interface.

In some contexts, the addresses of an interface should instead be used. In this case, functions can be used on the interface name to extract a list.

The inet4(interface) function will return IPv4 addresses, while inet6(interface) will return IPv6 addresses. Such list will be read on configuration load.

It is legal to pass an extracted list from an interface to a context expecting an interface. In this case, the parser will ignore the function generating the list and infer a direct reference to the interface.

Tables ¶

It is often useful to remove or add IP addresses for a particular rule or set of rules. Reloading the configuration however is a resource intensive operation and is not suitable for stream of constant changes. It is also inefficient to have many rules with the same logic just for different IP addresses.

As a solution, Fast Path CG-Firewall provides a high-throughput container of IP addresses that can be used as stand-in where those would be normally used. Tables are dynamic containers designed for large IP sets and frequent updates, without reloading the entire ruleset. Inserting or removing entries can be done dynamically and the base set of entries can be loaded from a separate file upon ruleset reload.

Specifying ranges of addresses is supported by using a netmask. Here is an example of table configuration:

table <blacklist> file "/etc/ip_blacklist"
table <permitted> order 11 dynamic

group "external" on $ext_if {
    block in final from <blacklist>
    pass stateful final from <permitted>
}

The static table identified as blacklist is loaded from the file /etc/ip_blacklist. The dynamic table is initialized empty. Both table can then be filled or emptied using fp-npfctl:

# fp-npfctl table blacklist flush
# fp-npfctl table permitted add 10.0.2.1
# fp-npfctl table permitted add 10.0.2.0/24
# fp-npfctl table permitted rem 10.0.2.1

The insertion of 10.0.2.0/24 within permitted is allowed as it supersedes the existing entry 10.0.2.1. When removing 10.0.2.1, the entry 10.0.2.0/24 remains untouched.

All tables share the same entry allocator, which will allocate at most the number passed as a parameter to the --max-table-entries runtime option.

the order of the table is the power of two used to determine the size of the table hash and its number of buckets. If it is too low, the table can become bloated and performance could suffer. In this case the table should be resized by reloading the ruleset, which can be a disruptive operation.

Bucket statistics are available with the stats table command in fp-npfctl:

# fp-npfctl table blacklist stats
blacklist: 1024 buckets (non empty = 396, ratio 38.67%)
        entries per bucket: min = 1, max = 4, average = 1.30
          297 buckets (75.00%) have 1 entry
           83 buckets (20.96%) have 2 entries
           14 buckets (3.54%) have 3 entries
            2 buckets (0.51%) have 4 entries

Synopsis

table <tid> [order <u8>] <dynamic|file "/absolute/path">

...

group "anygroup" {
    pass to/from <tid>
}

Where tid is the table identifier. It is then used to make reference to addresses within the set in subsequent rules, or as identifier when inserting or removing rules using fp-npfctl.

Groups ¶

Having one huge ruleset for all interfaces or directions would be inefficient; therefore, the Fast Path CG-Firewall requires that all rules be defined within groups. A group can be understood as a higher level rule containing subrules.

The main properties of a group are its interface and traffic direction.

Packets matching a group criteria are passed to the ruleset of that group. If a packet does not match any group, it is passed to the default group, which must always be defined.

Synopsis

group "<name>" [in|out] [on <iface>] {
    # List of rules
}

Where name and iface are identifiers.

Note: the default group name is written without double-quotes (").

Rules ¶

A rule statement describes an action that the filter should apply on all matched packets. It is composed of an action (block or pass) and a match statement.

All rules within a group are matched against a packet in the order they are declared. The action of the last matching rule is applied. If a rule has the final option set, it is immediately applied and all subsequent rules are skipped.

A rule can be set stateful, which means that a state of the matched flow of packets will be tracked and verified.

A rule match is constituted of direction (in or out), interface and either a 5-tuple or pcap filter. The pcap filter uses the standard pcap syntax. A 5-tuple filter describes source and destination IP addresses, the layer 4 protocol and layer 4 port (or port range), each element being optional.

Proto¶

The proto keyword can be used to match packets by layer 4 protocol (TCP, UDP, ICMP or other). Its parameter should be a protocol number or its symbolic name. This keyword can additionally have protocol-specific options, such as flags.

symbolic name	number	name
ip	0	IP
icmp	1	ICMP
igmp	2	IGMP
tcp	6	TCP
udp	17	UDP
raw	255	RAW # RAW IP interface

proto tcp flags match[/mask]

Where match is the set of TCP flags to be matched out of the mask, both sets being represented as a string combination of S (SYN), A (ACK), F (FIN), R (RST). The flags that are not present in mask are ignored:

proto tcp flags S/SAFR

This filter would match only TCP segments which would have only the syn flag set among syn, ack, fin, and rst. A segment with syn and ecn would match this filter, while a segment with syn and ack would not.

Return¶

To notify a sender of a blocking decision, three return options can be used in conjunction with a block rule.

return-rst: Return a TCP rst message, when the packet being blocked is a TCP packet. Applies to IPv4 and IPv6.
return-icmp: Return an ICMP Unreachable message. Applies to IPv4 and IPv6.
return: Behaves like return-rst with TCP packets and return-icmp with all other packets.

Stateful¶

Stateful packet inspection is enabled using the stateful keywords. It creates a state which is uniquely identified by a 5-tuple (source and destination IP addresses, port numbers, proto).

In both cases, a full TCP state tracking is performed for TCP connections and a limited tracking for message-based protocols (UDP and ICMP).

By default, a stateful rule implies SYN-only flag check (flags S/SAFR) for the TCP packets. It is not advisable to change this behavior; however, it can be overridden with the aforementioned flags keyword.

Packets in the backward direction, after having been confirmed to belong to the same connection, are passed without ruleset inspection. An example configuration fragment with a stateful rule:

group "external" on $ext_if {
    block all
    pass stateful in final proto tcp flag S/SA to $server port ssh
}

In this example, all incoming and outgoing traffic on the $ext_if interface will be blocked, with the exception of inbound SSH traffic (with the destination being the IP address of the server) and implicitly passed backward stream (outgoing reply packets) of the SSH connections. Since initial TCP packets opening a connection are SYN packets, such rules often have additional TCP filtering criterion. The expression flags S/SA extracts SYN and ACK flags and checks that SYN is set and ACK is not.

Synopsis

As a quick reference for the grammar, the following is a relaxed BNF-like description of a rule:

rule            = ( "block" [ block-opts ] | "pass" )
                  [ "stateful" ]
                  [ "in" | "out" ] [ "final" ] [ "on" interface ]
                  ( 5tuple-filter | "pcap-filter" pcap-filter-expr )

block-opts      = "return-rst" | "return-icmp" | "return"

5tuple-filter   = [ "family" family-opt ] [ proto ] ( "all" | filt-opts )

family-opt      = "inet4" | "inet6"

proto           = "proto" protocol [ proto-opts ]
proto-opts      = "flags" tcp-flags [ "/" tcp-flag-mask ] |
                  "icmp-type" type [ "code" icmp-code ]

tcp-flag-mask   = tcp-flags
tcp-flags       = [ "S" ] [ "A" ] [ "F" ] [ "R" ]

filt-opts       = "from" filt-addr [ port-opts ] "to" filt-addr
                  [ port-opts ]

port-opts       = "port" ( port-num | port-from "-" port-to | var-name )

filt-addr       = [ "!" ] [ interface | addr-mask | "any" ]
addr-mask       = addr [ "/" mask ]

A “fully-featured” rule could be:

pass stateful in final on eth0 family inet4 \
  proto tcp flags S/SA from $source port $sport to $dest port $dport

block out final pcap-filter "tcp and dst 10.1.1.252"

Network Address Translation ¶

Fast Path CG-Firewall supports various forms of network address translation (NAT). The translation may be dynamic (stateful) or static (stateless). The following mappings are available:

->: for outbound NAT (translation of the source).
<-: for inbound NAT (translation of the destination).
<->: bi-directional NAT (combination of inbound and outbound NAT).

NAT rules are expressed in a form of segment mapping:

map   = "map" interface
        ( "static" [ "algo" algorithm ] | "dynamic" ) [proto]
        net-seg ( "->" | "<->" | "<-" ) net-seg
        [ "port-algo" ( "parity" | "random" ) ]
        [ "endpoint-mapping" ( "dependent" | "independent" ) ]
        [ "endpoint-filtering" ( "dependent" | "independent" ) ]
        [ "pass" [proto] filt-opts ]

Translations are implicitly filtered by limiting the operation to the network segments specified. Considering the following example:

map $ext_if dynamic $localnet -> $ext_if

group "external" on $ext_if {
   pass stateful out final proto tcp from $localnet
}

map $ext_if dynamic $localnet -> $ext_if

Translation would be performed only on packets from the network defined in $localnet to any other network (0.0.0.0/0), where the translated address is the (only) one on the interface $ext_if. It is also possible to specify the translated address directly, which is necessary if the interface has more than one IP address.

The rule pass … permits all outgoing packets from the specified network. It additionally has stateful tracking enabled with the stateful keyword. Therefore, any incoming packets belonging to the connections which were created by initial outgoing packets will be implicitly passed.

The following two lines are example fragments of bi-directional NAT and port 8080 forwarding to a local IP address, port 80:

map $ext_alt_if dynamic $local_host_1 <-> $ext_alt_if
map $ext_if dynamic $local_host_2 port 80 <- $ext_if port 8080

In the examples above, NPF determines the filter criteria from the segments on the left and right hand side implicitly. Filter criteria can be specified explicitly using an optional pass … syntax in conjunction with map. In such case the criteria has to be full, i.e. for both the source and the destination.

The port-algo keyword allows choosing between two methods to allocate NAT ports:

parity: this algo preserves the parity of the port, i.e. an even port will be mapped to an even port and an odd port will be mapped to an odd port.
random: This algo chooses a port randomly.

The dynamic NAT implies network address and port translation (NAPT). The L4 protocols supported are TCP, UDP, and ICMP. The port translation can be controlled explicitly. For example, the following provides “port forwarding”, redirecting the public port 9022 to the port 22 of an internal host:

map $ext_if dynamic proto tcp 10.1.1.2 port 22 <- $ext_if port 9022

The NAT can have different address translation algorithms, chosen using the algo keyword. The currently available algorithms are:

nat64: IPv4-to-IPv6 address translation, using RFC 6052 for address conversion.
npt66: IPv6-to-IPv6 network prefix translation (NPTv6: RFC 6296). This algorithm only works in static NAT.

If no algorithm is specified, then IPv4 addresses are mapped 1:1.

In dynamic source NAT, it is possible to configure endpoint mapping and filtering behaviour. The endpoint-mapping can be set:

dependent: the NAT reuses the port mapping for subsequent packets sent from the same internal IP address and port to the same external IP address and port.
independent: the NAT reuses the port mapping for subsequent packets sent from the same internal IP address and port to any external IP address and port.

The endpoint-filtering can be set:

dependent: inbound packets from external endpoints are filtered-out if they do not match an existing mapping.
independent: inbound packets from external endpoints are only filtered-out if their destination IP address and port do not match an existing NAT external IP address and port mapping.

Application Level Gateways parameter ¶

Each supported ALG can be enabled as follows:

alg "ftp"
alg "h323_ras"
alg "h323_q931"
alg "pptp"
alg "rtsp"
alg "sip_udp"
alg "sip_tcp"
alg "tftp"
alg "dns_udp"

dns_udp¶

This ALG modifies the connection timeout of DNS packets over UDP, in order to delete the conntrack as soon as the query is replied. This prevents to keep these conntracks until they timeout, consuming resources.

Control tool ¶

Fast Path CG-Firewall can be controlled through the fp-npfctl application.

Usage:

# fp-npfctl
Usage:  fp-npfctl [vrf-exec ID] start | stop | flush | show
        fp-npfctl [vrf-exec ID] validate | reload [<rule-file>]
        fp-npfctl [vrf-exec ID] option { show | reset | set }
        fp-npfctl [vrf-exec ID] table { <tid> | all } { add | rem | test } <address/mask>
        fp-npfctl [vrf-exec ID] table { <tid> | all } { list | stats | flush }
        fp-npfctl pool-usage
        fp-npfctl htable-stats
        fp-npfctl stats { show | reset } [json]

The ‘start’ and ‘stop’ commands do not actually change (i.e. load or unload) the active configuration. Running ‘start’ will only enable the passing of packets through Fast Path CG-Firewall, while ‘stop’ will disable such passing.

Therefore, configuration should first be loaded using the ‘reload’ command and then nating enabled with ‘start’.

Similarly, clearing of the active configuration is done by performing the ‘stop’ and ‘flush’ commands.

Such behaviour allows users to efficiently disable and enable filtering without actually changing the active configuration, as it may be unnecessary.

option ¶

The “option” command displays the different parameters of Fast Path CG-Firewall.

# fp-npfctl option show
nat64_update_tcp_mss = 1
nat64_force_frag4 = 0
nat64_force_frag6 = 0
nat64_drop_udp4_zero_checksum = 1
nat64_lowest_ipv6_mtu = 0
icmp_connection_timeout_closed = 0
icmp_connection_timeout_new = 30
icmp_connection_timeout_established = 60
udp_connection_timeout_closed = 0
udp_connection_timeout_new = 30
udp_connection_timeout_established = 120
gre_pptp_connection_timeout_closed = 0
gre_pptp_connection_timeout_new = 600
gre_pptp_connection_timeout_established = 18000
tcp_connection_timeout_syn_sent = 30
tcp_connection_timeout_simsyn_sent = 30
tcp_connection_timeout_syn_received = 60
tcp_connection_timeout_established = 7440
tcp_connection_timeout_fin_sent = 120
tcp_connection_timeout_fin_received = 120
tcp_connection_timeout_close_wait = 60
tcp_connection_timeout_fin_wait = 120
tcp_connection_timeout_last_ack = 30
tcp_connection_timeout_time_wait = 120
tcp_connection_timeout_closed = 10
tcp_check_window = 1
tcp_strict_order_rst = 1

The “option” command can also be used to change the value of a parameter.

# fp-npfctl option set tcp_connection_timeout_last_ack 1

The “option” command can also be used to reset the value of a parameter to the default value.

# fp-npfctl option reset tcp_connection_timeout_last_ack

table ¶

The “table” command manages the existing tables. The tables must have been created within the ruleset previously loaded.

It is possible to add, remove and match an address within a table, list all existing addresses within the set, show statistics and flush completely a table.

Either the table id (tid) must be given, or all can be used to target all existing tables.

Example:

# fp-npfctl table blacklist stats
npf_table_blacklist:
        1024 buckets (with > 1 entry = 396, ratio 38.67%)
        entries per bucket: min = 1, max = 4, average = 1.30
          297 buckets (75.00%) have 1 entry
           83 buckets (20.96%) have 2 entries
           14 buckets (3.54%) have 3 entries
            2 buckets (0.51%) have 4 entries
# fp-npfctl table blacklist add 10.0.1.0/24
fp-npfctl: success
# fp-npfctl table blacklist stats
npf_table_blacklist:
        1024 buckets (with > 1 entry = 397, ratio 38.77%)
        entries per bucket: min = 1, max = 4, average = 1.29
          298 buckets (75.06%) have 1 entry
           83 buckets (20.91%) have 2 entries
           14 buckets (3.53%) have 3 entries
            2 buckets (0.50%) have 4 entries
# fp-npfctl table blacklist test 10.0.1.0/24
fp-npfctl: matching entry found
# fp-npfctl table blacklist test 10.0.1.1
fp-npfctl: matching entry found
# fp-npfctl table blacklist rem 10.0.1.1
fp-npfctl: no matching entry
# fp-npfctl table blacklist rem 10.0.1.1/24
fp-npfctl: success
# fp-npfctl table blacklist flush
fp-npfctl: success
# fp-npfctl table blacklist stats
npf_table_blacklist:
        1024 buckets (with > 1 entry = 0, ratio 0.00%)
        entries per bucket: min = 0, max = 0, average = 0.00

pool-usage ¶

The “pool-usage” command displays the percentage of entries used in each memory pools used to allocate the different Fast Path CG-Firewall objects.

Example:

# fp-npfctl pool-usage
npf_table_pool : 0/1056 (0.00%)
npf_conn_pool : 1280499/30009160 (4.27%)
npf_nat_pool : 1280499/30009160 (4.27%)

htable-stats ¶

The “htable-stats” command displays the average number of elements per hash table entry. It does not take into account empty entries.

Example:

# fp-npfctl htable-usage
npf_expect_conndb:
        33554432 buckets (with > 1 entry = 0, ratio 0.00%)
        elements per list: min = 0, max = 0, average = 0.00
npf_conndb:
        33554432 buckets (with > 1 entry = 8626383, ratio 25.71%)
        entries per bucket: min = 1, max = 7, average = 1.16
        7409667 buckets (85.90%) have 1 entry
        1099909 buckets (12.75%) have 2 entries
         108236 buckets (1.25%) have 3 entries
           8128 buckets (0.09%) have 4 entries
            475 buckets (0.01%) have 5 entries
             27 buckets (0.00%) have 6 entries
              2 buckets (0.00%) have 7 entries
npf_ext_nat_htable:
        33554432 buckets (with > 1 entry = 2938885, ratio 8.76%)
        entries per bucket: min = 1, max = 4, average = 1.05
        2806136 buckets (95.48%) have 1 entry
         128763 buckets (4.38%) have 2 entries
           3972 buckets (0.14%) have 3 entries
             66 buckets (0.00%) have 4 entries
npf_int_nat_htable:
        33554432 buckets (with > 1 entry = 2930995, ratio 8.74%)
        entries per bucket: min = 1, max = 5, average = 1.05
        2790907 buckets (95.22%) have 1 entries
         135396 buckets (4.62%) have 2 entries
           4532 buckets (0.15%) have 3 entries
            117 buckets (0.00%) have 4 entries
              4 buckets (0.00%) have 5 entries

stats ¶

The “stats” command display the global statistics of Fast Path CG-Firewall:

# fp-npfctl stats show
Packets passed:
       0 default pass
       255 ruleset pass
       1689 state pass
Packets blocked:
       0 default block
       0 ruleset block
State and NAT entries:
       255 state allocations
       0 state destructions
       255 NAT entry allocations
       0 NAT entry destructions
       0 NAT port allocation failures
NAT64 Stats:
       0 udp null checksum packet drops
Network buffers:
       0 non-contiguous cases
       0 contig alloc failures
Invalid packet state cases:
       0 cases in total
       0 TCP case RST
       0 TCP case I
       0 TCP case II
       0 TCP case III
Packet race cases:
       0 NAT association race
       0 duplicate state race
Other:
       0 unexpected errors

The “stats” can also be displayed in json format:

# fp-npfctl stats show json
{
  "Packets passed": {
    "default pass": 0,
    "ruleset pass": 255,
    "state pass": 1785
  },
  "Packet race cases": {
    "duplicate state race": 0,
    "NAT association race": 0
  },
  "Packets blocked": {
    "default block": 0,
    "ruleset block": 0
  },
  "State and NAT entries": {
    "NAT port allocation failures": 0,
    "state allocations": 255,
    "state destructions": 0,
    "NAT entry allocations": 255,
    "NAT entry destructions": 0
  },
  "Network buffers": {
    "non-contiguous cases": 0,
    "contig alloc failures": 0
  },
  "Other": {
    "unexpected errors": 0
  },
  "Invalid packet state cases": {
    "cases in total": 0,
    "TCP case III": 0,
    "TCP case II": 0,
    "TCP case RST": 0,
    "TCP case I": 0
  }
}

It is also possible to reset the stats:

# fp-npfctl stats reset

Runtime options for Fast Path CG-Firewall ¶

Runtime options will configure how much memory is reserved for the Fast Path CG-Firewall to function. These options will have an impact on performance at the expense of memory. They can be configured in the fast path configuration file.

Maximum number of tracked connections supported --mod-opt=fp-npf:--max-conntrack=<conntrack-number>.

Default is 1024.
Size of the conntrack hash table --mod-opt=fp-npf:--conntrack-hash-table-size=<htable-size>.

Default is the value of max-conntrack.
Size of the expect conntrack hash table --mod-opt=fp-npf:--expect-conntrack-hash-table-size=<htable-size>.

Default is the value of max-conntrack.
Maximum number of NAT translations supported --mod-opt=fp-npf:--max-nat=<nat-number>.

Default is 1024.
Size of the external endpoint NAT hash table --mod-opt=fp-npf:--ext-endpoint-nat-hash-table-size=<htable-size>.

Default is the value of max-nat.
Size of the internal endpoint NAT hash table --mod-opt=fp-npf:--int-endpoint-nat-hash-table-size=<htable-size>.

Default is the value of max-nat.
Maximum number of table entries --mod-opt=fp-npf:--max-table-entries=<table-entries-number>.

Default is 1024.