FPN RPC¶

Overview¶

FPN RPC is a framework to define remote procedure calls to configure or monitor the fast path. The schema of the RPCs, as well as their binary encoding, are defined using Protobuf.

FPN RPC is composed of:

a library (libfpn-rpc), which implements the fpn-rpc framework, that is used on the server (fast path side), and can also be used on the client side.
a test program (fpn-rpc-test), that demonstrates how to use the library, and providing basic examples and benchmarks.
a python CLI (fpn-rpc-cli), which is a client that dynamically parses the protobuf schemas to build an interactive CLI.
a control thread on the fast path side, that acts as a RPC server.
several fast path modules that register RPC and provide callback functions to process them.

RPC definition and organization¶

A protobuf file contains the definition of services, usually for a given fast path module. The common service names are Config (for all configuration operations) and State (for statistics or dumps).

Each service contains several RPCs. A RPC is composed of a request and a reply message. These messages can reference other sub-types (enum, messages) that are declared in this file, or in another one.

Some RPCs (like dumps) may receive several reply message for one request. In this case, the rpc must be identified with the stream keyword in the schema.

Example:

file module-1.proto (defines package module_1)
- service Config
  - rpc DoSomething
    - request message DoSomethingRequest
    - reply message DoSomethingReply
  - rpc DoSomethingElse
    - request message DoSomethingElseRequest
    - reply message DoSomethingElseReply
- service State
  - rpc DumpSomething
    - request message DumpSomethingRequest
    - reply message DumpSomethingReply
file module-2.proto (defines package module_2)
- service Config
  - rpc DoSomething
    - request message DoSomethingRequest
    - reply message DoSomethingReply
(…)

All the protobuf schemas are installed on the target in /usr/share/fast-path/rpc.

Transport¶

The RPCs are transported over a UNIX SEQPACKET socket. The messages are encoded in protobuf using the fpn_rpc.Message message type, defined in fpn-rpc.proto.

This outer message carries another protobuf-encoded message, whose type is identified by the service name and the rpc name. The outer message contains additional metadata, like the message type (request, reply, error).

The messages are ordered on a connection. A client sends a fpn_rpc.Message whose type is REQUEST, and the server replies with either:

a REPLY message containing the inner reply rpc
zero to several REPLY_STREAM messages, each of them containing an inner reply rpc, followed by a REPLY_END message
an ERR_xxx message

A client should not send another request until the the reply of previous one is not fully received. To send parallel requests or do parallel monitoring, several connections must be opened.

CLI¶

fpn-rpc-cli is an interactive CLI written in Python that can be used to configure the fast path manually. The RPC data in the CLI is formatted in JSON, to make it human-readable.

Synopsis

# fpn-rpc-cli [-h] [-u PATH] [-i IGNORE_SCHEMA_LIST] [-v]

Parameters

-h, --help¶: Show help.

-u PATH, --unix-path PATH¶: Path to fpn-rpc unix server. Default: /run/fast-path/fpn-rpc.sock.

-i IGNORE_SCHEMA_LIST, --ignore-schema-list IGNORE_SCHEMA_LIST¶: Comma-separated list of schemas files (without .proto extension) to be ignored by cli. Default: fpn-rpc-test.

-c CMD, --command CMD¶: Execute the given command instead of running interactively.

-v, --verbose¶: Display debug information.

Example

Example of an interactive CLI session.

 root@dut-vm:~# fpn-rpc-cli
 Welcome to fpn-rpc-cli. Type help or ? to list commands.

 fpn-rpc> ?

 Documented commands (type help <topic>):
 ========================================
 help  list_rpcs  list_services  rpc  show_rpc

 fpn-rpc> help list_services

         List registered services.
         list_services [detail]

 fpn-rpc> list_services
 fp_if.Config
 fp_gtp.Config
 fp_gtp.State
 fp_ip.Config
 fpn-rpc> help list_rpcs

         List registered rpcs.
         list_rpcs <service> [detail]

 fpn-rpc> list_rpcs fp_ip.Config
 AddRoute
 DelRoute
 fpn-rpc> list_rpcs fp_ip.Config detail
 AddRoute
   request: AddRouteRequest
   reply: AddRouteReply
 DelRoute
   request: DelRouteRequest
   reply: DelRouteReply
 fpn-rpc> help show_rpc

         Show the schemas of RPC request and reply
         show_rpc <service> <rpc>

 fpn-rpc> show_rpc fp_ip.Config AddRoute
 === request: AddRouteRequest
   uint32 vrfid
   string prefix
   string nexthop
   string ifname
   uint32 metric
 === reply: AddRouteReply
   enum result
     ADD_RT_OK
     ADD_RT_ERR_EXIST
     ADD_RT_ERR_INVALID_PARAMS
     ADD_RT_ERR_INTERNAL
   string err_info
 fpn-rpc> help rpc

         Send a JSON rpc request to the fast path.
         rpc <service> <rpc> <json>

fpn-rpc> rpc fp_ip.Config AddRoute {"vrfid": 0, "prefix": "10.100.0.0/16", "nexthop": "10.200.0.1" }
{ "result": "ADD_RT_OK" }

Test¶

Without arguments, fpn-rpc-test does unit testing and benchmarks.

Synopsis

# fpn-rpc-test [<options>]

Parameters

-h, --help¶: Show help.

-t, --test¶: Run pack/unpack tests (default).

-c, --simple-client¶: Run as simple client, using libfpn-rpc.

-a, --standalone-client¶: Run as standalone client, without using libfpn-rpc.

-e, --event-client¶: Run as event-driven client for benchmark.

-d, --dump-client¶: Run as simple dump client.

-m, --monitor-client¶: Run as simple monitoring client.

-s, --server¶: Run as server.

-u <path>, --unix-path <path>¶: Path to the unix socket (default: /tmp/fpn-rpc-test).

-v, --verbose¶: Dump debug information.

Example

Start a server in one terminal:

# fpn-rpc-test -s
rx: 0 msgs   tx: 0 msgs
rx: 0 msgs   tx: 0 msgs
rx: 0 msgs   tx: 0 msgs
(...)

Start a simple client in another terminal:

# fpn-rpc-test -c
result: ADD_OK

Example in C¶

The following code demonstrate how to send a rpc, without using libfpn-rpc. It requires to compile and link the user protobuf schemas as well as fpn-rpc.proto, which is needed for rpc transport.

/*
 * Copyright 2021 6WIND S.A.
 */

#include <errno.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/socket.h>
#include <sys/stat.h>
#include <sys/un.h>
#include <unistd.h>

#include "client-standalone.h"
#include "fpn-rpc-test.pb-c.h"
#include "fpn-rpc.pb-c.h"

/* connect to a unix server */
static int unix_connect(const char *path)
{
	struct sockaddr_un addr;
	int s = -1;
	int ret;

	s = socket(AF_UNIX, SOCK_SEQPACKET, 0);
	if (s < 0) {
		fprintf(stderr, "socket() failed: %s\n", strerror(errno));
		goto fail;
	}

	memset(&addr, 0, sizeof(addr));
	addr.sun_family = AF_UNIX;
	ret = snprintf(addr.sun_path, sizeof(addr.sun_path), "%s", path);
	if (ret < 0 || (size_t)ret >= sizeof(addr.sun_path)) {
		fprintf(stderr, "unix path too long: %s\n", path);
		errno = ENAMETOOLONG;
		goto fail;
	}

	if (connect(s, (struct sockaddr *)&addr, sizeof(addr)) < 0) {
		fprintf(stderr, "connect() failed: %s\n", strerror(errno));
		goto fail;
	}

	return s;

fail:
	if (s >= 0)
		close(s);
	return -1;
}

static int send_add_route4_req(int s)
{
	FpnRpcTest__AddRoute4Request req = FPN_RPC_TEST__ADD_ROUTE4_REQUEST__INIT;
	FpnRpc__Message outer = FPN_RPC__MESSAGE__INIT;
	unsigned char inner_buf[65536];
	unsigned char outer_buf[65536];
	size_t inner_size;
	size_t outer_size;
	int ret;

	/* the inner rpc, containing the user message */
	req.vrfid = 0;
	req.prefix = "10.0.0.0/8";
	req.nexthop = "1.2.3.4";

	inner_size = fpn_rpc_test__add_route4_request__get_packed_size(&req);
	if (inner_size > sizeof(inner_buf)) {
		fprintf(stderr, "failed to pack inner rpc: message too big\n");
		return -1;
	}
	fpn_rpc_test__add_route4_request__pack(&req, inner_buf);

	/* the type of the outer message is always fpn_rpc.Message */
	outer.data.len = inner_size;
	outer.data.data = (unsigned char *)inner_buf;
	outer.msgtype = FPN_RPC__MSG_TYPE__REQUEST;
	outer.service_name = (char *)"fpn_rpc_test.Config";
	outer.rpc_name = (char *)"AddRoute4";

	outer_size = fpn_rpc__message__get_packed_size(&outer);
	if (outer_size > sizeof(outer_buf)) {
		fprintf(stderr, "failed to pack outer rpc: message too big\n");
		return -1;
	}
	fpn_rpc__message__pack(&outer, outer_buf);

	/* send it on the unix socket */
	ret = write(s, outer_buf, outer_size);
	if (ret < 0) {
		fprintf(stderr, "failed to write to unix socket: %s\n", strerror(errno));
		return -1;
	}
	if ((size_t)ret != outer_size) {
		fprintf(stderr, "unexpected write() return value: %d instead of %zd\n",
			ret, outer_size);
		return -1;
	}

	return 0;
}

static int recv_add_route4_rpl(int s)
{
	const ProtobufCEnumValue *result_val;
	FpnRpcTest__AddRoute4Reply *rpl = NULL;
	FpnRpc__Message *outer = NULL;
	unsigned char outer_buf[65536];
	int ret;

	ret = read(s, outer_buf, sizeof(outer_buf));
	if (ret < 0) {
		fprintf(stderr, "failed to read from unix socket: %s\n", strerror(errno));
		goto fail;
	}
	if (ret == 0) {
		fprintf(stderr, "failed to read from unix socket: short read\n");
		goto fail;
	}

	outer = fpn_rpc__message__unpack(NULL, ret, outer_buf);
	if (outer == NULL) {
		fprintf(stderr, "failed to unpack outer rx message\n");
		goto fail;
	}
	if (protobuf_c_message_check(&outer->base) == false) {
		fprintf(stderr, "failed to validate outer rx message\n");
		goto fail;
	}

	if (outer->msgtype != FPN_RPC__MSG_TYPE__REPLY) {
		fprintf(stderr, "invalid message type\n");
		goto fail;
	}
	if (outer->service_name == NULL || strcmp(outer->service_name, "fpn_rpc_test.Config")) {
		fprintf(stderr, "invalid service name\n");
		goto fail;
	}
	if (outer->rpc_name == NULL || strcmp(outer->rpc_name, "AddRoute4")) {
		fprintf(stderr, "invalid rpc name\n");
		goto fail;
	}

	rpl = fpn_rpc_test__add_route4_reply__unpack(NULL, outer->data.len, outer->data.data);
	if (rpl == NULL) {
		fprintf(stderr, "failed to unpack inner rx message\n");
		goto fail;
	}
	if (protobuf_c_message_check(&rpl->base) == false) {
		fprintf(stderr, "failed to validate inner rx message");
		goto fail;
	}

	result_val = protobuf_c_enum_descriptor_get_value(
		&fpn_rpc_test__add_route_reply_result__descriptor, rpl->result);
	printf("result: %s\n", result_val ? result_val->name : NULL);

	fpn_rpc_test__add_route4_reply__free_unpacked(rpl, NULL);
	fpn_rpc__message__free_unpacked(outer, NULL);
	return 0;

fail:
	fpn_rpc_test__add_route4_reply__free_unpacked(rpl, NULL);
	fpn_rpc__message__free_unpacked(outer, NULL);
	return -1;
}

int client_standalone(const char *unix_path)
{
	int ret = -1;
	int s;

	s = unix_connect(unix_path);
	if (s < 0)
		goto out;

	if (send_add_route4_req(s) < 0)
		goto out;

	if (recv_add_route4_rpl(s) < 0)
		goto out;

	ret = 0;

out:
	if (s >= 0)
		close(s);

	return ret;
}