Introduction

Rattan is a fast and extensible Internet path emulator framework.

We provide a simple and easy-to-use API to create and manage network emulations. Rattan is designed to be used in a wide range of scenarios, from testing network applications to debugging complex network performance issues.

Our modular design makes it easy to extend Rattan with different network effects or conditions. We provide a set of built-in modules that can be used to emulate different network conditions, such as bandwidth, latency, packet loss, ISP policies and etc.

We support Linux only at the moment. Currently, kernel version v5.4, v5.15, v6.8 and v6.10 are tested.

Design Targets

High Performance. Rattan provides both high peak performance and execution efficiency.
Flexible. Rattan tries to be agnostic of the underlying path emulation model.
Extensible. Rattan provides rich features out-of-the-box, meanwhile tends to be easily extensible for custom conditions.
User-Friendly. Rattan aims to provide a simple and intuitive interface for quick usage on common cases and ensure complete controllability under the hood to cover the corner as well.

Basic Concepts

Run rattan will generate some network namespaces and veth pairs to emulate the network environment. The network topology is like:

   ns-left                                                           ns-right
+-----------+ [Internet]                               [Internet] +-----------+
|    vL0    |    |                 ns-rattan                 |    |    vR0    |
|  external | <--+          +---------------------+          +--> |  external |
|   vL1-L   |               |  vL1-R  [P]   vR1-L |               |   vR1-R   |
| .1.1.x/32 | <-----------> |.1.1.2/32   .2.1.2/32| <-----------> | .2.1.x/32 |
|   vL2-L   |               |  vL2-R        vR2-L |               |   vR2-R   |
| .1.2.y/32 | <-----------> |.1.2.2/32   .2.2.2/32| <-----------> | .2.2.y/32 |
~    ...    ~   Veth pairs  ~  ...           ...  ~   Veth pairs  ~    ...    ~
+-----------+               +---------------------+               +-----------+

Then Rattan will build and link the cells according to the configuration file. Each cell emulates some network characteristics, such as bandwidth, delay, loss, etc.

ATTENTION: Check firewall settings before running Rattan CLI. Please make sure you allow the following addresses:

10.1.1.0/24
10.2.1.0/24

For example, you can run the following commands if using ufw:

ufw allow from 10.1.1.0/24
ufw allow from 10.2.1.0/24

Contributing

Rattan is free and open source. You can find the source code on GitHub and issues and feature requests can be posted on the GitHub issue tracker. Rattan relies on the community to fix bugs and add features: if you'd like to contribute, please read the CONTRIBUTING guide and consider opening a pull request.

Installation

The Rattan project contains a CLI tools (named rattan) for common use cases and a Rust library (named rattan-core) for you to build your own tools on top of.

There are multiple ways to install the Rattan CLI tool. Choose any one of the methods below that best suit your needs.

ATTENTION: As we are still in the early stages of development without a released version, we only support building from source currently.

Pre-compiled binaries

Executable binaries are available for download on the GitHub Releases page. Download the binary and extract the archive. The archive contains an rattan executable which you can run to start your distributed platform.

To make it easier to run, put the path to the binary into your PATH or install it in a directory that is already in your PATH. Note that, you have to grant the binary the necessary capabilities to run. For example, you can run the following commands on Linux:

# install the binary
sudo install -m 755 rattan /usr/local/bin/rattan
# grant the necessary capabilities
sudo setcap 'cap_dac_override,cap_dac_read_search,cap_sys_ptrace,cap_net_admin,cap_sys_admin,cap_net_raw+ep' /usr/local/bin/rattan

Besides, for certain operating systems that enable systemd-networkd, you have to configure it to not change MAC address of veth interfaces. You can do this by adding the following lines to /lib/systemd/network/80-rattan.link:

[Match]
OriginalName=ns*-v*-*
Driver=veth

[Link]
MACAddressPolicy=none

And then restart the systemd-networkd service:

sudo systemctl daemon-reload
sudo systemctl restart systemd-networkd.service

We also contain a install script scripts/install.sh in the repository, which you can use to install the binary on Ubuntu:

./scripts/install.sh rattan

Build from source using Rust

Dependencies

We recommend developers to install the following dependencies for better testing and development experience. And you have to install some of the dependencies to build rattan with specific features:

sudo apt install ethtool, iputils-ping, iperf3, pkg-config, m4, clang, llvm, libelf-dev, libpcap-dev, gcc-multilib

Installing with Cargo

To build the rattan executable from source, you will first need to install Rust and Cargo. Follow the instructions on the Rust installation page.

Once you have installed Rust, the following command can be used to build and install rattan:

cargo install rattan

This will automatically download rattan from crates.io, build it, and install it in Cargo's global binary directory (~/.cargo/bin/ by default).

You can run cargo install rattan again whenever you want to update to a new version. That command will check if there is a newer version, and re-install rattan if a newer version is found.

To uninstall, run the command cargo uninstall rattan.

Installing the latest git version with Cargo

The version published to crates.io will ever so slightly be behind the version hosted on GitHub. If you need the latest version you can build the git version of rattan yourself. Cargo makes this super easy!

cargo install --git https://github.com/stack-rs/rattan.git rattan

Again, make sure to add the Cargo bin directory to your PATH.

Building from source

If you want to build the binary from source, you can clone the repository and build it using Cargo.

git clone https://github.com/stack-rs/rattan.git
cd rattan
cargo build --release

Then you can find the binary in target/release/rattan and install or run it as you like (e.g., run ./scripts/install.sh target/release/rattan).

Modifying and contributing

If you are interested in making modifications to Rattan itself, check out the Contributing Guide for more information.

Bundled CLI Tool

The Rattan project provides a CLI tool named rattan for common use cases. The tool is built on top of the rattan-core library, which you can use to build your own tools.

We provide different features under different subcommands:

rattan link: Run a templated channel with command line arguments. We include a set of predefined cells and templated channels in the tool to help you get started quickly. You can emulate a simple network path with just a few commands.
rattan run: Run the instance according to the configuration. For more complex configurations or reproduction purpose, you can use the flexible configuration file to define your own channel with specific cells, network paths and even routes.

There are also global options that you can use to customize the behavior of the tool:

--generate and --generate-path are used to generate a configuration file from your settings instead of running an instance.
--packet-log is used to enable compressed packet log and specify the file to store the records.
--file-log and --file-log-path are used to enable logging to a file and specify the path to store the log.

For more detailed usage, you can run rattan --help to see the available commands and options:

Usage: rattan [OPTIONS] <COMMAND>

Commands:
  link  Run a templated channel with command line arguments
  run   Run the instance according to the config
  help  Print this message or the help of the given subcommand(s)

Options:
      --generate              Generate config file instead of running a instance
      --generate-path <File>  Generate config file to the specified path instead of stdout
      --packet-log <File>     The file to store compressed packet log (overwrite config) (default: None)
      --file-log              Enable logging to file
      --file-log-path <File>  File log path, default to $CACHE_DIR/rattan/core.log
  -h, --help                  Print help (see more with '--help')
  -V, --version               Print version

Predefined Cells and Templated Channels

The templated channels consist of a set of cells defined in the rattan-core library, which you can use to quickly set up a network path for testing. You can use these channels through the rattan link subcommand with just a few command line arguments.

For now, we only provide a simple bidirectional channel, with each direction comprising three cells: bandwidth, delay and loss. This channel is suitable for most common network emulation scenarios.

You can configure the parameters of each cell by passing the arguments to the subcommand.

For example, to create a channel with 10Mbps bandwidth, infinite queue, 50ms latency and 1% loss rate in both directions, you can run the following command:

rattan link uplink-bandwidth 10Mbps uplink-delay 50ms uplink-loss 0.1 downlink-bandwidth 10Mbps downlink-delay 50ms downlink-loss 0.1

Then you will be prompted to a shell with the network path set up.

For detailed usage, you can run rattan link --help to see the available cells and options:

Run a templated channel with command line arguments

Usage: rattan link [OPTIONS] [-- <COMMAND>...]

Arguments:
  [COMMAND]...  Command to run. Only used when in compatible mode

Options:
      --uplink-loss <Loss>              Uplink packet loss
      --downlink-loss <Loss>            Downlink packet loss
      --generate                        Generate config file instead of running a instance
      --uplink-delay <Delay>            Uplink delay
      --downlink-delay <Delay>          Downlink delay
      --generate-path <File>            Generate config file to the specified path instead of stdout
      --packet-log <File>               The file to store compressed packet log (overwrite config) (default: None)
      --uplink-bandwidth <Bandwidth>    Uplink bandwidth
      --file-log                        Enable logging to file
      --uplink-trace <File>             Uplink trace file
      --file-log-path <File>            File log path, default to $CACHE_DIR/rattan/core.log
      --uplink-queue <Queue Type>       Uplink queue type [possible values: infinite, droptail, drophead, codel]
      --uplink-queue-args <JSON>        Uplink queue arguments
      --downlink-bandwidth <Bandwidth>  Downlink bandwidth
      --downlink-trace <File>           Downlink trace file
      --downlink-queue <Queue Type>     Downlink queue type [possible values: infinite, droptail, drophead, codel]
      --downlink-queue-args <JSON>      Downlink queue arguments
  -s, --shell <SHELL>                   Shell used to run if no command is specified. Only used when in compatible mode [default: default] [possible values: default, sh, bash, zsh, fish]
  -h, --help                            Print help (see more with '--help')
  -V, --version                         Print version

Flexible Configuration

Generally, our specialized network emulations necessitate a different number of cells or more complex channels, and they may even require the configuration of routes, such as a multi-path network path with ISP QoS policies. Our CLI tool achieves flexibility and configurability through the use of TOML config files. The configuration file primarily consists of six sections.

We provide an example config file config.example.toml as reference. You can just copy and modify it to suit your needs.

For detailed usage, you can run rattan config --help to see the available cells and options:

Run the instance according to the config

Usage: rattan run [OPTIONS] --config <Config File>

Options:
  -c, --config <Config File>        Use config file to run a instance
      --left [<LEFT_COMMAND>...]    Command to run in left ns. Can be used in compatible and isolated mode
      --generate                    Generate config file instead of running a instance
      --right [<RIGHT_COMMAND>...]  Command to run in right ns. Only used in isolated mode
      --generate-path <File>        Generate config file to the specified path instead of stdout
      --packet-log <File>           The file to store compressed packet log (overwrite config) (default: None)
      --file-log                    Enable logging to file
      --file-log-path <File>        File log path, default to $CACHE_DIR/rattan/core.log
  -h, --help                        Print help (see more with '--help')
  -V, --version                     Print version

General Section

This section determines how to run the rattan itself in general.

packet_log. Now we only have this option in the section, which is used to enable compressed packet log and specify the file to store the records. The meta data of flows will be stored in the file of name packet_log.flows

Environment Section

This section determines how to build the environment.

mode. You can configure rattan to function in Compatible mode or Isolated mode. The ns-left side is always a new network namespace. The difference of the two modes lies in whether ns-right side is the host network namespace (the namespace run rattan) or a new network namespace. The default is Compatible.
left_veth_count and right_veth_count. You can configure the number of veth pairs to create in the ns-left (i.e., between ns-left and ns-rattan) and ns-right (i.e., between ns-right and ns-rattan) network namespaces. The default is 1.

HTTP Section

This section configures Rattan's HTTP control server.

enable. You can enable or disable the HTTP control server. The default is false.
port. You can configure the port number of the HTTP control server. The default is 8086.

Cells Section

This section describes the emulation cells. It is a table of cells, where each cell has a unique string ID, defined like [cells.<ID>].

Each cell MUST have a "type" field, which determines the cell type and cell configuration fields. Possible cell types:

Bw: Fixed bandwidth
BwReplay: Bandwidth trace replayer
Delay: Fixed delay
Loss: Loss pattern
Custom: Custom cell, only specify the cell ID used in link section, and the cell must be built by the user

For example, the following is a cell configuration for a fixed bandwidth cell:

[cells.up_1]
type = "Bw"
bw_type = "NetworkLayer"
bandwidth = "1Gbps 200Mbps"
queue = "DropTail"
queue_config = { packet_limit = 60 }

For detailed parameters, you can check the documentation of rattan-core or the example config file config.example.toml.

Links Section

This section describes how to link the cells.

"left" and "right" are preserved cell IDs, which respectively represent the ns-left side veth (named vL1-R) and the ns-right side veth (named vR1-L). But if a network namespace has more than one veth NICs, ns-left side for example, "left1", "left2" etc. will be defined.

The other cell IDs are defined by user in the cells section.

Each line describes a one-way link from a cell egress to a cell ingress.

For example, to emulate a network topology with two network paths like following:

+-------+  -------->  shadow --->  up_1  --->  up_2  ---> +-------+
| left1 | <==+           ^                                | right |
+-------+    }= router <-|------- down_2 <--- down_1 <--- +-------+
| left2 | <==+           |
+-------+  --------------+

Note: The double line "<===" from router is connected in Cells Section

You can configure the link section like this:

[links]
left1 = "shadow"
left2 = "shadow"
shadow = "up_1"
up_1 = "up_2"
up_2 = "right"
right = "down_1"
down_1 = "down_2"
down_2 = "router"
# We do not need to link the Router Cell to other cells anymore, as we should do it in Cells Section

Resource Section

This section describes how rattan use system resource

The section is preserved for future use but not used now.

cores. The CPU cores that rattan is allowed to use.

Commands Section

This section describes the commands to run in rattan.

When in Compatible mode, only the left and shell options are used, and the commands will run in the ns-left network namespace.

When in Isolated mode, only the left and right options are used, and the commands will run in the ns-left and ns-right network namespace, respectively.

left and right. The commands to run in the ns-left and ns-right network namespace, respectively. Should be an array of strings.
shell. The shell used to run if no command is specified. Only used when in compatible mode. The default is Default (i.e., the default shell defined in environment variable $SHELL).

Write a New Cell (TODO)

Write Your Tailored Tool (TODO)

Packet I/O

Currently, we support two packet I/O mechanisms: AF_PACKET and AF_XDP. The former is a socket-based mechanism, while the latter is a driver-based mechanism. The AF_PACKET version is more portable, while the AF_XDP version is more performant.

Example

By default, the AF_PACKET version is used. To use the AF_XDP version, you need to enable the camellia feature.

cargo run --example channel --release # AF_PACKET version
cargo run --example channel-xdp --release --features="camellia" # AF_XDP version

Flamegraph

To generate a flamegraph, you can use the flamegraph tool. First, install it:

cargo install flamegraph

Then, run the following command:

cargo flamegraph --root --example channel

This will generate a flamegraph, which you can open with a browser to inspect hot spot.

Packet Log Spec

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|       LH.length       | LH.ty.|   GPH.length  |GPH.ac.|GPH.ty.|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                          GP.timestamp                         |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|           GP.length           |       PRH.length      |PRH.ty.|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                          PRT (custom)                         |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Generated with protocol, where:

LH is short for log entry header
GPH is short for general packet entry header
GP is short for general packet entry (the body part)
PRH is short for protocol entry header
PRT is short for protocol entry (the body part)
ty. is short for type
ac. is short for action

protocol "LH.length:12,LH.ty.:4,GPH.length:8,GPH.ac.:4,GPH.ty.:4,GP.timestamp:32,GP.length:16,PRH.length:12,PRH.ty.:4,PRT (custom):32"

We currently provide TCP log spec (a variant of the protocol entry body part, i.e. PRT (custom)):

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                          tcp.flow_id                          |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                            tcp.seq                            |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                            tcp.ack                            |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|             ip.id             |         ip.frag_offset        |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|          ip.checksum          |   tcp.flags   |    padding    |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

protocol "tcp.flow_id:32,tcp.seq:32,tcp.ack:32,ip.id:16,ip.frag_offset:16,ip.checksum:16,tcp.flags:8,padding:8"

Rattan Documentation