Testing TCP Congestion Control

From packets2photons
Jump to navigation Jump to search

This guide is an abbreviated set of notes on how I might test TCP congestion control. I typically would use a Raspberry Pi network on my desk, for more synthetic tests as well as a Cloud provider such as Amazon EC2 or Digital Ocean for some more real-world tests.


To work out what TCP algorithm is currently in use on your Linux system, you can do a:

cat /proc/sys/net/ipv4/tcp_congestion_control

To check what TCP variants your current system supports, run the following:

sysctl net.ipv4.tcp_available_congestion_control

To change the Linux TCP congestion control algorithm to one of those listed, run the following command, note that I am using tee to prevent having to become root:

echo bbr | sudo tee /proc/sys/net/ipv4/tcp_congestion_control

If you wanted to make the change permanent, and persist past a reboot then you should add the following line to /etc/sysctl.conf:


As above, I was interested in the performance of BBR in comparison with Cubic and Reno. To do this, I have run synthetic tests in a raspberry Pi network shown below.

Synthetic Raspberry Pi Network

Each Raspberry Pi has 2 Ethernet network adaptors so I have a basic string topology shown below.

5 node Raspberry Pi network, strung together with Vantec USB to Ethernet adaptors
5 node Raspberry Pi network, strung together with Vantec USB to Ethernet adaptors

To emulate latency I use NetEm, a lot more documentation is found here, but a sample command to emulate 300 ms of latency on the central node is below:

sudo tc qdisc add dev eth0 root netem delay 300ms

For these tests created a 10MB file in RAM, to prevent and access times from getting in the way, then linked to it from my www directory, which Apache serves files from.

dd if=/dev/zero of=/dev/shm/output.dat count=10024 bs=1024

Then linked to this from the webserver:

sudo ln -s /dev/shm/output.dat /var/www/html/output.dat

Make sure anything you create is owned by www-data:

sudo chown www-data /var/www/html/*

To download the file, I use the following:

wget -O/dev/null

This downloads the file to /dev/null and removes any interaction from local media from interfering with the network results.