This is a network tester playbook, which uses HP Netperf and Flent under the hood. The test is based on a mixed TCP/UDP/ICMP workload that is executed over three different client-server set configurations (1xM, Nx1, NxM). This software requires python 3.6 and ansible >= 2.4, which is used to install and execute the tests across all machines. The playbook has been tested on the following OS platforms: RHEL/CentOS 7.x, Ubuntu 18.04, and Rocky 9.
Start cloning this repository on the host where you would like to run the test. Go to the root of the repository, then install ansible packages with the following:
ansible-galaxy collection install -r requirements.yaml
All further dependencies will be installed by the install role of the playbook. Be aware that this includes adding package repositories and installing software from them. See the network_diag_install tasks and defaults for more details.
To complete this test, it is essential that the host running the ansible code can perform password-less sudo and that all target hosts are accessible via SSH.
We recommend using SSH keys for authentication, although passwords also can be used.
You may rely on your current SSH client configuration or provide authentication data
explicitly in the file group_vars/netdiag_hosts.yaml. Refer to
Ansible documentation for further information.
Log in once to each host for them to be added to trusted hosts.
Make sure that the firewall configuration of all hosts allows pings and TCP connections on ports in the range of 5000-65536.
Further, the following conditions must be met:
- TCP networks: Ethernet flow control (global pause frames) disabled, ECN configured and tuned, and homogeneous MTU
Hosts are divided into two sets: clients and servers. The sets may overlap, but since this playbook is about network testing, disjoint set configuration is recommended. Populate the inventory_file accordingly. When you open the inventory_file, it will look like this:
You will now populate the netdiag_clients section with the hostname or IP’s of the client machines, with one per line and no punctuation between lines. Then you will do the same for servers in the netdiag_servers section. An example of an inventory_file for a disjoint set configuration is shown below:
For a typical Quobyte installation, you will want to run two tests - one between actual client and server machines and one between one half of the servers as clients, and the other half as servers.
To verify that the configuration is correct and that all of the listed machines are accessible, run the following:
ansible -i inventory_file -m ping -b all
The command must produce no errors. The output should look something like this, and be indicated with a SUCCESS message for each machine:
Due to extensive use of sockets, the user on behalf of which tests are executed should be granted very high or unlimited file descriptor limits on target hosts. This is typically done by the ansible command execution automatically.
To install dependencies, run the test, and clean up afterwards in one go, run:
ansible-playbook -i inventory_file netdiag.yaml
You can also split that into three phases:
ansible-playbook -i inventory_file netdiag.yaml -t install
ansible-playbook -i inventory_file netdiag.yaml -t test
ansible-playbook -i inventory_file netdiag.yaml -t cleanup
Test results are stored on an ansible host at /tmp/quobyte_netdiag_data. The results
can be inspected with the flent-gui and help identify a range of issues in the network.
It is important to include a representative set of servers and client machines,
in particular if the machines are on different leaf switches, e.g. include machines
so that communication between them must traverse the spine.
Once you have the results from the test runs, you can visualize them with flent-gui.
Installation of this tool depends on your distribution. Once you have the flent-gui installed, you have to copy the flent config file from
network-tester/roles/network_diag_traffic/files/rrul_be_nflows_mulithost.conf into the flent config directory. The location of that directory depends on how you installed flent-gui
(package vs. python pip), e.g. /usr/local/lib/python3.9/site-packages/flent/tests.
When the configuration file is in place, navigate to your results directory,
/tmp/quobyte_netdiag_data. From here, you may run
flent-gui
The GUI will pop up and from there you may select files from your results directory to visualize. Example plots rendered from the flent-gui are pictured below:
Product names and brands found in the repository are property of their respective owners. The playbook is distributed under Apache License 2.0.