Locust Load Test Introduction

In this tutorial, we will introduce you to Locust Load Tests on Neocortix Cloud Services Scalable Compute. In particular, we will introduce you to the Ansible orchestration package, and the Locust Load Testing package, and how to install them and get them running.

The Locust framework is provided by https://locust.io/ and its main documentation is at https://docs.locust.io/en/stable/.

In this tutorial example, a Locust master node will run on a conventional server and the worker nodes ("slaves" in Locust terminology) will run on Neocortix Cloud Services Scalable Compute devices.

We use the open-source Ansible tool for managing nodes. It is handy for sending the same SSH commands to many machines, and for installing software on them. Its main documentation is at https://docs.ansible.com/ansible/latest/index.html.

Prerequisites

1.  A conventional Linux server (or VM) with a public IP address, to act as master
2. Sudo privileges (or the equivalent) on that server, so you can install software and expose some TCP ports to the internet
3.  A Neocortix Cloud Services account with permissions to create instances
4.  A web server that you are legally allowed to load-test

Note: this procedure was tested on newly-created VMs with 8 GB RAM, running Ubuntu 16.04.4 or 18.04.2. Details will be different if you use a distribution that does not use apt for software installation.

Preparing The Master Node

Install ansible, using the following steps:

sudo apt-get update
sudo apt-get install software-properties-common
sudo apt-add-repository --yes --update ppa:ansible/ansible
sudo apt-get install ansible

Note: Official Ansible installation instructions are at
https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html.

Install python, pip, and python packages:

sudo apt-get install python3-pip rsync
pip3 install --user flask gevent msgpack-python pyzmq python-dateutil geoip2 pandas

Get the code

cd ~
git clone https://github.com/neocortix/ncscli.git # only if not already cloned
git clone https://github.com/locustio/locust.git

Open firewall ports 5557, 5558 and 8089 for incoming TCP

Run this command, to make sure your installation of Locust was successful, substituting the URL of the website you want to test.

cd ~/ncscli/examples/loadtest; python3 runLocust.py --host=http://YourWebSite.com --master --loglevel INFO -f master_locust.py

You should see messages like the following, if the installation is good

[2018-12-02 18:40:14,483] ip-172-26-4-17/INFO/locust.main: Starting web monitor at *:8089
[2018-12-02 18:40:14,484] ip-172-26-4-17/INFO/locust.main: Starting Locust 0.9.0

In a browser, browse to port 8089 of your master host, e.g. http://YourMasterHost.com:8089/ You should see the locust UI:

Locust0

Kill locust, using ctrl-c from your terminal session.

Preparing Worker Instances

From the loadtest directory, launch as many instance as you would like, to act as worker nodes. Here is an example command that launches 10 instances, and stores the returned information about the instances in a file called launched.json:

~/ncscli/ncscli/ncs.py sc --authToken YourAuthTokenHere launch --json --sshClientKeyName YourClientKeyNameHere --count 10 > launched.json

For more information on launching instances, see the tutorial Launching Multiple Nodes (Script Interface) .

Use the following two commands to install the prerequisites for locust to run on the worker nodes

cat launched.json | ~/ncscli/ncscli/jsonToInv.py > launched.inv ANSIBLE_HOST_KEY_CHECKING=False

ansible-playbook installPrereqsQuicker.yml -i launched.inv

Running Load Tests

Assuming you have a launched.inv file prepared, as shown in the previous step, run this ansible command to start locust as a worker on all your worker nodes.

ansible-playbook startWorkers.yml --e "victimUrl=http://YourWebSite.com masterHost=YourMasterHost" -i launched.inv

Start the master process using the following python command.

python3 -u runLocust.py --host=http://YourWebSite.com --master --heartbeat-liveness=30 -f master_locust.py

Locust will print some progress and status information to the stdout as shown below. You may note the number of worker (slave) nodes that show up.

[2018-12-02 18:40:14,483] ip-172-26-4-17/INFO/locust.main: Starting web monitor at *:8089
[2018-12-02 18:40:14,484] ip-172-26-4-17/INFO/locust.main: Starting Locust 0.9.0

Now that locust is up and running on all the nodes, browse to port 8089 of your master host to control testing. You should see the front page of the Locust UI. At the top right it shows the number of workers (slaves) it knows about. This will match the number of workers (slaves) you started, unless something has gone wrong.

Locust front

Enter a number of users to simulate. For a first test, just enter the number of workers (slaves). You could also enter a small multiple of the number of workers (slaves) (e.g. 4X) to make the victim work harder.

Enter a number for "hatch rate", the number of simulated users spawned per second. This controls how quickly the load test ramps up at the beginning. The ideal number depends on the system you are testing. For example, if you are simulating 1000 users but you don't want them all showing up at the same time, enter a hatch rate of less than 1000.

Click "Start Swarming" to start the test. Locust will then show its main UI.

Locust stats

To stop a test, click "Stop" in the Locust web UI.

To exit the processes, use ctrl-c in the master's terminal window, or kill it with SIGINT. When shutting down, the master sends a message to all workers telling them to exit.