:boom: gocannon - HTTP benchmarking tool

Gocannon is a lightweight HTTP benchmarking tool, intended to measure changes in backend application performance over time. It keeps a detailed log of each request that is sent, not just the histogram of their latencies.

Installation

The easiest way to install Gocannon is to download pre-built binaries for your platform that are automatically attached to each release. If you wish to build Gocannon from source, clone this repository and run go build. Alternatively, you can run Gocannon as a container using jauderho/gocannon image.

Usage

``` usage: gocannon []

Flags: --help Show context-sensitive help (also try --help-long and --help-man). -d, --duration=10s Load test duration. -c, --connections=50 Maximum number of concurrent connections. --cpus Maximum number of logical CPUs that can be utilised. Defaults to the number of available CPU threads (or the GOMAXPROCS env variable). -t, --timeout=200ms HTTP client timeout. -m, --mode="reqlog" Statistics collection mode: reqlog (logs each request) or hist (stores histogram of completed requests latencies). -o, --output=file.csv File to save the request log in CSV format (reqlog mode) or a text file with raw histogram data (hist mode). -i, --interval=250ms Interval for statistics calculation (reqlog mode). --preallocate=1000 Number of requests in req log to preallocate memory for per connection (reqlog mode). --method=GET The HTTP request method (GET, POST, PUT, PATCH or DELETE). -b, --body="{data..." HTTP request body. -h, --header="k:v" ... HTTP request header(s). You can set more than one header by repeating this flag. --trust-all Omit SSL certificate validation. -f, --format=default Load test report format. Either 'default' (verbose), 'json' or 'yaml'. When json or yaml is specified, apart from the load test results, no additional info will be written to std out. --plugin=/to/p.so Plugin to run Gocannon with (path to .so file). --version Show application version.

Args: HTTP target URL with port (i.e. http://localhost:80/test or https://host:443/x) ```

Below is an example of a load test conducted using gocannon against an Express.js server (notice the performance improvement over time under sustained load):

```

gocannon http://localhost:3000/static -d 3s -c 50 Attacking http://localhost:3000/static with 50 connections over 3s gocannon goes brr... Total Req: 14990 Req/s: 4996.67 Interval stats: (interval = 250ms) |--REQS--| |------------------------LATENCY-------------------------| Count AVG P50 P75 P90 P99 673 18.004823ms 17.768106ms 18.810048ms 20.358396ms 31.970312ms 777 16.090895ms 16.096573ms 17.475942ms 18.049402ms 19.867212ms 922 13.630393ms 13.573315ms 14.279727ms 15.720507ms 17.725149ms 987 12.715278ms 12.050508ms 14.378844ms 15.661408ms 20.669683ms 1137 10.997768ms 10.201962ms 11.468307ms 13.736441ms 15.374815ms 1354 9.186035ms 8.689116ms 9.831591ms 10.417963ms 11.393158ms 1412 8.889394ms 8.444931ms 8.834144ms 10.480375ms 11.886929ms 1428 8.792173ms 7.880262ms 9.580714ms 10.753837ms 14.997604ms 1599 7.817586ms 7.578824ms 7.645575ms 9.083118ms 10.047772ms 1505 8.244456ms 7.580401ms 8.307621ms 9.611815ms 17.449891ms 1617 7.745439ms 7.422272ms 7.487905ms 9.132874ms 11.457255ms

1579 7.901125ms 7.525814ms 7.655212ms 9.577895ms 10.203331ms

 14990    9.986321ms  8.540818ms 11.252258ms 15.309979ms 19.632593ms

Responses by HTTP status code: 200 -> 14990 Requests ended with timeout/socket error: 1 ```

Saving request log to a CSV file

When used with the --output=filename.csv flag, raw request data (which includes the request HTTP code, starting timestamp, response timestamp both in nanoseconds since the beginning of the test as well as the ID of the connection which was used for that request) can be written into the specified csv file in the following format:

code;start;end;connection; 200;509761;30592963;0; 200;694043;34837869;0; 200;963463;36094909;0; 200;841397;37319812;0; [...]

In the CSV output file, the requests are sorted primarily by connection ID and secondarily by response timestamp (end column).

Preallocating memory for request log

Gocannon stores data of each completed request in a slice (one per each connection). If a given slice runs out of its initial capacity, the underlying array is re-allocated to a newly selected memory address with double the capacity. In order to minimize the performance impact of slice resizing during the load test, you can specify the number of requests in log per connection to preallocate memory for using the --preallocate flag.

Histogram mode

There are some use cases in which request log mode is not desirable. If you don't need a detailed log of each request that was completed during the load test, you can use histogram mode using the --mode=hist flag. When using histogram mode, upon each completed request, gocannon will only save its latency in a histogram (in a manner similar to how wrk or bombardier does that). That way, memory usage of the stats collector will remain constant over the entire duration of the load test. This results in gocannon having a total memory footprint of about 3MB when conducting a load test in histogram mode (when the timeout is set to a default 200ms).

Below is an example of a load test conducted with gocannon in histogram mode:

```

gocannon http://localhost:3000/static --duration=2m --mode=hist Attacking http://localhost:3000/static with 50 connections over 2m0s gocannon goes brr... Total Req: 13715707 Req/s: 114297.56 |------------------------LATENCY (μs)-----------------------| AVG P50 P75 P90 P99 436.45 278 508 925 2681 Responses by HTTP status code: 200 -> 13715707 ```

Similarly to saving CSV output in request log mode, you can write the histogram data to a text file using the --output=filename flag. The output contains the number of hits in each latency histogram bin (having width of 1μs) truncated up until the last non-zero value:

0 0 1 1 15 45 [...]

Custom plugins

Gocannon supports user-provided plugins, which can customize the requests sent during the load test. A custom plugin has to satisfy the GocannonPlugin interface defined in the common package, which also contains types required for plugin development. An example implementation with additional comments is provided in _example_plugin folder.

In order to build a plugin, use the following command inside its directory:

go build -buildmode=plugin -o plugin.so plugin.go

Once you obtain a shared object (.so) file, you can provide a path to it via --plugin flag. Bare in mind that a custom plugin .so file and the Gocannon binary using it must both be compiled using the same Go version.

Using Gocannon as a library

You can use Gocannon as a library for performing programmatic load tests. Consult GoDoc for API reference and main.go for example implementation. Please bare in mind that you cannot reuse a Gocannon instance to perform multiple load tests.

Load testing recommendations

• Killing non-essential background processes: Since the activity of background processes running alongside the SUT may introduce anomalies in the obtained experiment results, it is advised to disable non-essential services for the load test duration. Additionally, when conducting a load test against a SUT running on the same machine as gocannon, you may want to assign the SUT and gocannon processes to separate sets of logical cores (i.e. via taskset) and update the GOMAXPROCS env variable accordingly (so as to reflect the number of cores available to gocannon).
• Disabling Turbo Boost, EIST and ensuring a constant CPU clock: Changes of the CPU clock over the duration of the load test may introduce inconsistencies in the collected metrics, especially given that the detection of a suitable workload after the test start and subsequent increase of the CPU clock may cause observable improvement in application throughput shortly after the start of the load test. If your goal is to measure how the application performance changes over time under sustained load, you may want to disable Turbo Boost and EIST (or their equivalents in your system) as well as ensure that the CPU clock is not being reduced at idle (i.e. by setting the CPU mode to performance via the cpupower linux package).
• Repeating the test runs: Performing multiple test runs improves the statistical significance of the obtained results and minimizes impact of potential one-off anomalies (i.e. background process activity during the load test) in the obtained data on the overall result. When combining data from multiple test runs, remember not to average the percentiles. Instead, place the obtained request latencies of all runs in a single dataset and calculate the desired percentiles on that dataset.

License

Copyright Paweł Kuffel 2021-2022, licensed under Apache 2.0 License.

The Gocannon logo contains the Go Gopher mascot which was originally designed by Renee French (http://reneefrench.blogspot.com/) and licensed under Creative Commons 3.0 Attributions license.