Benchmarking Tool

Linux command-line Benchmarking Tool

Copyright © 2020 Teal Dulcet

A port of the hyperfine Benchmarking Tool to Bash.

• Does NOT require installing Rust, downloading dependencies or compiling anything.
• Includes the same features (except Linux only), produces the same output (with some improvements) and supports most of the same command line options.
• Outputs most of the numbers with greater precision and outputs more information.
• Supports outputting in ASCII only (no Unicode characters) to support older terminals.
• Slightly faster when interactive output (the progress bar) is disabled, as it does not need to launch intermediate shells.

❤️ Please visit tealdulcet.com to support this script and my other software development.

Benchmark of the GNU factor and uutils factor commands.

Also see the Testing and Benchmarking scripts.

Usage

Run: ./time.sh [OPTIONS] <command(s)>...\ All the options can also be set by opening the script in an editor and setting the variables at the top. See Help below for full usage information.

1. Download the script (time.sh). Run: wget https://raw.github.com/tdulcet/Benchmarking-Tool/main/time.sh.
2. Execute the script once to make sure there are no errors. For example, run: chmod u+x time.sh and ./time.sh 'sleep 0.3'.
3. If you want the script to be available for all users, install it. Run: sudo cp time.sh /usr/local/bin/benchmark and sudo chmod +x /usr/local/bin/benchmark.

Help

``` $ benchmark -h Usage: benchmark [OPTION(S)] ... or: benchmark All the options can also be set by opening the script in an editor and setting the variables at the top. See examples below.

Options: -w Warmup Perform NUM warmup runs before the actual benchmark. This can be used to fill (disk) caches for I/O-heavy programs. Default: 0 -m Min-runs Perform at least NUM runs for each command. Default: 10 -M Max-runs Perform at most NUM runs for each command. Default: no limit -r Runs Perform exactly NUM runs for each command. If this option is not specified, it will automatically determines the number of runs. -s Setup Execute command before each set of runs. This is useful for compiling your software with the provided parameters, or to do any other work that should happen once before a series of benchmark runs, not every time as would happen with the prepare option. -p Prepare Execute command before each run. This is useful for clearing disk caches, for example. The prepare option can be specified once for all commands or multiple times, once for each command. In the latter case, each preparation command will be run prior to the corresponding benchmark command. -f Conclude Execute command after each timing run. This is useful for killing long-running processes started (e.g. a web server started in prepare), for example. The conclude option can be specified once for all commands or multiple times, once for each command. In the latter case, each conclude command will be run after the corresponding benchmark command. -c Cleanup Execute command after the completion of all benchmarking runs for each individual command to be benchmarked. This is useful if the commands to be benchmarked produce artifacts that need to be cleaned up. -i Ignore-failure Ignore non-zero exit codes of the benchmarked programs. -u ASCII Do not use Unicode characters in output. -N No color Do not use color in output. -S Disable interactive Disable interactive output and progress bars. -C Export CSV Export the timing summary statistics as CSV to the given FILE. -j Export JSON Export the timing summary statistics and timings of individual runs as JSON to the given FILE. -o Output Control where the output of the benchmark is redirected. can be: null: Redirect both stdout and stderr to '/dev/null' (default). pipe: Feed stdout through a pipe before discarding it and redirect stderr to '/dev/null'. inherit: Output the stdout and stderr. : Write both stdout and stderr to the given FILE. -n Command-name Give a meaningful name to a command. This can be specified multiple times if several commands are benchmarked. -h Display this help and exit -v Output version information and exit

Examples: Basic benchmark $ benchmark 'sleep 0.3'

Benchmark two commands
$ benchmark 'find -iname "*.jpg"' 'fd -e jpg -uu'

Benchmark piped commands
$ benchmark 'seq 0 10000000 | factor' 'seq 0 10000000 | uu-factor'

Warmup runs
$ benchmark -w 3 'grep -R TODO *'

Parameterized benchmark
$ benchmark -p 'make clean' 'make -j '{1..12}
This performs benchmarks for 'make -j 1', 'make -j 2', … 'make -j 12'.

Parameterized benchmark with step size
$ benchmark 'sleep 0.'{3..7..2}
This performs benchmarks for 'sleep 0.3', 'sleep 0.5' and 'sleep 0.7'.

Parameterized benchmark with list
$ benchmark {gcc,clang}' -O3 main.c'
This performs benchmarks for 'gcc -O3 main.c' and 'clang -O3 main.c'.

```

Contributing

Pull requests welcome! Ideas for contributions:

• Support more of hyperfine's options
• Add option to use the GNU time command (/usr/bin/time)
• Add more examples
• Improve the performance
• Add tests