SYCL-Bench

SYCL Benchmark Suite, work in progress

Benchmarks support the following command line arguments: * --size=<problem-size> - total problem size. For most benchmarks, global range of work items. Default: 3072 * --local=<local-size> - local size/work group size, if applicable. Not all benchmarks use this. Default: 256 * --num-runs=<N> - the number of times that the problem should be run, e.g. for averaging runtimes. Default: 5 * --device=<d> - changes the SYCL device selector that is used. Supported values: cpu, gpu, default. Default: default * --output=<output> - Specify where to store the output and how to format. If <output>=stdio, results are printed to standard output. For any other value, <output> is interpreted as a file where the output will be saved in csv format. * --verification-begin=<x,y,z> - Specify the start of the 3D range that should be used for verifying results. Note: Most benchmarks do not implement this feature. Default: 0,0,0 * --verification-range=<x,y,z> - Specify the size of the 3D range that should be used for verifying results. Note: Most benchmarks do not implement this feature. Default: 1,1,1 * --no-verification - disable verification entirely * --no-ndrange-kernels - do not run kernels based on ndrange parallel for * --warmup-run - run benchmarks once before evaluation to discard possible "warmup" times, e.g., JIT compilation

Usage

Clone sycl-bench repo $ git clone https://github.com/bcosenza/sycl-bench.git

Navigate into repo and create build folder $ cd bench $ mkdir build && cd build

Compile with CMake $ cmake -DSYCL_IMPL=[target SYCL implementation] [-DSYCL_BENCH_HAS_FP64_SUPPORT=ON|OFF] [other compiler arguments] .. $ cmake --build . $ sudo make install Example compiling with CMake for DPC++: $ cmake -DSYCL_IMPL=LLVM -DCMAKE_CXX_COMPILER=/path/to/llvm/build/bin/clang++ ..

Each test should now have an executable in the build folder Run individual tests as such: $ ./arith --device=cpu --output=output.csv

Packaging

SYCL-Bench provides a CMake target package (and package_source) to package a SYCL-Bench installation. Users can configure what generators to use to build the packages by passing a semicolon-separated list of generators to use to the CPACK_GENERATOR CMake flag and then build the enabled packages by building the package target:

cmake -Bbuild -DCPACK_GENERATOR="TGZ;ZIP" cmake --build build --target package

For more information, check the CPack documentation for the relevant CMake version.

Packages built via the package target will contain all files contained in a SYCL-Bench installation (binaries, scripts, benchmark inputs). Packages built via the package_source target will additionally contain the source files.

Attribution

If you use SYCL-Bench, please cite the following papers: @inproceedings{SYCL-Bench:IWOCL:2024, author = {Crisci, Luigi and Carpentieri, Lorenzo and Thoman, Peter and Alpay, Aksel and Heuveline, Vincent and Cosenza, Biagio}, title = {SYCL-Bench 2020: Benchmarking SYCL 2020 on AMD, Intel, and NVIDIA GPUs}, year = {2024}, isbn = {9798400717901}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3648115.3648120}, doi = {10.1145/3648115.3648120}, booktitle = {Proceedings of the 12th International Workshop on OpenCL and SYCL}, articleno = {1}, numpages = {12}, keywords = {GPU, HPC, SYCL, benchmark, heterogeneous computing, portability}, location = {<conf-loc>, <city>Chicago</city>, <state>IL</state>, <country>USA</country>, </conf-loc>}, series = {IWOCL '24} }

@inproceedings{SYCL-Bench:Euro-Par:2020, author = {Lal, Sohan and Alpay, Aksel and Salzmann, Philip and Cosenza, Biagio and Hirsch, Alexander and Stawinoga, Nicolai and Thoman, Peter and Fahringer, Thomas and Heuveline, Vincent}, title = {{SYCL-Bench: A Versatile Cross-Platform Benchmark Suite for Heterogeneous Computing}}, year = {2020}, publisher = {Springer International Publishing}, booktitle = {Euro-Par 2020: 26th International European Conference on Parallel and Distributed Computing}, series = {Euro-Par ’20} }

@inproceedings{SYCL-Bench:IWOCL:2020, author = {Lal, Sohan and Alpay, Aksel and Salzmann, Philip and Cosenza, Biagio and Stawinoga, Nicolai and Thoman, Peter and Fahringer, Thomas and Heuveline, Vincent}, title = {{SYCL-Bench: A Versatile Single-Source Benchmark Suite for Heterogeneous Computing}}, year = {2020}, isbn = {9781450375313}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3388333.3388669}, doi = {10.1145/3388333.3388669}, booktitle = {Proceedings of the International Workshop on OpenCL}, articleno = {10}, numpages = {1}, keywords = {Heterogeneous Computing, SYCL Benchmarks &Runtime}, location = {Munich, Germany}, series = {IWOCL ’20} }