AutoPas-CoolMUC-Jobs

SLURM job scripts to test AutoPas on the LRZ Linux Cluster. This Readme is adapted from bachelor thesis Improving OpenMP Loop Scheduling in AutoPas, Appendix B.

[!WARNING] Please edit your valid email into the scripts before submitting them.

Setup

```

Log into the linux cluster.

ssh -Y lxlogin1.lrz.de -l

Clone AutoPas and the Slurm job scripts:

cd $HOME && mkdir -p ba && cd ba \ && git clone https://github.com/AutoPas/AutoPas-CoolMUC-Jobs.git \ && git clone https://github.com/AutoPas/AutoPas.git \ && cd AutoPas && git checkout feature/improving-openmp-loop-scheduling \ && mkdir build && cd build

Build MD-Flexible:

CC=which clang CXX=which clang++ cmake -DMDFLEXIBLEUSEMPI=OFF \ -DAUTOPASLOG_ITERATIONS=ON .. && make -j8 md-flexible && cd ..

Build master-branch-based MD-Flexible for explodingLiquid:

git checkout \ feature/improving-openmp-loop-scheduling-dynamic-vl-unloaded \ && mkdir build-dvl-off && cd build-dvl-off

CC=which clang CXX=which clang++ cmake -DMDFLEXIBLEUSEMPI=OFF \ -DAUTOPASLOG_ITERATIONS=ON .. && make -j8 md-flexible && cd .. ```

Option 1: individual tests with salloc.

```

Allocate a CoolMUC-2 job:

cd build/examples/md-flexible \ && salloc --partition=cm2_inter --time=01:00:00

Prioritize Auto4OMP's libomp.so:

export LDLIBRARYPATH=\ $HOME/ba/AutoPas/build/deps/auto4omp-build/runtime/src\ :$LDLIBRARY_PATH

Make sure Auto4OMP's libomp.so is linked:

ldd md-flexible

Configure OpenMP to use 28 parallel threads:

export OMPNUMTHREADS=28

Get the GPU frequency and pass it to Auto4OMP:

export KMPCPUSPEED=$(lscpu | grep "CPU max MHz" | \ tr -d ' ' | cut -d ":" -f2 | cut -d "." -f1)

Optional: log Auto4OMP’s selection decisions.

Beware, this worsens performance!

export KMPTIMELOOPS=./auto4omp.log

Execute.

The possible inputs are at AutoPas/examples/md-flexible/input.

Pass the input file name without a path, as all inputs

are copied directly under AutoPas/build/examples/md-flexible.

This example tests the homogeneous lc_c08 small input

with Auto4OMP's ExhaustiveSel.

srun ./md-flexible --openmp-kind exhaustiveSel \ --yaml-filename homogeneousLCC08Small.yaml

exit ```

Option 2: submit a job script for a full test.

This takes a few hours per job, depending on the input. Beware, the jobs assume AutoPas is at ~/ba/AutoPas. For custom directories, the scripts have to be adjusted. This example tests the small homogeneous lc_c08 input.

```

Move to the job's subdirectory so the logs are written there:

cd $HOME/ba/AutoPas-CoolMUC-Jobs/auto4omp/homogeneousLCC08Small \ && sbatch homogeneousLCC08Small.sh && cd $HOME

To track a submitted job:

squeue -M cm2_tiny

To cancel a submitted job (replace with the job's ID from sbatch or squeue):

scancel -M cm2_tiny

To summarize job outputs (assuming a single output exists per input):

INPUTDIR=$HOME/ba/AutoPas-CoolMUC-Jobs/auto4omp/homogeneousLCC08Small \ && grep $INPUTDIR/$(ls $INPUT_DIR | grep ".out" -m 1) \ -e == -e LCC -e VCL -e VLP -e Simulate

To summarize Auto4OMP's log (e.g., exhaustiveSel):

(prints the number of times each scheduling technique was used.)

KMPLOG=$INPUTDIR/auto-3.log \ && echo "static:" && grep -o " STATIC " $KMPLOG | wc -l \ && echo "dynamic:" && grep -o " SS " $KMPLOG | wc -l \ && echo "trapezoidal:" && grep -o " TSS " $KMPLOG | wc -l \ && echo "auto:" && grep -o "LLVM" $KMPLOG | wc -l \ && echo "guided:" && grep -o " GSS " $KMPLOG | wc -l \ && echo "steal:" && grep -o " Steal " $KMPLOG | wc -l \ && echo "fac2a:" && grep -o " mFac2 " $KMPLOG | wc -l \ && echo "awfb:" && grep -o " AWF-B " $KMPLOG | wc -l \ && echo "awfc:" && grep -o " AWF-C " $KMPLOG | wc -l \ && echo "awfd:" && grep -o " AWF-D " $KMPLOG | wc -l \ && echo "awfe:" && grep -o " AWF-E " $KMPLOG | wc -l \ && echo "afa:" && grep -o " mAF " $KMP_LOG | wc -l ```