EffiMAP

Project Code: Tepig

For more details, please refer to the following publication.

Publication

Z. Wei, H. Wang, I. Ashraf and W. K. Chan, "Predictive Mutation Analysis of Test Case Prioritization for Deep Neural Networks," 2022 IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS), Guangzhou, China, 2022, pp. 682-693, doi: 10.1109/QRS57517.2022.00074. keywords: {Deep learning;Analytical models;Computational modeling;Neural networks;Software quality;Predictive models;Computational efficiency;Test case prioritization;mutation analysis;testing},

@INPROCEEDINGS{10062402, author={Wei, Zhengyuan and Wang, Haipeng and Ashraf, Imran and Chan, W.K.}, booktitle={2022 IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS)}, title={Predictive Mutation Analysis of Test Case Prioritization for Deep Neural Networks}, year={2022}, volume={}, number={}, pages={682-693}, keywords={Deep learning;Analytical models;Computational modeling;Neural networks;Software quality;Predictive models;Computational efficiency;Test case prioritization;mutation analysis;testing}, doi={10.1109/QRS57517.2022.00074} }

Preparation

The dataset and the model will be fetched automatically. No need to handle by the user.

Installation

The project is maintained with Pipenv. Please refer to the link for installing Pipenv.

The dependencies are very convenient to install by one command. The versions are same as proposed here.

bash pipenv sync

How to run

The executions are well organized with the help of arguments. you can run --help command to check it.

for example,

``` ~/workspace/effimap{main} > pipenv run python src/extract.py -h usage: extract.py [-h] [--datadir DATADIR] [--outputdir OUTPUTDIR] [--device {cpu,cuda}] [--gpu {0,1,2,3}] [--seed SEED] [-b BATCHSIZE] [-m {resnet32,mlp,svhn,stl10,resnet18,resnet20,msgdn}] [-d {cifar10,cifar100,mnist,svhn,stl10,tinyimagenet,nuswide}] [-e EPOCHS] [--lr LR] [--fuzzenergy FUZZENERGY] [--nummodelmutants NUMMODELMUTANTS] [--numinputmutants NUMINPUTMUTANTS] [--task {classify,regress,multilabels}] [--primasplit {val,test}] --strategy {prima,furret}

optional arguments: -h, --help show this help message and exit --datadir DATADIR --outputdir OUTPUTDIR --device {cpu,cuda} --gpu {0,1,2,3} --seed SEED -b BATCHSIZE, --batchsize BATCHSIZE -m {resnet32,mlp,svhn,stl10,resnet18,resnet20,msgdn}, --model {resnet32,mlp,svhn,stl10,resnet18,resnet20,msgdn} -d {cifar10,cifar100,mnist,svhn,stl10,tinyimagenet,nuswide}, --dataset {cifar10,cifar100,mnist,svhn,stl10,tinyimagenet,nuswide} -e EPOCHS, --epochs EPOCHS --lr LR --fuzzenergy FUZZENERGY --nummodelmutants NUMMODELMUTANTS --numinputmutants NUMINPUTMUTANTS --task {classify,regress,multilabels} --primasplit {val,test} --strategy {prima,furret} ```