Replication package for the thesis "Resource Characterization and Optimization for End-to-End Testing"

This repository contains the replication package of the thesis Resource Characterization and Optimization for End-to-End Testing

The replication package comprises the test scripts, Jenkinsfiles and configuration files used during the evaluation revision of the Chapter IV, but also the necessary links to the other replication packages used during the thesis, structured in the different chapters (IV,V and VI)

CHAPTER IV: RETORCH – End-to-End optimization through the smart characterization of the resources Evaluation.

Located in /chapterIV folder contains all the evaluation data and scripting of this chapter. The folder is composed by three different subfolders

• /chapterIV/jenkinsfiles: contains the Jenkinsfiles of the different execution plans: baseline, sequential, 3-parallel, 4-parallel, and the RETORCH.
• /chapterIV/envfiles contains the .env files used to deploy the SUTS in the RETORCH's schedullings and the baseline
• /chapterIV/monitoring contains the docker-compose to deploy and monitor the SUT as well as the python scripting to convert the prometheus API output to a Excel-format compatible output.

Monitoring

The monitorization of the E2E test suite execution is done through prometheus and CAdvisor. After the execution plan is executed, we got the memory and processor usage thrrough the prometheus API with the following requests:

Memory:

url http://[URL]:[PORT]/api/v1/query_range?query=rate(container_cpu_usage_seconds_total{id="/"}[1m])&start=[STARTUNIXTIME]&end=[ENDUNIXTIME]&step=1

Processor:

url http://[URL]:[PORT]/api/v1/query_range?query=container_memory_usage_bytes{id="/"}/1024/1024/1024&start=[STARTUNIXTIME]&end=[ENDUNIXTIME]&step=1

CHAPTER V: RETORCH* - Optimizing the End-to-End Execution in the Cloud

The data and the code necessary to replicate the evaluation was made accesible in GIIS repository on 10th August 2023. It can be accessed in the following repository: Replication package for 'RETORCH*: A Cost and Resource aware Model for E2E Testing in the Cloud'

CHAPTER VI: Easing E2E testing process using Large Language Models (LLMs).

The data and the code necessary to replicate the evaluation was made accesible in GIIS repository on 20th September 2024. It can be accessed in the following repository: Replication package for 'Exploratory study of the usefulness of LLMs in System testing'', the data is available in the Zenodo repository.

Citing this work

Resource Characterization and Optimization for End-to-End Testing

• Cristian Augusto, "Resource Characterization and Optimization for End-to-End Testing", University of Oviedo, pp. 1–TO-DO, Gijón,Asturias (Spain), TO-DO 2025, Thesis available - Download citation

RETORCH: an approach for resource-aware orchestration of end-to-end test cases:

• Cristian Augusto, Jesús Morán, Antonia Bertolino, Claudio de la Riva, and Javier Tuya, “RETORCH: an approach for resource-aware orchestration of end-to-end test cases”, Software Quality Journal, vol. 28, no. 3, 2020. https://doi.org/10.1007/s11219-020-09505-2 - Full Article available - Authors version - Download citation

RETORCH*: A Cost and Resource aware Model for E2E Testing in the Cloud:

• Cristian Augusto, Jesús Morán, Antonia Bertolino, Claudio de la Riva, and Javier Tuya, “RETORCH: A Cost and Resource aware Model for E2E Testing in the Cloud”, *Journal of Systems and Software, vol. 221 , pages. 112237, 2025. https://doi.org/10.1016/j.jss.2024.112237 - Full Paper available - Authors version - Download citation

Exploratory study of the usefulness of LLMs in System testing

• Cristian Augusto, Jesús Morán, Antonia Bertolino, Claudio de la Riva, and Javier Tuya, "Exploratory study of the usefulness of LLMs in System testing", Proceedings of the 36th International Conference on Testing Software and Systems, pp. 1–17, Springer Cham, 2025, https://doi.org/10.1007/978-3-031-80889-0_17 - Full Article available - Authors version - Download citation

Acknowledgments

This work was supported in part by the project PID2022-137646OB-C32 under Grant MCIN/ AEI/10.13039/501100011033/FEDER, UE (EQUAVEL) and the project TestBUS (PID2019-105455GB-C32) supported by the Ministry of Science and Innovation (SPAIN)