env2sif

Introduction

env2sif is a tool made to alleviate creation and editing of Singularity images on a HPC servers where users do not have sudo rights.

Tool summary

This tool enables you to: - create conda or python virtual environments based on environment scripts (yml for conda, txt for python) - define operating system and version you wish to build your image upon (Ubuntu, Debian, CentOS, Alpine) - define conda and python versions you wish to install in your image - add packages to an existing conda/python image in a form of yml/txt file or comma separated list (e.g. pandas,numpy) - delete packages from an existing conda/python in a form of comma separated list (e.g. pandas,numpy)

Usage

Software requirements

Requirements:

• python 3
• singularity-ce >= 3.10.0
  
• --fakeroot rights for Singularity (reed more about fakeroot and giving fakeroot rights here)

Input files

There are three types of input files supported by this tool:

environment.yml:

name: <environment_name> channels: - <channel1_name> - <channel2_name> dependencies: - <package1>=<package1_version> - <package2>=<package2_version>

environment.txt:

<package1>==<package1_version> <package2>==<package2_version>

image.sif

Note Input singularity image that you wish to edit. It is crucial not to forget .sif extension in order for the tool to recognize the input type.

How to run the tool

If you wish to create virtual environment from yml/txt file, download the tool and run it as follows...

bash $ python env2sif.py --input_file <YOUR_INPUT_YML_OR_TXT_FILE> --output_file <YOUR_OUTPUT_SIF_FILE> --environment_type <CONDA_OR_PYTHON>

If you wish to edit already existing singularity image, download the tool and run it as follows... bash $ python env2sif.py --input_file <YOUR_INPUT_SIF_FILE> --output_file <YOUR_INPUT_SIF_FILE> --environment_type <CONDA_OR_PYTHON> --add <YML/TXT_FILE_OR_PACKAGE_LIST> --delete <PACKAGE_LIST>

If you wish to edit already existing singularity image and save the resulting environment in another image, download the tool and run it as follows... bash $ python env2sif.py --input_file <YOUR_INPUT_SIF_FILE> --output_file <YOUR_OUTPUT_SIF_FILE> --environment_type <CONDA_OR_PYTHON> --add <YML/TXT_FILE_OR_PACKAGE_LIST> --delete <PACKAGE_LIST>

If you wish to check installed System, Conda, Python versions and installed Conda/Python packages in created image, run the following... bash $ sigularity test <SIF_FILE>

See help message for all of the available options when running the pipeline:

``` $ python env2sif.py --help usage: env2sif.py [-h] -i INPUTFILE -o OUTPUTFILE -e ENVIRONMENTTYPE [-a ADD] [-d DELETE] [-t TEMPLATE] [-s SYSTEM] [-v SYSTEMVERSION] [-r] [-n] [-m MICROMAMBAVERSION] [-p PYTHONVERSION]

Create Singularity container from environment file or edit existing image

options: -h, --help show this help message and exit -i INPUTFILE, --inputfile INPUTFILE Specify environment file (.yml for conda, .txt for python) or an existing .sif image that you would like to edit -o OUTPUTFILE, --outputfile OUTPUTFILE Specify output container file path -e ENVIRONMENTTYPE, --environmenttype ENVIRONMENTTYPE Choose between conda or python environment build -a ADD, --add ADD Add new packages to an existing environment in a form of .yml file (.txt for python) or comma separated list of packages -d DELETE, --delete DELETE Delete packages from an existing environment in a form of comma separated list of packages -t TEMPLATE, --template TEMPLATE Specify building template file (default ../templates/condapytemplate.def) -s SYSTEM, --system SYSTEM Specify container system from CentOS, Ubuntu, Debian or Alpine (default Alpine) -v SYSTEMVERSION, --systemversion SYSTEMVERSION Specify container system version (default Ubuntu focal, CentOS 8, Debian bookworm and Alpine 3.18) -r, --systemremove Remove system image with installed python or micromamba versions -n, --notslim Do not install slim version of the system -m MICROMAMBAVERSION, --micromambaversion MICROMAMBAVERSION Specify container micromamba version that will install conda environment (default 1.5.7) -p PYTHONVERSION, --pythonversion PYTHONVERSION Specify container python version (default 3.12.2) ```

Tool output

image.sif

Note System images with installed conda and python versions are saved in ../images folder and reused in future builds. This ensures better reproducibility since public repositories might change their availability. You can disable saving system images by adding -r/--system_remove parameter when using env2sif.py.

Authors & Acknowledgements

The env2sif pipeline was originally developed by Ivan Baksic at TRON - Translational Oncology at the Medical Center of the Johannes Gutenberg University Mainz gGmbH (non-profit).

Maintenance is now lead by Ivan Baksic.

Main developers:

• Ivan Baksic
  

Contributing & Support

If you would like to contribute to this tool, please see the contributing guidelines.

Please report issues using the issue tracker of GitHub.

Citations

An list of references for the tools used by the tool can be found in the CITATIONS.md file.

CHANGELOG

• CHANGELOG