https://github.com/crc-fonda/mmlib
Science Score: 23.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
○codemeta.json file
-
○.zenodo.json file
-
✓DOI references
Found 1 DOI reference(s) in README -
✓Academic publication links
Links to: zenodo.org -
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (11.9%) to scientific vocabulary
Last synced: 10 months ago
·
JSON representation
Repository
Basic Info
- Host: GitHub
- Owner: CRC-FONDA
- License: mit
- Language: Python
- Default Branch: master
- Size: 19.9 MB
Statistics
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
- Releases: 0
Created about 3 years ago
· Last pushed about 3 years ago
https://github.com/CRC-FONDA/mmlib/blob/master/
Efficiently Managing Deep Learning Models in a Distributed Environment
This repository contains the code to our EDBT '22 paper.
# MMlib MMlib is a library that implements different approaches to save and recover models. Approach names: - baseline approach - implemented by the `BaselineSaveService` - parameter update approach - implemented by `WeightUpdateSaveService` (set `improved_version=False`) - improved parameter update approach - implemented by `WeightUpdateSaveService` (set `improved_version=True`) - provenance approach - implemented by `ProvenanceSaveService` Next to the approaches to save and recover models we also implemented a **probing tool** - the corresponding code is in `probe.py` ## Examples - For examples on how to use MMlib and the probing tool checkout the [examples](examples) directory. ## Installation ### Option 1: Docker - **Requirements**: Docker installed - **Build Library** - clone this repo - run the script `generate-archives-docker.sh` - it runs a docker container and builds the *mmlib* in it - the created `dist` directory is copied back to repository root - it contains the `.whl` file that can be used to install the library with pip (see below) - **Install** - to install mmlib run: `pip install/dist/mmlib-0.0.1-py3-none-any.whl` ### Option 2: Local Build - **Requirements**: Python 3.8 and Python `venv` - **Build Library** - run the script `generate-archives.sh` - it creates a virtual environment, activates it, and installs all requirements - afterward it builds the library, and a `dist` directory containing the `.whl` file is created - **Install** - to install mmlib run: `pip install /dist/mmlib-0.0.1-py3-none-any.whl` ## Cite Our Work If you use MMlib or insights from the paper, please cite us. ```bibtex @inproceedings{strassenburg_2022_mmlib, author = {Nils Strassenburg and Ilin Tolovski and Tilmann Rabl}, title = {Efficiently Managing Deep Learning Models in a Distributed Environment}, booktitle = {Proceedings of the 25th International Conference on Extending Database Technology (EDBT 2022) Edinburgh, UK, March 29 - April 1}, pages = {234--246}, publisher = {OpenProceedings.org}, year = {2022}, doi = {10.48786/edbt.2022.12} } ``` ## Experimental Data [https://zenodo.org/record/8131544/files/mmlib-data.tar.gz?download=1](https://zenodo.org/record/8131544/files/mmlib-data.tar.gz?download=1)
Owner
- Name: CRC-FONDA
- Login: CRC-FONDA
- Kind: organization
- Repositories: 6
- Profile: https://github.com/CRC-FONDA