Data

Tools

Indexes

Applications

Experiments

Open Source Science

2D-VQ-AE-2

2D Vector-Quantized Auto-Encoder for compression of Whole-Slide Images in Histopathology

https://github.com/SURF-ML/2D-VQ-AE-2

Science Score: 44.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ✓
    CITATION.cff file
    Found CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (10.0%) to scientific vocabulary

Keywords

examode neural-image-compression pytorch pytorch-lightning vq-vae-2 whole-slide-imaging
Last synced: 7 months ago · JSON representation ·

Repository

2D Vector-Quantized Auto-Encoder for compression of Whole-Slide Images in Histopathology

Basic Info
  • Host: GitHub
  • Owner: SURF-ML
  • Language: Python
  • Default Branch: main
  • Homepage:
  • Size: 39 MB
Statistics
  • Stars: 16
  • Watchers: 3
  • Forks: 2
  • Open Issues: 4
  • Releases: 4
Topics
examode neural-image-compression pytorch pytorch-lightning vq-vae-2 whole-slide-imaging
Created almost 5 years ago · Last pushed over 1 year ago
Metadata Files
Readme Citation

README.md

2D-VQ-AE-2

2D Vector-Quantized Auto-Encoder for compression of Whole-Slide Images in Histopathology

How to run

Installation

See INSTALL.md.
We use PDM as python package manager (https://github.com/pdm-project/pdm).

Locally

set CAMELYON16_PATH and run train.py: bash CAMELYON16_PATH=<camelyon-path> python train.py

Lisa

set CAMELYON16_PATH, and append --multirun to automatically submit a sbatch job through submitit.
- If CAMELYON16_PATH is a folder, the dataloader loads the dataset over the network. - If CAMELYON16_PATH is a .tar, the file is copied to $SCRATCH of the allocated node, and the dataset is loaded locally. bash CAMELYON16_PATH=<camelyon-path> python train.py --multirun Change node type by overwriting the node config, e.g.: bash CAMELYON16_PATH=<camelyon-path> python train.py hydra/launcher/node@hydra.launcher=gpu_titanrtx --multirun

Results

Note on Mean Squared-Error results: input is channel-wise normalised to 0-mean, 1-std, using the following values, based on 10k patches: | | Red | Green | Blue | |------------------------|--------|--------|--------| | Mean | 0.7279 | 0.5955 | 0.7762 | | Standard Deviation | 0.2419 | 0.3083 | 0.1741 |

Top: original, bottom: reconstruction.
Input dimensionality: 256×256×3@0.5μm ordinal 8-bit, latent dimensionality: 32×32@16μm categorical 8-bit (i.e. 99.47% compression), 0.900 MSE.

image image

Input dimensionality: 512×512×3@0.25μm ordinal 8-bit, latent dimensionality: 32×32@16μm categorical 8-bit (i.e. 99.87% compression), 0.800 MSE.

9233614 9233614_2

Research

If this repository has helped you in your research we would value to be acknowledged in your publication.

Acknowledgement

This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 825292. This project is better known as the ExaMode project. The objectives of the ExaMode project are: 1. Weakly-supervised knowledge discovery for exascale medical data.
2. Develop extreme scale analytic tools for heterogeneous exascale multimodal and multimedia data.
3. Healthcare & industry decision-making adoption of extreme-scale analysis and prediction tools.

For more information on the ExaMode project, please visit www.examode.eu.

enter image description here enter image description here

Owner

  • Name: SURF-ML
  • Login: SURF-ML
  • Kind: organization
  • Email: hpml@surf.nl

Official repository of the SURF Machine Learning Team.

Citation (CITATION.cff) 

# This CITATION.cff file was generated with cffinit.
# Visit https://bit.ly/cffinit to generate yours today!

cff-version: 1.2.0
title: >-
  2D Vector-Quantized Auto-Encoder for compression of
  Whole-Slide Images in Histopathology 
message: >-
  If you use this software, please cite it using the
  metadata from this file.
type: software
authors:
  - given-names: Robert Jan
    email: robert-jan.schlimbach@surf.nl
    family-names: Schlimbach
    affiliation: SURF
    orcid: 'https://orcid.org/0000-0001-9031-3331'
repository-code: 'https://github.com/sara-nl/2D-VQ-AE-2'
url: 'https://www.examode.eu/'
keywords:
  - Neural Data Compression
  - Whole-Slide Imaging
  - Vector-Quantization
  - Examode
  - Auto-Encoder
license: MIT

GitHub Events

Total
Last Year

Issues and Pull Requests

Last synced: 7 months ago

All Time
  • Total issues: 5
  • Total pull requests: 8
  • Average time to close issues: 27 days
  • Average time to close pull requests: 16 days
  • Total issue authors: 1
  • Total pull request authors: 1
  • Average comments per issue: 0.4
  • Average comments per pull request: 1.0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 8
Past Year
  • Issues: 0
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • robogast (5)
Pull Request Authors
  • dependabot[bot] (8)
Top Labels
Issue Labels
Pull Request Labels
dependencies (8)

Dependencies

pyproject.toml pypi
  • albumentations >=1.1.0
  • git +https://github.com/sara-nl/hydra-2.0.git@179f7467ce274cce13c91ccc212b9e2212823718#egg=hydra-optuna-sweeper&subdirectory=plugins/hydra_optuna_sweeper
  • git +https://github.com/sara-nl/hydra-2.0.git@179f7467ce274cce13c91ccc212b9e2212823718#egg=hydra-submitit-launcher&subdirectory=plugins/hydra_submitit_launcher
  • h5py >=3.6.0
  • hydra @ git+https://github.com/sara-nl/hydra-2.0.git@179f7467ce274cce13c91ccc212b9e2212823718
  • matplotlib >=3.5.2
  • pytorch-lightning >=1.6.1
  • torch ==1.11.0+cu115
  • torchmetrics >=0.8.0
  • torchvision ==0.12.0+cu115
setup.py pypi