Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/compvis/iin

A Disentangling Invertible Interpretation Network

https://github.com/compvis/iin

Science Score: 10.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ○
    codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ✓
    Academic publication links
    Links to: arxiv.org
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (6.6%) to scientific vocabulary
Last synced: 9 months ago · JSON representation

Repository

A Disentangling Invertible Interpretation Network

Basic Info
Statistics
  • Stars: 123
  • Watchers: 7
  • Forks: 17
  • Open Issues: 10
  • Releases: 0
Created about 6 years ago · Last pushed about 5 years ago
Metadata Files
Readme License

README.md

A Disentangling Invertible Interpretation Network for Explaining Latent Representations

PyTorch code accompanying the CVPR 2020 paper

A Disentangling Invertible Interpretation Network for Explaining Latent Representations
Patrick Esser*, Robin Rombach*, Björn Ommer
* equal contribution

teaser
arXiv | BibTeX | Project Page

Table of Contents

Requirements

A suitable conda environment named iin can be created and activated with:

conda env create -f environment.yaml conda activate iin

Optionally, you can then also conda install tensorflow-gpu=1.14 to speed up FID evaluations.

Data

MNIST, FashionMNIST and CIFAR10 will be downloaded automatically the first time they are used and CelebA will prompt you to download it. The content of each dataset can be visualized with

edexplore --dataset iin.data.<dataset>

where <dataset> is one of MNISTTrain, MNISTTest, FashionMNISTTrain, FashionMNISTTest, CIFAR10Train, CIFAR10Test, CelebATrain, CelebATest, FactorCelebATrain, FactorCelebATest, ColorfulMNISTTrain, ColorfulMNISTTest, SingleColorfulMNISTTrain, SingleColorfulMNISTTest.

Training

Autoencoders

To train autoencoders, run

edflow -b configs/<dataset>_ae.yaml -t

where <dataset> is one of mnist, fashionmnist, cifar, celeba, cmnist. To enable logging to wandb, adjust configs/project.yaml and add it to above command:

edflow -b configs/<dataset>_ae.yaml configs/project.yaml -t

Classifiers

To train a classifier on ColorfulMNIST, run

edflow -b configs/cmnist_clf.yaml -t

Once you have a checkpoint, you can estimate factor dimensionalities using

edflow -b configs/cmnist_clf.yaml configs/cmnist_dimeval.yaml -c <path to .ckpt>

For the pretrained classifier, this gives

[INFO] [dim_callback]: estimated factor dimensionalities: [22, 11, 31]

and to compare this to an autoencoder, run

edflow -b configs/cmnist_ae.yaml configs/cmnist_dimeval.yaml -c <path to cmnist ae .ckpt>

which gives

[INFO] [dim_callback]: estimated factor dimensionalities: [13, 17, 34]

Invertible Interpretation Networks

Unsupervised on AE

To train unsupervised invertible interpretation networks, run

edflow -b configs/<dataset>_iin.yaml [configs/project.yaml] -t

where <dataset> is one of mnist, fashionmnist, cifar, celeba. If, instead of using one of the pretrained models, you trained an autoencoder yourself, adjust the first_stage config section accordingly.

Supervised

For supervised, disentangling IINs, run

edflow -b configs/<dataset>_diin.yaml [configs/project.yaml] -t

where <dataset> is one of cmnist or celeba, or run

edflow -b configs/cmnist_clf_diin.yaml [configs/project.yaml] -t

to train a dIIN on top of a classifier, with factor dimensionalities as estimated above (dimensionalities of factors can be adjusted via the Transformer/factor_config configuration entry).

Evaluation

Evaluations run automatically after each epoch of training. To start an evaluation manually, run

edflow -p logs/<log_folder>/configs/<config>.yaml

and, optionally, add -c <path to checkpoint> to evaluate a specific checkpoint instead of the last one.

Pretrained Models

Download logs.tar.gz (~2.2 GB) and extract the pretrained models:

tar xzf logs.tar.gz

Results

Using spectral normalization for the discriminator, this code slightly improves upon the values reported in Tab. 2 of the paper.

| Dataset | Checkpoint | FID | |--------------|------------|--------| | MNIST | 105600 | 5.252 | | FashionMNIST | 110400 | 9.663 | | CelebA | 84643 | 19.839 | | CIFAR10 | 32000 | 38.697 |

Full training logs can be found on Weights & Biases.

BibTeX

@inproceedings{esser2020invertible, title={A Disentangling Invertible Interpretation Network for Explaining Latent Representations}, author={Esser, Patrick and Rombach, Robin and Ommer, Bj{\"o}rn}, booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition}, year={2020} }

Owner

  • Name: CompVis - Computer Vision and Learning LMU Munich
  • Login: CompVis
  • Kind: organization
  • Email: assist.mvl@lrz.uni-muenchen.de
  • Location: Germany

Computer Vision and Learning research group at Ludwig Maximilian University of Munich (formerly Computer Vision Group at Heidelberg University)

GitHub Events

Total
  • Issues event: 1
  • Issue comment event: 1
  • Fork event: 1
Last Year
  • Issues event: 1
  • Issue comment event: 1
  • Fork event: 1

Issues and Pull Requests

Last synced: over 1 year ago

All Time
  • Total issues: 11
  • Total pull requests: 0
  • Average time to close issues: about 1 year
  • Average time to close pull requests: N/A
  • Total issue authors: 10
  • Total pull request authors: 0
  • Average comments per issue: 0.36
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 0
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • westream (2)
  • xuedue (1)
  • san9569 (1)
  • vjysd (1)
  • SpiritBear000 (1)
  • WenBingo (1)
  • MKowal2 (1)
  • JiafengtTang (1)
  • dal-code (1)
  • pandsia2007 (1)
  • LittlelittleZ (1)
Pull Request Authors
Top Labels
Issue Labels
Pull Request Labels

Dependencies

environment.yaml pypi
  • albumentations ==0.4.3
  • opencv-python ==4.1.2.30
  • pudb ==2019.2
setup.py pypi
  • numpy >=1.17
  • scipy >=1.0.1
  • torch >=1.4.0
  • torchvision >=0.5