https://github.com/animesh/latentshift

Science Score: 10.0%

This score indicates how likely this project is to be science-related based on various indicators:

○
CITATION.cff file
○
codemeta.json file
○
.zenodo.json file
○
DOI references
✓
Academic publication links
Links to: arxiv.org
○
Academic email domains
○
Institutional organization owner
○
JOSS paper metadata
○
Scientific vocabulary similarity
Low similarity (7.2%) to scientific vocabulary

Last synced: 6 months ago · JSON representation

Repository

Basic Info

Host: GitHub
Owner: animesh
Language: Jupyter Notebook
Default Branch: main
Size: 14.9 MB

Statistics

Stars: 0
Watchers: 1
Forks: 0
Open Issues: 0
Releases: 0

Fork of ieee8023/latentshift

Created over 3 years ago · Last pushed over 1 year ago

https://github.com/animesh/latentshift/blob/main/

# Latent Shift - A Simple Autoencoder Approach to Counterfactual Generation


[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ieee8023/latentshift/blob/main/example.ipynb)

# The idea

Read the paper about Latent Shift: https://arxiv.org/abs/2102.09475

Watch a video: https://www.youtube.com/watch?v=1fxSDP8DheI

Read the paper about Counterfactual Alignment: https://arxiv.org/abs/2312.02186

The main diagram:
![latentshift.gif](docs/latentshift.gif)


## Animations/GIFs

| Smiling | Arched Eyebrows|
| ----------- | ----------- |
|  |   | 

|Mouth Slightly Open  | Young  |
| ----------- | ----------- |
|  |   | 

# Generating a transition sequence

For a predicting of `smiling`

![gen_sequence.png](docs/gen_sequence.png)

# Multiple different targets






  
# Comparison to traditional methods 

For a predicting of `pointy_nose`

![comparison.png](docs/comparison.png)

# Getting Started

```bash
$pip install latentshift
````


```python3
import latentshift
# Load classifier and autoencoder
model = latentshift.classifiers.FaceAttribute(download=True)
ae = latentshift.autoencoders.VQGAN(weights="faceshq", download=True)

# Load image
input = torch.randn(1, 3, 1024, 1024)

# Defining Latent Shift module
attr = captum.attr.LatentShift(model, ae)

# Computes counterfactual for class 3.
output = attr.attribute(input, target=3)
```