https://github.com/christophreich1996/hypermixer
PyTorch reimplementation of the paper "HyperMixer: An MLP-based Green AI Alternative to Transformers" [arXiv 2022].
Science Score: 10.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
○codemeta.json file
-
○.zenodo.json file
-
○DOI references
-
✓Academic publication links
Links to: arxiv.org -
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (4.8%) to scientific vocabulary
Keywords
computer-vision
deep-learning
machine-learning
mixer
nlp
Last synced: 10 months ago
·
JSON representation
Repository
PyTorch reimplementation of the paper "HyperMixer: An MLP-based Green AI Alternative to Transformers" [arXiv 2022].
Basic Info
- Host: GitHub
- Owner: ChristophReich1996
- License: mit
- Language: Python
- Default Branch: master
- Homepage: https://arxiv.org/abs/2203.03691
- Size: 5.86 KB
Statistics
- Stars: 16
- Watchers: 4
- Forks: 0
- Open Issues: 0
- Releases: 0
Topics
computer-vision
deep-learning
machine-learning
mixer
nlp
Created over 4 years ago
· Last pushed about 4 years ago
https://github.com/ChristophReich1996/HyperMixer/blob/master/
# HyperMixer: An MLP-based Green AI Alternative to Transformers
[](https://github.com/ChristophReich1996/HyperMixer/blob/master/LICENSE)
This is a very quick (and maybe wrong) PyTorch reimplementation of the [HyperMixer](https://arxiv.org/pdf/2203.03691.pdf).
# Usage
```python
import torch
from hyper_mixer import HyperMixerBlock
hyper_mixer_block = HyperMixerBlock(dim=32)
input = torch.rand(3, 256, 32)
output = hyper_mixer_block(input)
print(output.shape)
```
# Reference
```bibtex
@article{Liu2021,
title={{HyperMixer: An MLP-based Green AI Alternative to Transformers}},
author={Mai, Florian and Pannatier, Arnaud and Fehr, Fabio and Chen, Haolin and Marelli, Francois
and Fleuret, Francois and Henderson, James},
journal={arXiv preprint arXiv:2203.03691},
year={2022}
}
```
Owner
- Name: Christoph Reich
- Login: ChristophReich1996
- Kind: user
- Location: Germany
- Company: Technical University of Munich
- Website: christophreich1996.github.io
- Twitter: ChristophR1996
- Repositories: 41
- Profile: https://github.com/ChristophReich1996
ELLIS Ph.D. Student @ Technical University of Munich, Technische Universität Darmstadt & University of Oxford | Prev. NEC Labs
GitHub Events
Total
Last Year
Dependencies
requirements.txt
pypi
- pytorch >=1.7.0
- timm >=0.4.12