https://github.com/cctry/e.t.
Science Score: 13.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
○codemeta.json file
-
○.zenodo.json file
-
✓DOI references
Found 1 DOI reference(s) in README -
○Academic publication links
-
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (5.8%) to scientific vocabulary
Last synced: 10 months ago
·
JSON representation
Repository
Basic Info
- Host: GitHub
- Owner: cctry
- Language: Cuda
- Default Branch: main
- Size: 15.6 KB
Statistics
- Stars: 6
- Watchers: 1
- Forks: 3
- Open Issues: 2
- Releases: 0
Created over 4 years ago
· Last pushed over 4 years ago
https://github.com/cctry/E.T./blob/main/
# E.T.
## Intro
This repository is associated with the paper published in SC21.
> E.T. Re-Thinking Self-Attention for Transformer Models on GPUs
It contains some implemented kernels mentioned in the paper and a few examples of encoder.
## Platform
Tested on NVIDIA V100S GPU with CUDA 11.4.
## Example
There are three examples of encoders in ```test```, all of which use random data.
1. On-the-fly attention with tensor-tile pruned linear transformations (```encoder_tile_test```)
2. Attention-aware pruning with pruned self-attention (```encoder_prune_test```)
3. Sequence-aware optimized encoder (```encoder_length_test```)
## build
```
mkdir build && cd build
cmake ..
make -j
```
## Cite
```
@inproceedings{,
author = {Chen, Shiyang and Huang, Shaoyi and Pandey, Santosh and Li, Bingbing and Gao, Guang R. and Zheng, Long and Ding, Caiwen and Liu, Hang},
title = {E.T.: Re-Thinking Self-Attention for Transformer Models on GPUs},
year = {2021},
isbn = {9781450384421},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
doi = {10.1145/3458817.3476138},
booktitle = {Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis},
articleno = {25},
numpages = {18},
location = {St. Louis, Missouri},
series = {SC '21}
}
```
Owner
- Login: cctry
- Kind: user
- Company: Rutgers University
- Repositories: 25
- Profile: https://github.com/cctry