Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/arogozhnikov/kronecker-attention-pytorch

Implementation of Kronecker Attention in Pytorch

https://github.com/arogozhnikov/kronecker-attention-pytorch

Science Score: 23.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ○
    codemeta.json file
  • ○
    .zenodo.json file
  • ✓
    DOI references
    Found 3 DOI reference(s) in README
  • ✓
    Academic publication links
    Links to: arxiv.org
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (5.1%) to scientific vocabulary
Last synced: 10 months ago · JSON representation

Repository

Implementation of Kronecker Attention in Pytorch

Basic Info
  • Host: GitHub
  • Owner: arogozhnikov
  • License: mit
  • Default Branch: master
  • Size: 126 KB
Statistics
  • Stars: 0
  • Watchers: 1
  • Forks: 1
  • Open Issues: 0
  • Releases: 0
Fork of lucidrains/kronecker-attention-pytorch
Created almost 6 years ago · Last pushed almost 6 years ago

https://github.com/arogozhnikov/kronecker-attention-pytorch/blob/master/ 



## Kronecker Attention Pytorch

Implementation of Kronecker Attention in Pytorch. Results look less than stellar, but if someone found some context where this architecture works well, please post in the issues and let everyone know.

## Install

```bash
$ pip install kronecker_attention_pytorch
```

## Usage

```python
import torch
from kronecker_attention_pytorch import KroneckerSelfAttention

attn = KroneckerSelfAttention(
    chan = 32,
    heads = 8,
    dim_heads = 64
)

x = torch.randn(1, 32, 256, 512)
attn(x) # (1, 32, 256, 512)
```

## Citations

```bibtex
@article{Gao_2020,
   title={Kronecker Attention Networks},
   url={http://dx.doi.org/10.1145/3394486.3403065},
   journal={Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining},
   publisher={ACM},
   author={Gao, Hongyang and Wang, Zhengyang and Ji, Shuiwang},
   year={2020},
   month={Aug}
}
```

Owner

  • Name: Alex Rogozhnikov
  • Login: arogozhnikov
  • Kind: user
  • Location: San Francisco
  • Company: Aperture Science

ML + Science, einops, scientific tools

GitHub Events

Total
Last Year