https://github.com/amir22010/transformer-xl
Science Score: 10.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
○codemeta.json file
-
○.zenodo.json file
-
○DOI references
-
✓Academic publication links
Links to: arxiv.org -
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (6.7%) to scientific vocabulary
Last synced: 10 months ago
·
JSON representation
Repository
Basic Info
- Host: GitHub
- Owner: Amir22010
- License: apache-2.0
- Language: Python
- Default Branch: master
- Size: 112 KB
Statistics
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
- Releases: 0
Fork of kimiyoung/transformer-xl
Created about 7 years ago
· Last pushed about 7 years ago
https://github.com/Amir22010/transformer-xl/blob/master/
# Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context This repository contains the code in both **PyTorch** and **TensorFlow** for our paper >[Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](http://arxiv.org/abs/1901.02860) >Zihang Dai\*, Zhilin Yang\*, Yiming Yang, Jaime Carbonell, Quoc V. Le, Ruslan Salakhutdinov (*: equal contribution) >Preprint 2018 ## TensorFlow - The source code is in the `tf/` folder, supporting (1) single-node multi-gpu training, and (2) multi-host TPU training. - Besides the source code, we also provide pretrained "TensorFlow" models with state-of-the-art (SoTA) performances reported in the paper. - Please refer to `tf/README.md` for details. ## PyTorch - The source code is in the `pytorch/` folder, supporting single-node multi-gpu training via the module `nn.DataParallel`. - Please refer to `pytorch/README.md` for details. ## Results Transformer-XL achieves new state-of-the-art results on multiple language modeling benchmarks. Transformer-XL is also the first to break through the 1.0 barrier on char-level language modeling. Below is a summary. Method | enwiki8 | text8 | One Billion Word | WT-103 | PTB (w/o finetuning) -- | -- | -- | -- | -- | -- Previous Best | 1.06 | 1.13 | 23.7 | 20.5 | 55.5 Transformer-XL | **0.99** | **1.08** | **21.8** | **18.3** | **54.5** ## Acknowledgement A large portion of the `getdata.sh` script comes from the [awd-lstm](https://github.com/salesforce/awd-lstm-lm/) repo. Happy Language Modeling :)
Owner
- Name: Amir Khan
- Login: Amir22010
- Kind: user
- Location: India
- Repositories: 3
- Profile: https://github.com/Amir22010
working on developing a state of art AI solutions mainly in computer vision, chat bots and nlp domain. building an awesome AI as a professional developer 😍.