Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/chen-yang-liu/mlat

Official pytorch implementation of paper "Remote Sensing Image Captioning Based on Multi-Layer Aggregated Transformer"

https://github.com/chen-yang-liu/mlat

Science Score: 23.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ○
    codemeta.json file
  • ○
    .zenodo.json file
  • ✓
    DOI references
    Found 1 DOI reference(s) in README
  • ✓
    Academic publication links
    Links to: ieee.org
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (8.3%) to scientific vocabulary
Last synced: 10 months ago · JSON representation

Repository

Official pytorch implementation of paper "Remote Sensing Image Captioning Based on Multi-Layer Aggregated Transformer"

Basic Info
  • Host: GitHub
  • Owner: Chen-Yang-Liu
  • Language: Python
  • Default Branch: main
  • Homepage:
  • Size: 63.7 MB
Statistics
  • Stars: 25
  • Watchers: 2
  • Forks: 1
  • Open Issues: 2
  • Releases: 0
Created over 4 years ago · Last pushed about 3 years ago

https://github.com/Chen-Yang-Liu/MLAT/blob/main/ 

# Remote Sensing Image Captioning Based on Multi-layer Aggregated Transformer
Here, we provide the pytorch implementation of the paper: "Remote Sensing Image Captioning Based on Multi-Layer Aggregated Transformer". 

For more information, please see our published paper in [[IEEE](https://ieeexplore.ieee.org/document/9709791) | [Lab Server](http://levir.buaa.edu.cn/publications/Captioning-Based-on-Multilayer-Aggregated-Transformer.pdf)]  ***(Accepted by GRSL 2022)***

![MLAT](images/MLAT.png)

## Train
Firstly, download the RSICD dataset. Then preprocess the data as follows:
```python
python create_input_files.py --karpathy_json_path ./RSICD_captions/dataset_rsicd.json --image_folder ./RSICD_captions/images/
```
After that, you can find some resulted metadata files in `./data/`
Secondly, train model: 
```python
python train5.py
```
Note: During training, beam search is not used when computing scores on the validation set. To get the evaluation score with the beam search strategy, run the following command to get the score on the test set:
## Test
```python
python eval.py
```

## Citation: 
```
@ARTICLE{9709791,
  author={Liu, Chenyang and Zhao, Rui and Shi, Zhenwei},
  journal={IEEE Geoscience and Remote Sensing Letters}, 
  title={Remote-Sensing Image Captioning Based on Multilayer Aggregated Transformer}, 
  year={2022},
  volume={19},
  number={},
  pages={1-5},
  doi={10.1109/LGRS.2022.3150957}}
```
## Reference:
Thanks to the following repository:
[a-PyTorch-Tutorial-to-Image-Captioning](https://github.com/sgrvinod/a-PyTorch-Tutorial-to-Image-Captioning.git)

Owner

  • Name: Liu Chenyang
  • Login: Chen-Yang-Liu
  • Kind: user
  • Location: Beijing

Liu Chenyang

GitHub Events

Total
  • Issues event: 5
  • Watch event: 3
  • Issue comment event: 4
  • Fork event: 2
Last Year
  • Issues event: 5
  • Watch event: 3
  • Issue comment event: 4
  • Fork event: 2