Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/chen-yang-liu/rsicc

Official LEVIR-CC dataset and Pytorch implementation for Remote Sensing Image Change Captioning With Dual-Branch Transformers: A New Method and a Large Scale Dataset

https://github.com/chen-yang-liu/rsicc

Science Score: 36.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ○
    .zenodo.json file
  • ✓
    DOI references
    Found 1 DOI reference(s) in README
  • ✓
    Academic publication links
    Links to: arxiv.org, scholar.google, ieee.org
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (11.5%) to scientific vocabulary

Keywords

remote-sensing-image-change-captioning
Last synced: 9 months ago · JSON representation

Repository

Official LEVIR-CC dataset and Pytorch implementation for Remote Sensing Image Change Captioning With Dual-Branch Transformers: A New Method and a Large Scale Dataset

Basic Info
  • Host: GitHub
  • Owner: Chen-Yang-Liu
  • Language: Python
  • Default Branch: main
  • Homepage:
  • Size: 67.2 MB
Statistics
  • Stars: 122
  • Watchers: 2
  • Forks: 7
  • Open Issues: 12
  • Releases: 0
Topics
remote-sensing-image-change-captioning
Created about 4 years ago · Last pushed about 2 years ago
Metadata Files
Readme

README.md

Remote Sensing Image Change Captioning With Dual-Branch Transformers: A New Method and a Large Scale Dataset

**[Chenyang Liu](https://chen-yang-liu.github.io/), [Rui Zhao](https://ruizhaocv.github.io), [Hao Chen](http://chenhao.in/), [Zhengxia Zou](https://scholar.google.com.hk/citations?hl=en&user=DzwoyZsAAAAJ), and [Zhenwei Shi*✉](https://scholar.google.com.hk/citations?hl=en&user=kNhFWQIAAAAJ)** ![visitors](https://visitor-badge.glitch.me/badge?page_id=Chen-Yang-Liu.RSICC.visitor) ![GitHub stars](https://badgen.net/github/stars/Chen-Yang-Liu/RSICC)

Share us a :star: if this repo does help

🥳 New

🔥 Our survey "Remote Sensing Temporal Vision-Language Models: A Comprehensive Survey": Arxiv || Github 🔥

LEVIR-CC Dataset

Download Link

RSICCfromer

Here, we provide the pytorch implementation of the paper: "Remote Sensing Image Change Captioning With Dual-Branch Transformers: A New Method and a Large Scale Dataset".

For more information, please see our published paper in [IEEE | Lab Server] (Accepted by TGRS 2022)

RSICCformer_structure

Installation and Dependencies

python git clone https://github.com/Chen-Yang-Liu/RSICC cd RSICC conda create -n RSICCformer_env python=3.6 conda activate RSICCformer_env pip install -r requirements.txt

Data preparation

Firstly, put the downloaded dataset in ./LEVIR_CC_dataset/. Then preprocess dataset as follows: python python create_input_files.py --min_word_freq 5 After that, you can find some resulted files in ./data/.

Besides, the resulted files can also be downloaded from here: [Google Drive | Baidu Pan (code:nq9y)]. Extract it to ./data/.

!NOTE: For a fair comparison, we suggest that future researchers ensure min_word_freq <= 5 or use our preprocessed data above with min_word_freq = 5.

Inference Demo

You can download our RSICCformer pretrained model——by [Google Drive | Baidu Pan (code:2fbc)]

After downloaded the pretrained model, you can put it in ./models_checkpoint/.

Then, run a demo to get started as follows: python python caption.py --img_A ./Example/A/train_000016.png --img_B ./Example/B/train_000016.png --path ./models_checkpoint/ After that, you can find the generated caption in ./eval_results/

Train

Make sure you performed the data preparation above. Then, start training as follows: python python train.py --data_folder ./data/ --savepath ./models_checkpoint/

Evaluate

python python eval.py --data_folder ./data/ --path ./models_checkpoint/ --Split TEST We recommend training 5 times to get an average score.

Citation:

@ARTICLE{9934924, author={Liu, Chenyang and Zhao, Rui and Chen, Hao and Zou, Zhengxia and Shi, Zhenwei}, journal={IEEE Transactions on Geoscience and Remote Sensing}, title={Remote Sensing Image Change Captioning With Dual-Branch Transformers: A New Method and a Large Scale Dataset}, year={2022}, volume={60}, number={}, pages={1-20}, doi={10.1109/TGRS.2022.3218921}}

Reference:

Thanks to the following repository: a-PyTorch-Tutorial-to-Image-Captioning

Owner

  • Name: Liu Chenyang
  • Login: Chen-Yang-Liu
  • Kind: user
  • Location: Beijing

Liu Chenyang

GitHub Events

Total
  • Issues event: 2
  • Watch event: 26
  • Issue comment event: 2
  • Fork event: 2
Last Year
  • Issues event: 2
  • Watch event: 26
  • Issue comment event: 2
  • Fork event: 2

Issues and Pull Requests

Last synced: about 2 years ago

All Time
  • Total issues: 12
  • Total pull requests: 0
  • Average time to close issues: about 15 hours
  • Average time to close pull requests: N/A
  • Total issue authors: 11
  • Total pull request authors: 0
  • Average comments per issue: 1.83
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 5
  • Pull requests: 0
  • Average time to close issues: about 1 hour
  • Average time to close pull requests: N/A
  • Issue authors: 5
  • Pull request authors: 0
  • Average comments per issue: 1.4
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • kamranayesh (2)
  • Aki-Tomoya (2)
  • shizizuoing (1)
  • ZhanYang-nwpu (1)
  • ShizhenChang (1)
  • sunyuxi (1)
  • Zhou2019 (1)
  • Cemm23333 (1)
  • yzygit1230 (1)
  • sel0nidas (1)
  • ruzcko (1)
  • leeleelee315 (1)
  • mynameismahu (1)
  • karthiksenthil2803 (1)
Pull Request Authors
Top Labels
Issue Labels
Pull Request Labels

Dependencies

requirements.txt pypi
  • Cython ==0.29.22
  • GitPython ==3.1.18
  • JPype1 ==1.2.1
  • Keras-Preprocessing ==1.1.2
  • Markdown ==3.3.4
  • Pillow ==7.2.0
  • PyJWT ==1.7.1
  • PySocks ==1.7.1
  • PyWavelets ==1.1.1
  • PyYAML ==5.3.1
  • Werkzeug ==1.0.1
  • absl-py ==0.12.0
  • aiohttp ==3.6.3
  • astor ==0.8.1
  • async-timeout ==3.0.1
  • atomicwrites ==1.4.0
  • attrs ==20.3.0
  • blinker ==1.4
  • brotlipy ==0.7.0
  • cached-property ==1.5.2
  • cachetools ==4.2.1
  • certifi ==2021.5.30
  • cffi ==1.14.2
  • chardet ==4.0.0
  • click ==7.1.2
  • cloudpickle ==1.6.0
  • colorama ==0.4.4
  • coverage ==5.5
  • cryptography ==3.4.7
  • cycler ==0.10.0
  • cytoolz ==0.11.0
  • dask ==2021.3.0
  • dataclasses ==0.8
  • decorator ==5.0.5
  • docker-pycreds ==0.4.0
  • easydict ==1.9
  • einops ==0.4.1
  • future ==0.18.2
  • gast ==0.2.2
  • gitdb ==4.0.9
  • google-auth ==1.28.0
  • google-auth-oauthlib ==0.4.4
  • google-pasta ==0.2.0
  • grpcio ==1.36.1
  • h5py ==3.1.0
  • idna ==2.10
  • idna-ssl ==1.1.0
  • imageio ==2.9.0
  • importlib-metadata ==3.7.3
  • iniconfig ==1.1.1
  • joblib ==1.0.1
  • kiwisolver ==1.1.0
  • matplotlib ==3.3.4
  • mkl-fft ==1.3.0
  • mkl-random ==1.1.1
  • mkl-service ==2.3.0
  • more-itertools ==8.7.0
  • multidict ==4.7.6
  • networkx ==2.4
  • nltk ==3.2.4
  • numpy ==1.19.5
  • oauthlib ==3.1.0
  • olefile ==0.46
  • opt-einsum ==3.1.0
  • packaging ==20.9
  • pandas ==1.1.3
  • pathtools ==0.1.2
  • pluggy ==0.13.1
  • progressbar2 ==3.53.1
  • promise ==2.3
  • protobuf ==3.15.7
  • psutil ==5.9.0
  • py ==1.10.0
  • pyOpenSSL ==20.0.1
  • pyasn1 ==0.4.8
  • pyasn1-modules ==0.2.8
  • pycparser ==2.20
  • pynvml ==11.4.1
  • pyparsing ==2.4.7
  • pyreadline ==2.1
  • pytest ==6.2.2
  • python-dateutil ==2.8.1
  • python-utils ==2.5.6
  • pytz ==2021.1
  • regex ==2021.4.4
  • requests ==2.25.1
  • requests-oauthlib ==1.3.0
  • rsa ==4.7.2
  • scikit-image ==0.15.0
  • scikit-learn ==0.24.1
  • scipy ==1.5.3
  • sentry-sdk ==1.5.5
  • shortuuid ==1.0.8
  • six ==1.15.0
  • smmap ==5.0.0
  • tensorboard ==2.4.0
  • tensorboard-plugin-wit ==1.8.0
  • tensorlayer ==2.2.3
  • termcolor ==1.1.0
  • thop ==0.0.31.post2005241907
  • threadpoolctl ==2.1.0
  • toml ==0.10.2
  • toolz ==0.11.1
  • torch ==1.10.0
  • torchstat ==0.0.7
  • torchsummaryX ==1.3.0
  • torchvision ==0.2.1
  • tornado ==6.1
  • tqdm ==4.19.9
  • typing-extensions ==3.7.4.3
  • urllib3 ==1.26.4
  • vit-pytorch ==0.32.2
  • wandb ==0.12.10
  • win-inet-pton ==1.1.0
  • wincertstore ==0.2
  • wordcloud ==1.8.2.2
  • wrapt ==1.12.1
  • yarl ==1.6.3
  • yaspin ==2.1.0
  • zipp ==3.4.1