GrokLST: Towards High-Resolution Benchmark and Toolkit for Land Surface Temperature Downscaling

Qun Dai, Chunyang Yuan, Yimian Dai, Yuxuan Li, Xiang Li, Kang Ni, Jianhui Xu, Xiangbo Shu, Jian Yang

Paper link: GrokLST: Towards High-Resolution Benchmark and Toolkit for Land Surface Temperature Downscaling

This repository is the official site for "GrokLST: Towards High-Resolution Benchmark and Toolkit for Land Surface Temperature Downscaling".

Abstract

Land Surface Temperature (LST) is a critical parameter for environmental studies, but obtaining high-resolution LST data remains challenging due to the spatio-temporal trade-off in satellite remote sensing. Guided LST downscaling has emerged as a solution, but current methods often neglect spatial non-stationarity and lack a open-source ecosystem for deep learning methods. To address these limitations, we propose the Modality-Conditional Large Selective Kernel (MoCoLSK) Networks, a novel architecture that dynamically fuses multi-modal data through modality-conditioned projections. MoCoLSK re-engineers our previous LSKNet to achieve a confluence of dynamic receptive field adjustment and multi-modal feature integration, leading to enhanced LST prediction accuracy. Furthermore, we establish the GrokLST project, a comprehensive open-source ecosystem featuring the GrokLST dataset, a high-resolution benchmark, and the GrokLST toolkit, an open-source PyTorch-based toolkit encapsulating MoCoLSK alongside 40+ state-of-the-art approaches. Extensive experimental results validate MoCoLSK's effectiveness in capturing complex dependencies and subtle variations within multispectral data, outperforming existing methods in LST downscaling. Our code, dataset, and toolkit are available at https://github.com/GrokCV/GrokLST.

Preface

The task of land surface temperature downscaling is similar to super-resolution tasks in the field of computer vision (CV). Researchers working on image super-resolution may want to take note of this! Currently, there are not many teams using deep learning methods for land surface temperature downscaling (most still rely on traditional methods), and there are almost no publicly available datasets for training deep learning models. This is considered a relatively underdeveloped field. However, land surface temperature downscaling has significant practical implications, especially in environmental science, climate research, agriculture, urban planning, and other areas. Therefore, this field holds great potential for development and real-world impact.

In light of this, we have proposed the first high-resolution land surface temperature downscaling dataset, GrokLST, along with the corresponding GrokLST toolkit. Our toolkit includes over 40 downscaling (or super-resolution) methods, covering both single-image super-resolution (SISR) and guided-image super-resolution (GISR) techniques. We hope our work can contribute to advancing the field!

Introduction

GrokLST DATASET DOWNLOAD at:

• Baidu Netdisk
• OneDrive

MoCoLSK: Modality-Conditional Large Selective Kernel Module

GrokLST Toolkit

• Abstract
• Preface
• Introduction
• Installation
  • Step 1: Create a conda environment
  • Step 2: Install PyTorch
  • Step 3: Install OpenMMLab 2.x Codebases
  • Step 4: Install groklst
• Folder Introduction
  • configs Folder
  • data Folder and GrokLST Dataset Introduction
  • groklst Folder (Core Code)
  • tools Folder
• Training
  • Single GPU Training
  • Multi-GPU Training
• Testing
  • Single GPU Testing
  • Multi-GPU Testing
• BUG LOG
• Acknowledgement
• Citation
• License

Installation

Step 1: Create a conda environment

shell conda create --name groklst python=3.9 -y source activate groklst

Step 2: Install PyTorch

```shell

CUDA 12.1

conda install pytorch==2.1.2 torchvision==0.16.2 torchaudio==2.1.2 pytorch-cuda=12.1 -c pytorch -c nvidia -y ```

Step 3: Install OpenMMLab 2.x Codebases

```shell

openmmlab codebases

pip install -U openmim dadaptation chardet --no-input

mim install mmengine "mmcv==2.1.0" "mmdet>=3.0.0" "mmpretrain>=1.0.0rc7"

mim install mmengine "mmcv==2.1.0" "mmdet>=3.0.0" "mmsegmentation>=1.0.0" "mmpretrain>=1.0.0rc7" 'mmagic'

other dependencies

pip install -U ninja scikit-image --no-input pip install kornia==0.6.5 # 0.7.1 pip install albumentations==1.3.1 pip install diffusers==0.24.0 pip install huggingface_hub==0.25.2 pip install cupy-cuda12x ```

Folder Introduction

• Overview of the GrokLST toolkit folder

shell GrokLST configs (Configuration files) data (Datasets) groklst (Core code) tools (Training and testing tools) work_dirs (Stores models, log files, etc.)

configs Folder

```shell

- Structure and function

configs base datasets (Configurations for different datasets, including pipelines, dataloaders, and evaluators for train, val, and test) groklstdataset groklstdatasetx2-256-512sisr.py (SISR configuration, includes only LST without guidance data) groklstdatasetx2-256-512.py (GISR configuration, includes LST and guidance data) schedules (Configurations for iterations, optimizers, etc.) defaultruntime.py (Default runtime configuration) gisr (Guided Image Super-Resolution, GISR) ... mocolsk (Ours) mocolskx24xb1-10kgroklst.py mocolskx44xb1-10kgroklst.py mocolskx84xb1-10kgroklst.py ... sisr (Single Image Super-Resolution, SISR)

```

data Folder and GrokLST Dataset Introduction

1. Using the groklst dataset as an example to introduce its structure

shell data groklst (Each folder contains 641 mat files) 30m guidance (HR guidance) lst (LR LST) 60m guidance lst 120m guidance lst 240m guidance lst split (train:val:test=6:1:3) train.txt val.txt trainval.txt (tarin.txt + val.txt) test.txt

1. Using groklst / 30m as an example to explain the dataset:
2. guidance folder: Each mat file contains 10 fields of data, namely "dem", "deepblue", "blue", "green", "red", "vre", "nir", "ndmvi", "ndvi", and "ndwi";

• lst folder: This is the LST (Land Surface Temperature) data, where the 30m resolution LST data can be used as the GT label, and the LST data of other resolutions (60m, 120m, 240m) can be regarded as low-resolution data that needs super-resolution;

• split folder: This is the dataset partitioning strategy, with train:val=6:1:3 and trainval=7:3. Note that in our paper GrokLST: Towards High-Resolution Benchmark and Toolkit for Land Surface Temperature Downscaling, we use trainval.txt as the training set, and the data indexed in test.txt as the validation and test sets.

1. The GrokLST dataset is obtained by cropping the original large image data with a 0.5 overlap rate, as shown in the table below:

| Resolution | scale | crop size | crop step | h/w | guidance size (h,w,c) | | ---------- | ----- | --------- | --------- | --- | --------------------- | | 30m | - | 512 | 256 | 512 | 512x512x10 | | 60m | x2 | 256 | 128 | 256 | 256x256x10 | | 120m | x4 | 128 | 64 | 128 | 128x128x10 | | 240m | x8 | 64 | 32 | 64 | 64x64x10 |

groklst Folder (Core Code)

• Structure

shell groklst datasets transforms (Data processing pipelines) dropping_bands.py (Random band dropping) formatting_data.py (Data packaging) loading_data.py (Data loading) normalizing_data.py (Guidance data normalization) padding_bands.py (Band padding) groklst_dataset.py (GrokLSTDataset class) evaluation metrics (Various evaluation metrics) models data_preprocessors data_preprocessor.py (Data preprocessor) editors (All models are here!!!) ... mocolsk (Ours) ... losses loss_wrapper.py pixelwise_loss.py (Custom SmoothL1Loss) visualization custom_concat_visualizer.py (Custom visualizer) vis_backend.py visualizer.py

tools Folder

• Structure as follows

shell tools (Entry points for training and testing!!!) dist_test.sh (Distributed testing script) dist_train.sh (Distributed training script) test.py (Single GPU testing script) train.py (Single GPU training script)

Training

Single GPU Training

• Single GPU training command (default using GPU0)

```shell python tools/train.py ${CONFIG_FILE}

Example with MoCoLSK-Net

python tools/train.py configs/gisr/mocolsk/mocolskx84xb1-10k_groklst.py ```

Multi-GPU Training

• Multi-GPU training command (assuming a 4-GPU machine):

```shell

export CUDAVISIBLEDEVICES=0,1,2,3

PORT=29500 tools/disttrain.sh ${CONFIGFILE} 4

Example with MoCoLSK-Net

PORT=29500 tools/disttrain.sh configs/gisr/mocolsk/mocolskx84xb1-10kgroklst.py 4 ```

Testing

Single GPU Testing

• Single GPU testing command (default using GPU0):

```shell python tools/test.py ${CONFIGFILE} ${CHECKPOINTFILE}

Example with MoCoLSK-Net

python tools/test.py configs/gisr/mocolsk/mocolskx84xb1-10k_groklst.py your/model/path.pth ```

Multi-GPU Testing

• Multi-GPU testing command (assuming a 4-GPU machine):

```shell

export CUDAVISIBLEDEVICES=0,1,2,3

tools/disttest.sh ${CONFIGFILE} ${CHECKPOINT_FILE} 4

Example with MoCoLSK-Net

PORT=29500 tools/disttest.sh configs/gisr/mocolsk/mocolskx84xb1-10kgroklst.py your/model/path.pth 4 ```

BUG LOG

1. Mismatch Between State Dictionary Parameters and the Model Architecture During Checkpoint Loading.

Supported Algorithms

Downscaling / Super-Resolution Algorithms in GrokLST Toolkit

Acknowledgement

The authors would like to thank the International Research Center of Big Data for Sustainable Development Goals (CBAS) for kindly providing the SDGSAT-1 data.

Citation

If you use this toolbox or benchmark in your research, please cite this project.

bibtex @article{dai2024GrokLST, title={GrokLST: Towards High-Resolution Benchmark and Toolkit for Land Surface Temperature Downscaling}, author={Dai, Qun and Yuan, Chunyang and Dai, Yimian and Li, Yuxuan and Li, Xiang and Ni, Kang and Xu, Jianhui and Shu, Xiangbo and Yang, Jian}, year={2024}, journal={arXiv}, }

License

This project is released under the Attribution-NonCommercial 4.0 International.