HRDecoder: High-Resolution Decoder Network for Fundus Image Lesion Segmentation

Environment

Our code is based on MMsegmentation.

In this work, we used: - python=3.8.5 - pytorch=2.0.1 - mmsegmentation=0.16.0 - mmcv=1.7.1

The environment can be installed with: ``` conda create -n hrdecoder python=3.8.5 conda activate hrdecoder

pip install -r requirements.txt chmod u+x tools/* ```

Datasets

We carried out experiments on two public datasets, i.e. IDRiD and DDR. For IDRiD, please download the segmentation part from here. For DDR dataset, please download the segmentation part from here.

Please first change the original structure of datasets and organize like this: ``` IDRiD(segmentation part) |——images | |——train | | |——IDRiD01.jpg | | |——... | | |——IDRiD54.jpg | |——test |——labels | |——train | | |——EX | | | |——IDRiD01EX.tif | | | |——... | | | |——IDRiD54EX.tif | | |——HE | | |——SE | | |——MA | |——test | | |——EX | | |——HE | | |——SE | | |——MA

DDR(segmentation part) |——images | |——test | | |——007-1789-100.jpg | | |——... | | |——20170627170651362.jpg | |——val | |——train |——labels | |——test | | |——EX | | | |——007-1789-100.tif | | | |——... | | | |——20170627170651362.tif | | |——HE | | |——MA | | |——SE | |——val | | |——EX | | |——HE | | |——MA | | |——SE | |——train | | |——EX | | |——HE | | |——MA | | |——SE ```

Please note that the original GT with .tif format can not be directly used for training, so you can use the following command to convert the labels to .png format: python tools/convert_dataset/idrid.py python tools/convert_dataset/ddr.py Then the structure of dataset will be converted to: ``` IDRiD(segmentation part) |——images | |——train | | |——IDRiD01.jpg | | |——... | | |——IDRiD54.jpg | |——test |——labels | |——train | | |——IDRiD01.png | | |——... | | |——IDRiD54.png | |——test

DDR(segmentation part) |——images | |——test | | |——007-1789-100.jpg | | |——... | | |——20170627170651362.jpg | |——val | |——train |——labels | |——test | | |——007-1789-100.png | | |——... | | |——20170627170651362.png | |——val | |——train ```

Finally the structure of the project should be like this: HRDecoder |——configs |——mmseg |——tools |——data | |——IDRiD(segmentation part) | | |——images | | | |——train | | | |——test | | |——labels | | | |——train | | | |——test | |——DDR(segmentation part) | | |——images | | | |——train | | | |——val | | | |——test | | |——labels | | | |——train | | | |——val | | | |——test

Training and Testing

To train or test a model using MMsegmentation framework, you can use commands like this: ```

train

CUDAVISIBLEDEVICES=0,1,2,3 PORT=18940 tools/disttrain.sh configs/lesion/hrdecoderfcnhr48idrid2880x1920-slide.py 4 or CUDAVISIBLEDEVICES=0,1,2,3 PORT=18940 tools/disttrain.sh configs/lesion/efficient-hrdecoderfcnhr48idrid2880x1920-slide.py 4

test

CUDAVISIBLEDEVICES=4,5,6,7 PORT=17940 tools/disttest.sh configs/lesion/hrdecoderfcnhr48idrid2880x1920-slide.py workdirs/hrdecoderfcnhr48idrid2880x1920-slide/latest.pth 4 --eval mIoU or CUDAVISIBLEDEVICES=4,5,6,7 PORT=17940 tools/disttest.sh configs/lesion/efficient-hrdecoderfcnhr48idrid2880x1920-slide.py workdirs/efficient-hrdecoderfcnhr48idrid2880x1920-slide/latest.pth 4 --eval mIoU ```

The trained model will be stored to work_dirs/.

In this project, we supply two version of implemention: HRDecoder and Efficient-HRDecoder.

HRDecoder is the version utilized for presenting the results in our paper. While Efficient-HRDecoder is an improved version of HRDecoder to further reduce computational overhead and memory usage. We simply compress the dimension of the extracted features using a 1x1 convolutional layer right after the backbone. This helps save a lot of memory. The compression operation can be replaced with various multi-scale feature fusion methods, such as FPN, CATNet, APPNet, CARAFE++ and so on. We do not focus on this part but on the later HR representation learning module and HR fusion module, so we simply use upsampling and concat in HRDecoder, or use a 1x1 conv in Efficient-HRDecoder.

Models and Evaluation

We evaluate our method out IDRiD and DDR test set.

| Dataset | mIoU | mAUPR | download | | ------- | :----: | :----: | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | | IDRiD | 52.20 | 71.34 | config; model | | DDR | 32.24 | 49.24 | config; model |

For calculating FLOPs and FPs, you can use the following commands: ```

calculate FLOPs

CUDAVISIBLEDEVICES=0 python tools/getflops.py configs/lesion/hrdecoderfcnhr48ddr_2048.py

Calculate FPs

CUDAVISIBLEDEVICES=0 python tools/getfps.py configs/lesion/hrdecoderfcnhr48ddr_2048.py ``` Typically, we calculate FPs on DDR dataset.

Acknowledgements

Last, we thank these authors for sharing their ideas and source code: - MMsegmentation - HRNet - HRDA - CARAFE++ - CATNet - SenFormer - M2MRF

This manuscript was supported in part by the National Key Research and Development Program of China under Grant 2021YFF1201202, the Natural Science Foundation of Hunan Province under Grant 2024JJ5444 and 2023JJ30699, the Key Research and Development Program of Hunan Province under Grant 2023SK2029, the Changsha Municipal Natural Science Foundation under Grant kq2402227, and the Research Council of Finland (formerly Academy of Finland for Academy Research Fellow project) under Grant 355095. The authors wish to acknowledge High Performance Computing Center of Central South University for computational resources.