SSFPN: Selective Spatial Feature Pyramid Network for Remote Sensing Object Detection(later)

By Ziming Xu, Bin Ren, Kunhua Zhang

This project is based on mmrotate1.x.

Abstract

It is great important that feature fusion module generate multi-scale feature maps after backbone in object detection especially in Oriented small object detection. Although the FPN series of models have achieved promising results in general object detection, their application in remote sensing object detection remains under-explored. Following the prior knowledge that small oriented object need contextual information to be detected and various type of object need different long-range context. In this paper, we take these two reason into account and propose an intuitive and simple fusion module named Selective Spatial Feature Pyramid Network (SSFPN). SSFPN dynamically adjusts the large spatial receptive field across multi-scale feature maps, providing enhanced modeling of the contextual variations among different objects in remote sensing scenarios.

Environment

``` sys.platform: linux Python: 3.8.18 (default, Sep 11 2023, 13:40:15) [GCC 11.2.0] CUDA available: True MUSA available: False numpyrandomseed: 2147483648 GPU 0,1,2,3,4,5,6,7: NVIDIA GeForce RTX 3090 CUDAHOME: /usr/local/cuda NVCC: Not Available GCC: n/a PyTorch: 1.13.1 PyTorch compiling details: PyTorch built with: - GCC 9.3 - C++ Version: 201402 - Intel(R) oneAPI Math Kernel Library Version 2023.1-Product Build 20230303 for Intel(R) 64 architecture applications - Intel(R) MKL-DNN v2.6.0 (Git Hash 52b5f107dd9cf10910aaa19cb47f3abf9b349815) - OpenMP 201511 (a.k.a. OpenMP 4.5) - LAPACK is enabled (usually provided by MKL) - NNPACK is enabled - CPU capability usage: AVX2 - CUDA Runtime 11.7 - NVCC architecture flags: -gencode;arch=compute37,code=sm37;-gencode;arch=compute50,code=sm50;-gencode;arch=compute60,code=sm60;-gencode;arch=compute61,code=sm61;-gencode;arch=compute70,code=sm70;-gencode;arch=compute75,code=sm75;-gencode;arch=compute80,code=sm80;-gencode;arch=compute86,code=sm86;-gencode;arch=compute37,code=compute37 - CuDNN 8.5 - Magma 2.6.1 - Build settings: BLASINFO=mkl, BUILDTYPE=Release, CUDAVERSION=11.7, CUDNNVERSION=8.5.0, CXXCOMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXXFLAGS= -fabi-version=11 -Wno-deprecated -fvisibility-inlines-hidden -DUSEPTHREADPOOL -fopenmp -DNDEBUG -DUSEKINETO -DUSEFBGEMM -DUSEQNNPACK -DUSEPYTORCHQNNPACK -DUSEXNNPACK -DSYMBOLICATEMOBILEDEBUGHANDLE -DEDGEPROFILERUSEKINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wunused-local-typedefs -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow, LAPACKINFO=mkl, PERFWITHAVX=1, PERFWITHAVX2=1, PERFWITHAVX512=1, TORCHVERSION=1.13.1, USECUDA=ON, USECUDNN=ON, USEEXCEPTIONPTR=1, USEGFLAGS=OFF, USEGLOG=OFF, USEMKL=ON, USEMKLDNN=ON, USEMPI=OFF, USENCCL=ON, USENNPACK=ON, USEOPENMP=ON, USE_ROCM=OFF,

TorchVision: 0.14.1 OpenCV: 4.9.0 mmcv: 2.0.1 MMEngine: 0.10.3 mmdet: 3.0.0rc6 MMRotate: 1.0.0rc1+8b30525 ```

Install

shell conda create --name mmrotate1 python=3.8 -y conda activate mmrotate1 conda install pytorch==1.13.0 torchvision==0.14.0 torchaudio==0.13.0 pytorch-cuda=11.7 -c pytorch -c nvidia pip install -U openmim mim install mmengine mim install "mmcv==2.0.1" mim install "mmdet>3.0.0rc4,<3.1.0" git clone https://github.com/Atlantisming/SSFPN.git cd SSFPN pip install -v -e .

Dataset

SSFPN ├── data │ ├── DOTA │ │ ├── train │ │ ├── val │ │ ├── test

See more details for preparing datasets at mmrotate1.x tools.

Training

Single gpu for train:

shell CUDA_VISIBLE_DEVICES=0 python tools/train.py projects/LSKNet/configs/lsk_s_sspafpn_1x_dota_le90.py

Multiple gpus for train:

shell bash ./tools/dist_train.sh projects/LSKNet/configs/lsk_s_sspafpn_1x_dota_le90.py 2

Train in pycharm: If you want to train in pycharm, you can run it in train.py.

see more details at mmdetection.

Testing

shell CUDA_VISIBLE_DEVICES=0 python tools/test.py projects/LSKNet/configs/lsk_s_sspafpn_1x_dota_le90.py <CHECKPOINT_FILE>

Results on DOTAv1.0

| Model | mAP | Angle | lr schd | Batch Size | Configs | Download | | :--------------------------------------------------------: | :---: | :---: | :-----: | :--------: | :-----------------------------------------------------: | :----------------------: | | RTMDet-l (1024,1024,-) | 81.33 | - | 3x-ema | 8 | - | - | | OrientedRCNNR50 (1024,1024,200) | 81.37 | le90 | 1x | 2*8 | later | model | log | | LSKNetS (1024,1024,200) | 81.64 | le90 | 1x | 1*8 | later | model | log | | LSKNetS\ (1024,1024,200) | 81.85 | le90 | 1x | 1*8 | later | model | log |

Citations

If you find SSFPN useful in your research, please consider citing:

``` @inproceedings{zhou2022mmrotate, title = {MMRotate: A Rotated Object Detection Benchmark using PyTorch}, author = {Zhou, Yue and Yang, Xue and Zhang, Gefan and Wang, Jiabao and Liu, Yanyi and Hou, Liping and Jiang, Xue and Liu, Xingzhao and Yan, Junchi and Lyu, Chengqi and Zhang, Wenwei and Chen, Kai}, booktitle={Proceedings of the 30th ACM International Conference on Multimedia}, pages = {7331–7334}, numpages = {4}, year={2022} }

later ```