aodraw-mmdetection

(CVPR 2025 Highlight) Official repository of paper "AODRaw: Towards RAW Object Detection in Diverse Conditions" (https://arxiv.org/pdf/2411.15678)

https://github.com/lzyhha/aodraw-mmdetection

Science Score: 54.0%

This score indicates how likely this project is to be science-related based on various indicators:

✓
CITATION.cff file
Found CITATION.cff file
✓
codemeta.json file
Found codemeta.json file
✓
.zenodo.json file
Found .zenodo.json file
○
DOI references
✓
Academic publication links
Links to: arxiv.org
○
Academic email domains
○
Institutional organization owner
○
JOSS paper metadata
○
Scientific vocabulary similarity
Low similarity (11.3%) to scientific vocabulary

Keywords

object-detection object-detection-in-adverse-conditions raw-images-dataset raw-object-detecton

Last synced: 6 months ago · JSON representation ·

Repository

(CVPR 2025 Highlight) Official repository of paper "AODRaw: Towards RAW Object Detection in Diverse Conditions" (https://arxiv.org/pdf/2411.15678)

Basic Info

Host: GitHub
Owner: lzyhha
License: other
Language: Python
Default Branch: main
Homepage: https://github.com/lzyhha/AODRaw
Size: 64.5 MB

Statistics

Stars: 9
Watchers: 1
Forks: 1
Open Issues: 0
Releases: 0

Topics

object-detection object-detection-in-adverse-conditions raw-images-dataset raw-object-detecton

Created over 1 year ago · Last pushed 11 months ago

Metadata Files

Readme Contributing License Code of conduct Citation

[CVPR 2025 Highlight] Towards RAW Object Detection in Diverse Conditions

Paper link

Dataset link

Introduction
Install
Training and Evaluation
ModelZoo
- Models using down-sampled AODRaw
- Models using sliced AODRaw
Citation
License
Acknowledgement

Introduction

Existing object detection methods often consider sRGB input, which was compressed from RAW data using ISP originally designed for visualization. However, such compression might lose crucial information for detection, especially under complex light and weather conditions. We introduce the AODRaw dataset, which offers 7,785 high-resolution real RAW images with 135,601 annotated instances spanning 62 categories, capturing a broad range of indoor and outdoor scenes under 9 distinct light and weather conditions. Based on AODRaw that supports RAW and sRGB object detection, we provide a comprehensive benchmark for evaluating current detection methods. We find that sRGB pre-training constrains the potential of RAW object detection due to the domain gap between sRGB and RAW, prompting us to directly pre-train on the RAW domain. However, it is harder for RAW pre-training to learn rich representations than sRGB pre-training due to the camera noise. To assist RAW pre-training, we distill the knowledge from an off-the-shelf model pre-trained on the sRGB domain. As a result, we achieve substantial improvements under diverse and adverse conditions without relying on extra pre-processing modules.

Dataset

Please refer to AODRaw to download and preprocess our AODRaw dataset.

Install

Please refer to the README of mmdetection.

Training and Evaluation

Configs and pre-trained weights

We provide training and evaluation for RAW and sRGB object detection.
The images in the AODRaw are recorded at a resolution of $6000\times 4000$. It is unrealistic to feed such huge images into the detectors. Thus, we adopt two experiment settings: 1) down-sampling the images into a lower resolution of $2000\times1333$, corresponding to configs, and 2) slicing the images into a collection of $1280\times 1280$ patches, corresponding to configs. Please preprocess the AODRaw dataset or directly download the processed files in datasets and downloading.

Training and evaluation using down-sampled AODRaw: | Task | Pre-training domain | Config path | | --------------------- | -------------------- |-------------------- | | sRGB object detection | sRGB | configs/aodraw/...aodrawsrgb.py | | RAW object detection | sRGB | configs/aodraw/...aodrawraw.py | | RAW object detection | RAW | configs/aodraw/...aodrawraw_raw-pretraining.py |

Training and evaluation using sliced AODRaw: | Task | Pre-training | Config path | | --------------------- | -------------------- |-------------------- | | sRGB object detection | sRGB | configs/aodrawslice/...aodrawsrgbslice.py | | RAW object detection | sRGB | configs/aodrawslice/...aodrawrawslice.py | | RAW object detection | RAW | configs/aodrawslice/...aodrawrawslice_raw-pretraining.py |

Pre-trained weights for ConvNeXt-T and Swin-T: | Architecture | Pre-training domain | Downloading link | | --------------------- | -------------------- |-------------------- | | ConvNeXt-T | sRGB | Google and Baidu | | ConvNeXt-T | RAW | Google and Baidu | | Swin-T | RAW | Google and Baidu |

Training

Single GPU

shell python tools/train.py ${CONFIG_FILE} [optional arguments]

Multi GPU

shell bash tools/dist_train.sh ${CONFIG_FILE} ${GPU_NUM} [optional arguments]

For more training and evaluation command details, please refer to mmdetection.

Evaluation

Single GPU

shell python tools/test.py ${CONFIG_FILE} ${CHECKPOINT_FILE} [optional arguments]

Multi GPU

shell bash tools/dist_test.sh ${CONFIG_FILE} ${CHECKPOINT_FILE} ${GPU_NUM} [optional arguments]

For more training and evaluation command details, please refer to mmdetection.

ModelZoo

Models using down-sampled AODRaw

Please follow downsampling to preprocess the images or download preprocessed images in download.

| Detector |Backbone | Pre-training domain | Fine-tuning domain | AP | Config | Model | Pre-trained weights | | --------------------- | -------------------- | :--------------------: | :--------------------: | :--------------------: | :--------------------: | :--------------------: |:--------------------:| |Faster RCNN | ResNet-50 | sRGB | sRGB | 23.3 | Config | Google and Baidu | - | |Retinanet | ResNet-50 | sRGB | sRGB | 19.1 | Config | Google and Baidu | - | |GFocal | ResNet-50 | sRGB | sRGB | 24.2 | Config | Google and Baidu | - | |Sparse RCNN | ResNet-50 | sRGB | sRGB | 15.6 | Config | Google and Baidu | - | |Deformable DETR | ResNet-50 | sRGB | sRGB | 16.6 | confog | Google and Baidu | - | |Cascade RCNN| ResNet-50 | sRGB | sRGB | 25.6 | Config | Google and Baidu | - | |Faster RCNN | Swin-T | sRGB |sRGB | 28.4 | Config | Google and Baidu | - | |Faster RCNN | ConvNeXt-T | sRGB | sRGB | 29.7 | Config | Google and Baidu | Google and Baidu | |GFocal | Swin-T | sRGB | sRGB | 30.1 | Config | Google and Baidu | - | |GFocal | ConvNeXt-T | sRGB | sRGB | 32.1 | Config | Google and Baidu | Google and Baidu | |Cascade RCNN | Swin-T | sRGB | sRGB | 32.0 | Config | Google and Baidu | - | |Cascade RCNN | ConvNeXt-T | sRGB | sRGB | 34.0 | Config | Google and Baidu | Google and Baidu |

The directory imagesdownsampledsrgb is required for the above experiments.

The directory imagesdownsampledraw is required for the above experiments.

Models using sliced AODRaw

Please follow slicing to preprocess the images or download preprocessed images in download.

The directory imagessliceraw is required for the above experiments.

Citation

@article{li2024aodraw, title={Towards RAW Object Detection in Diverse Conditions}, author={Zhong-Yu Li and Xin Jin and Boyuan Sun and Chun-Le Guo and Ming-Ming Cheng}, journal={arXiv preprint arXiv:2411.15678}, year={2024}, }

License

The code is released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Public License for NonCommercial use only.

Acknowledgement

This repo is modified from mmdetection.

Owner

Name: Zhong-Yu Li
Login: lzyhha
Kind: user

Repositories: 1
Profile: https://github.com/lzyhha

Alchemist

Citation (CITATION.cff)

cff-version: 1.2.0
message: "If you use this software, please cite it as below."
authors:
  - name: "MMDetection Contributors"
title: "OpenMMLab Detection Toolbox and Benchmark"
date-released: 2018-08-22
url: "https://github.com/open-mmlab/mmdetection"
license: Apache-2.0

GitHub Events

Total

Watch event: 18
Push event: 1
Public event: 1
Fork event: 1

Last Year

Watch event: 18
Push event: 1
Public event: 1
Fork event: 1

Dependencies

.github/workflows/deploy.yml actions

actions/checkout v2 composite
actions/setup-python v2 composite

.circleci/docker/Dockerfile docker

pytorch/pytorch ${PYTORCH}-cuda${CUDA}-cudnn${CUDNN}-devel build

docker/Dockerfile docker

pytorch/pytorch ${PYTORCH}-cuda${CUDA}-cudnn${CUDNN}-devel build

docker/serve/Dockerfile docker

pytorch/pytorch ${PYTORCH}-cuda${CUDA}-cudnn${CUDNN}-devel build

docker/serve_cn/Dockerfile docker

pytorch/pytorch ${PYTORCH}-cuda${CUDA}-cudnn${CUDNN}-devel build

requirements/albu.txt pypi

albumentations >=0.3.2

requirements/build.txt pypi

cython *
numpy *

requirements/docs.txt pypi

docutils ==0.16.0
myst-parser *
sphinx ==4.0.2
sphinx-copybutton *
sphinx_markdown_tables *
sphinx_rtd_theme ==0.5.2
urllib3 <2.0.0

requirements/mminstall.txt pypi

mmcv >=2.0.0rc4,<2.2.0
mmengine >=0.7.1,<1.0.0

requirements/multimodal.txt pypi

fairscale *
jsonlines *
nltk *
pycocoevalcap *
transformers *

requirements/optional.txt pypi

cityscapesscripts *
emoji *
fairscale *
imagecorruptions *
scikit-learn *

requirements/readthedocs.txt pypi

mmcv >=2.0.0rc4,<2.2.0
mmengine >=0.7.1,<1.0.0
scipy *
torch *
torchvision *
urllib3 <2.0.0

requirements/runtime.txt pypi

matplotlib *
numpy *
pycocotools *
scipy *
shapely *
six *
terminaltables *
tqdm *

requirements/tests.txt pypi

asynctest * test
cityscapesscripts * test
codecov * test
flake8 * test
imagecorruptions * test
instaboostfast * test
interrogate * test
isort ==4.3.21 test
kwarray * test
memory_profiler * test
nltk * test
onnx ==1.7.0 test
onnxruntime >=1.8.0 test
parameterized * test
prettytable * test
protobuf <=3.20.1 test
psutil * test
pytest * test
transformers * test
ubelt * test
xdoctest >=0.10.0 test
yapf * test

requirements/tracking.txt pypi

mmpretrain *
motmetrics *
numpy <1.24.0
scikit-learn *
seaborn *

requirements.txt pypi

setup.py pypi

aodraw-mmdetection

Science Score: 54.0%

Keywords

Repository

Basic Info

Statistics

Topics

Metadata Files

README.md

[CVPR 2025 Highlight] Towards RAW Object Detection in Diverse Conditions

Table of Contents

Introduction

Dataset

Install

Training and Evaluation

Configs and pre-trained weights

Training

Single GPU

Multi GPU

Evaluation

Single GPU

Multi GPU

ModelZoo

Models using down-sampled AODRaw

Models using sliced AODRaw

Citation

License

Acknowledgement

Owner

Citation (CITATION.cff)

GitHub Events

Total

Last Year

Dependencies