Data

Tools

Indexes

Applications

Experiments

Open Source Science

mtfenet

https://github.com/waylans/mtfenet

Science Score: 67.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ✓
    CITATION.cff file
    Found CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ✓
    DOI references
    Found 1 DOI reference(s) in README
  • ✓
    Academic publication links
    Links to: arxiv.org, springer.com, mdpi.com, ieee.org
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (5.3%) to scientific vocabulary
Last synced: 6 months ago · JSON representation ·

Repository

Basic Info
  • Host: GitHub
  • Owner: waylans
  • Language: Python
  • Default Branch: main
  • Size: 9.79 MB
Statistics
  • Stars: 0
  • Watchers: 0
  • Forks: 0
  • Open Issues: 0
  • Releases: 0
Created 7 months ago · Last pushed 7 months ago
Metadata Files
Readme Citation Security

README.md

MTFENet: A Multi-Task Autonomous Driving Network for Real-Time Target Perception

This repository(MTFENet) is the official PyTorch implementation of the paper "MTFENet: A Multi-Task Autonomous Driving Network for Real-Time Target Perception".

The Illustration of MTFENet

MTFENet

Results

Evaluation of Model Size and Inference Speed.

| Networks | Size (Pixel)| Parameters (M) | FPS(bs=32) | |:----------------:|:-------------:|:----------------:|:-------------:| | YOLOP | 640 | 7.90 | 361.27 | | YOLOPv2 | 640 | 38.90 | 419.57 | | YOLOPv3 | 640 | 30.2 | 389.58 | | HybridNets | 640 | 12.83 | 243.30 | | A-YOLOM | 640 | 13.61 | 346.53 | | TriLiteNet | 640 | 2.35 | 423.50 | | MTFENet | 640 | 8.70 | 441.53 |

Comparison Between Our Method and the Current SOTA on the BDD100K Validation Set.

Networks Traffic Object Detection Achievement Drivable Area Segmentation
Achievement 		Laneline Segment Achievement
mAP50 (%) Recall (%) mIoU (%) Acc (%) IoU (%)
Faster R-CNN81.2064.90---
YOLOV5s77.2086.80---
MultiNet81.3060.2071.60--
DLT-Net89.4068.4072.10--
PSPNet--89.60--
ENet---34.1214.64
SCNN---35.7915.84
ENet-SAD---36.5616.02
YOLOP76.5088.2090.5084.4026.50
HybridNets77.3089.7091.5085.4031.60
YOLOPv283.4091.1093.2087.3127.25
A-YOLOM81.1086.9091.0084.9028.80
YOLOPv384.3096.9093.2088.3028.00
TriLiteNet72.3085.6092.4082.3029.80
MTFENet81.5088.4093.8087.6033.70

Visualization

video visualization Results

  • Note: The raw video comes from HybridNets
  • The results of our experiments are as follows:

Image Visualization Results

Real Road
Visual Comparison of Results on Sunny Day
Visual Comparison of Results on Sunny Day Visual Comparison of Results on Sunny Day
 Visual Comparison of Results at Night Visual Comparison of Results at Night
Visual Comparison of Results on Rainy Day Visual Comparison of Results on Rainy Day
 Visual Comparison of Results on Snow Day Visual Comparison of Results on Snow Day

Requirement

We implemented the algorithm in a Linux environment and conducted all experiments on an NVIDIA RTX 4090 GPU equipped with 24GB of memory. The development environment was based on Python==3.8.19 ,PyTorch 1.13.1, conda 24.1.2, and CUDA11.7, with pre-training conducted on the BDD100K dataset.

setup cd MTFENet pip install -e .

Dataset

We recommend the dataset directory structure to be the following:

```

The id represent the correspondence relation

├─dataset root │ ├─images │ │ ├─train2017 │ │ ├─val2017 │ ├─detection-object-9 │ │ ├─labels │ │ │ ├─train2017 │ │ │ ├─val2017 │ ├─seg-drivable-10 │ │ ├─labels │ │ │ ├─train2017 │ │ │ ├─val2017 │ ├─seg-lane-11 │ │ ├─labels │ │ │ ├─train2017 │ │ │ ├─val2017 ```

Update the your dataset path in the ./test_yaml/bdd-mtfenet-multi.yaml.

Training

python ./ultralytics/train.py

Evaluation

python ./ultralytics/val.py

Prediction

python ./ultralytics/predict.py

Notes: We would like to express our sincere appreciation to the authors of the following works for their valuable contributions to the field of multi-task visual perception. Their research has provided strong foundations and meaningful benchmarks that have significantly guided and inspired our study. We also gratefully acknowledge the open-source code repositories they provided, which facilitated fair comparison and reproducibility in our experiments:

Owner

  • Name: wnorth
  • Login: waylans
  • Kind: user

Citation (CITATION.cff) 

cff-version: 1.2.0
preferred-citation:
  type: software
  message: If you use this software, please cite it as below.
  authors:
  - family-names: Jocher
    given-names: Glenn
    orcid: "https://orcid.org/0000-0001-5950-6979"
  - family-names: Chaurasia
    given-names: Ayush
    orcid: "https://orcid.org/0000-0002-7603-6750"
  - family-names: Qiu
    given-names: Jing
    orcid: "https://orcid.org/0000-0003-3783-7069"
  title: "YOLO by Ultralytics"
  version: 8.0.0
  # doi: 10.5281/zenodo.3908559  # TODO
  date-released: 2023-1-10
  license: AGPL-3.0
  url: "https://github.com/ultralytics/ultralytics"

GitHub Events

Total
  • Push event: 2
  • Create event: 2
Last Year
  • Push event: 2
  • Create event: 2

Dependencies

docker/Dockerfile docker
  • pytorch/pytorch 2.0.0-cuda11.7-cudnn8-runtime build
requirements.txt pypi
  • Pillow >=7.1.2
  • PyYAML >=5.3.1
  • check-manifest *
  • coremltools >=6.0
  • coverage *
  • efficientnet_pytorch *
  • einops *
  • matplotlib >=3.2.2
  • mkdocs-material *
  • mkdocs-redirects *
  • mkdocs-ultralytics-plugin *
  • mkdocstrings *
  • opencv-python >=4.6.0
  • openvino-dev >=2022.3
  • pandas >=1.1.4
  • psutil *
  • pytest *
  • pytest-cov *
  • requests >=2.23.0
  • scipy >=1.4.1
  • seaborn >=0.11.0
  • sentry_sdk *
  • tensorflowjs *
  • timm *
  • torch >=1.7.0
  • torchvision >=0.8.1
  • tqdm >=4.64.0
setup.py pypi
ultralytics/nn/backbone/TransNeXt/swattention_extension/setup.py pypi