road-object-detection-tracking-in-complex-environments-based-on-yolov8-deepsort
Science Score: 36.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
○DOI references
-
✓Academic publication links
Links to: arxiv.org -
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (10.0%) to scientific vocabulary
Repository
Basic Info
- Host: GitHub
- Owner: FitzFitzFitz
- License: agpl-3.0
- Language: Python
- Default Branch: main
- Size: 66 MB
Statistics
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
- Releases: 0
Metadata Files
README.md
Road Object Detection & Tracking in Complex Environments Based on YOLOv8 & DeepSORT
Abstract
This project focuses on advancing multi-object tracking (MOT) techniques, crucial for autonomous driving and other applications, especially in challenging urban scenes. By enhancing target detection and tracking algorithms, we aim to improve MOT's performance and robustness significantly.
Introduction
Multi-object tracking in complex environments faces numerous challenges. This work reviews existing technologies in MOT, emphasizing target detection algorithms, and proposes improvements. We specifically explore the advancements in YOLOv8, including its network structure, loss function, and data augmentation techniques, and how these enhancements can be leveraged for better detection results.
Methodology
Our approach involves fine-tuning a model based on YOLOv8 tailored to our specific dataset requirements. We also improve the Re-ID model in DeepSORT by adopting a convolutional neural network with a local maximum pooling neural network (LMBN), enhancing feature extraction and target matching. The proposed DeepOC-SORT algorithm combines these advancements with Kalman filtering, observation center re-update, and momentum techniques for increased robustness.
Results
Testing on the MOT17 dataset demonstrated notable improvements: a 4.6% increase in the HOTA index and a 5% overall performance boost. Additionally, the frame rate on test videos rose by 18.7%, exceeding 40 frames per second, meeting real-time video analysis requirements. The algorithm's robustness in adverse conditions, such as rain or densely populated scenes, was also confirmed.
Keywords
Multi-object Tracking, DeepSORT, YOLOv8
Installation
git clone --recurse-submodules https://github.com/mikel-brostrom/yolov8_tracking.git # clone recursively
cd yolov8_tracking
pip install -r requirements.txt # install dependencies
Tutorials
* [Yolov5 training (link to external repository)](https://github.com/ultralytics/yolov5/wiki/Train-Custom-Data) * [Deep appearance descriptor training (link to external repository)](https://kaiyangzhou.github.io/deep-person-reid/user_guide.html) * [ReID model export to ONNX, OpenVINO, TensorRT and TorchScript](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/ReID-multi-framework-model-export) * [Evaluation on custom tracking dataset](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/How-to-evaluate-on-custom-tracking-dataset) * Inference acceleration with Nebullvm * [Yolov5](https://colab.research.google.com/drive/1J6dl90-zOjNNtcwhw7Yuuxqg5oWp_YJa?usp=sharing) * [ReID](https://colab.research.google.com/drive/1APUZ1ijCiQFBR9xD0gUvFUOC8yOJIvHm?usp=sharing)Experiments
In inverse chronological order: * [Evaluation of the params evolved for first half of MOT17 on the complete MOT17](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/Evaluation-of-the-params-evolved-for-first-half-of-MOT17-on-the-complete-MOT17) * [Segmentation model vs object detetion model on MOT metrics](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/Segmentation-model-vs-object-detetion-model-on-MOT-metrics) * [Effect of masking objects before feature extraction](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/Masked-detection-crops-vs-regular-detection-crops-for-ReID-feature-extraction) * [conf-thres vs HOTA, MOTA and IDF1](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/conf-thres-vs-MOT-metrics) * [Effect of KF updates ahead for tracks with no associations on MOT17](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/Effect-of-KF-updates-ahead-for-tracks-with-no-associations,-on-MOT17) * [Effect of full images vs 1280 input to StrongSORT on MOT17](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/Effect-of-passing-full-image-input-vs-1280-re-scaled-to-StrongSORT-on-MOT17) * [Effect of different OSNet architectures on MOT16](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/OSNet-architecture-performances-on-MOT16) * [Yolov5 StrongSORT vs BoTSORT vs OCSORT](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/StrongSORT-vs-BoTSORT-vs-OCSORT) * Yolov5 [BoTSORT](https://arxiv.org/abs/2206.14651) branch: https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/tree/botsort * [Yolov5 StrongSORT OSNet vs other trackers MOT17](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/MOT-17-evaluation-(private-detector)) * [StrongSORT MOT16 ablation study](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/Yolov5DeepSORTwithOSNet-vs-Yolov5StrongSORTwithOSNet-ablation-study-on-MOT16) * [Yolov5 StrongSORT OSNet vs other trackers MOT16 (deprecated)](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/MOT-16-evaluation)Custom object detection architecture
The trackers provided in this repo can be used with other object detectors than Yolov8. Make sure that the output of your detector has the following format: ```bash (x1,y1, x2, y2, obj, cls0, cls1, ..., clsn) ```Tracking
bash
$ python track.py --yolo-weights yolov8n.pt # bboxes only
yolov8n-seg.pt # bboxes + segmentation masks
Tracking methods
```bash $ python track.py --tracking-method deepocsort strongsort ocsort bytetrack botsort ```Tracking sources
Tracking can be run on most video formats ```bash $ python track.py --source 0 # webcam img.jpg # image vid.mp4 # video path/ # directory path/*.jpg # glob 'https://youtu.be/Zgi9g1ksQHc' # YouTube 'rtsp://example.com/media.mp4' # RTSP, RTMP, HTTP stream ```Select Yolov8 model
There is a clear trade-off between model inference speed and overall performance. In order to make it possible to fulfill your inference speed/accuracy needs you can select a Yolov5 family model for automatic download. These model can be further optimized for you needs by the [export.py](https://github.com/ultralytics/yolov5/blob/master/export.py) script ```bash $ python track.py --source 0 --yolo-weights yolov8n.pt --img 640 yolov8s.tflite yolov8m.pt yolov8l.onnx yolov8x.pt --img 1280 ... ```Select ReID model
Some tracking methods combine appearance description and motion in the process of tracking. For those which use appearance, you can choose a ReID model based on your needs from this [ReID model zoo](https://kaiyangzhou.github.io/deep-person-reid/MODEL_ZOO). These model can be further optimized for you needs by the [reid_export.py](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/blob/master/reid_export.py) script ```bash $ python track.py --source 0 --reid-weights lmbn_n_cuhk03_d.pt osnet_x0_25_market1501.pt mobilenetv2_x1_4_msmt17.engine resnet50_msmt17.onnx osnet_x1_0_msmt17.pt ... ```Filter tracked classes
By default the tracker tracks all MS COCO classes. If you want to track a subset of the classes that you model predicts, add their corresponding index after the classes flag, ```bash python track.py --source 0 --yolo-weights yolov8s.pt --classes 16 17 # COCO yolov8 model. Track cats and dogs, only ``` [Here](https://tech.amikelive.com/node-718/what-object-categories-labels-are-in-coco-dataset/) is a list of all the possible objects that a Yolov8 model trained on MS COCO can detect. Notice that the indexing for the classes in this repo starts at zeroUpdates with predicted-ahead bbox in StrongSORT
If your use-case contains many occlussions and the motion trajectiories are not too complex, you will most certainly benefit from updating the Kalman Filter by its own predicted state. Select the number of predictions that suits your needs here: https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/blob/b1da64717ef50e1f60df2f1d51e1ff91d3b31ed4/trackers/strong_sort/configs/strong_sort.yaml#L7 Save the trajectories to you video by: ```bash python track.py --source ... --save-trajectories --save-vid ```
MOT compliant results
Can be saved to your experiment folder `runs/track/Tracker hyperparameter tuning
We use a fast and elitist multiobjective genetic algorithm for tracker hyperparameter tuning. By default the objectives are: HOTA, MOTA, IDF1. Run it by ```bash $ python evolve.py --tracking-method strongsort --benchmark MOT17 --n-trials 100 # tune strongsort for MOT17 --tracking-method ocsort --benchmarkContact
For Yolov8 tracking bugs and feature requests please visit https://docs.ultralytics.com/
For other requests please send an email to: fitzfitzfitz.xia@gmail.com
Owner
- Name: Chagall
- Login: FitzFitzFitz
- Kind: user
- Repositories: 1
- Profile: https://github.com/FitzFitzFitz
GitHub Events
Total
Last Year
Dependencies
- nvcr.io/nvidia/pytorch 22.11-py3 build
- Pillow >=7.1.2
- PyYAML >=5.3.1
- easydict *
- filterpy *
- gdown *
- gitpython *
- ipython *
- lap *
- matplotlib >=3.2.2
- numpy ==1.23.1
- opencv-python >=4.1.1
- pandas >=1.1.4
- psutil *
- requests >=2.23.0
- scipy >=1.4.1
- seaborn >=0.11.0
- tensorboard >=2.4.1
- thop >=0.1.1
- torchvision >=0.8.1
- tqdm >=4.64.0
- ultralytics ==8.0.20