ai-object-tracking
Exploring BoxMOT: Pluggable Modules for Advanced Tracking in Segmentation and Object Detection
Science Score: 49.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
✓DOI references
Found 3 DOI reference(s) in README -
✓Academic publication links
Links to: arxiv.org, zenodo.org -
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (10.3%) to scientific vocabulary
Last synced: 9 months ago
·
JSON representation
Repository
Exploring BoxMOT: Pluggable Modules for Advanced Tracking in Segmentation and Object Detection
Basic Info
Statistics
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 6
- Releases: 0
Created over 2 years ago
· Last pushed over 2 years ago
Metadata Files
Readme
License
Citation
README.md
BoxMOT: pluggable SOTA tracking modules for segmentation, object detection and pose estimation models
Introduction
This repo contains a collections of pluggable state-of-the-art multi-object trackers for segmentation, object detection and pose estimation models. For the methods using appearance description, both heavy (CLIPReID) and lightweight state-of-the-art ReID models (LightMBN, OSNet and more) are available for automatic download. We provide examples on how to use this package together with popular object detection models such as: Yolov8, Yolo-NAS and YOLOX.
| Tracker | HOTA↑ | MOTA↑ | IDF1↑ |
| -------- | ----- | ----- | ----- |
| [BoTSORT](https://arxiv.org/pdf/2206.14651.pdf) | 77.8 | 78.9 | 88.9 |
| [DeepOCSORT](https://arxiv.org/pdf/2302.11813.pdf) | 77.4 | 78.4 | 89.0 |
| [OCSORT](https://arxiv.org/pdf/2203.14360.pdf) | 77.4 | 78.4 | 89.0 |
| [HybridSORT](https://arxiv.org/pdf/2308.00783.pdf) | 77.3 | 77.9 | 88.8 |
| [ByteTrack](https://arxiv.org/pdf/2110.06864.pdf) | 75.6 | 74.6 | 86.0 |
| [StrongSORT](https://arxiv.org/pdf/2202.13514.pdf) | | | |
| ![]()
| ![]()
| ![]()
| ![]()
|
NOTES: performed on the 10 first frames of each MOT17 sequence. The detector used is ByteTrack's YoloXm, trained on: CrowdHuman, MOT17, Cityperson and ETHZ. Each tracker is configured with its original parameters found in their respective official repository.
Tutorials
* [Yolov8 training (link to external repository)](https://docs.ultralytics.com/modes/train/) * [Deep appearance descriptor training (link to external repository)](https://kaiyangzhou.github.io/deep-person-reid/user_guide.html) * [ReID model export to ONNX, OpenVINO, TensorRT and TorchScript](https://github.com/mikel-brostrom/yolo_tracking/wiki/ReID-multi-framework-model-export) * [Evaluation on custom tracking dataset](https://github.com/mikel-brostrom/yolo_tracking/wiki/How-to-evaluate-on-custom-tracking-dataset) * [ReID inference acceleration with Nebullvm](https://colab.research.google.com/drive/1APUZ1ijCiQFBR9xD0gUvFUOC8yOJIvHm?usp=sharing)Experiments
In inverse chronological order: * [Evaluation of the params evolved for first half of MOT17 on the complete MOT17](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/Evaluation-of-the-params-evolved-for-first-half-of-MOT17-on-the-complete-MOT17) * [Segmentation model vs object detetion model on MOT metrics](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/Segmentation-model-vs-object-detetion-model-on-MOT-metrics) * [Effect of masking objects before feature extraction](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/Masked-detection-crops-vs-regular-detection-crops-for-ReID-feature-extraction) * [conf-thres vs HOTA, MOTA and IDF1](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/conf-thres-vs-MOT-metrics) * [Effect of KF updates ahead for tracks with no associations on MOT17](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/Effect-of-KF-updates-ahead-for-tracks-with-no-associations,-on-MOT17) * [Effect of full images vs 1280 input to StrongSORT on MOT17](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/Effect-of-passing-full-image-input-vs-1280-re-scaled-to-StrongSORT-on-MOT17) * [Effect of different OSNet architectures on MOT16](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/OSNet-architecture-performances-on-MOT16) * [Yolov5 StrongSORT vs BoTSORT vs OCSORT](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/StrongSORT-vs-BoTSORT-vs-OCSORT) * Yolov5 [BoTSORT](https://arxiv.org/abs/2206.14651) branch: https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/tree/botsort * [Yolov5 StrongSORT OSNet vs other trackers MOT17](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/MOT-17-evaluation-(private-detector)) * [StrongSORT MOT16 ablation study](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/Yolov5DeepSORTwithOSNet-vs-Yolov5StrongSORTwithOSNet-ablation-study-on-MOT16) * [Yolov5 StrongSORT OSNet vs other trackers MOT16 (deprecated)](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/MOT-16-evaluation)Tracking
Yolo models
```bash $ python examples/track.py --yolo-model yolov8n # bboxes only python examples/track.py --yolo-model yolo_nas_s # bboxes only python examples/track.py --yolo-model yolox_n # bboxes only yolov8n-seg # bboxes + segmentation masks yolov8n-pose # bboxes + pose estimation ```Tracking methods
```bash $ python examples/track.py --tracking-method deepocsort strongsort ocsort bytetrack botsort ```Tracking sources
Tracking can be run on most video formats ```bash $ python examples/track.py --source 0 # webcam img.jpg # image vid.mp4 # video path/ # directory path/*.jpg # glob 'https://youtu.be/Zgi9g1ksQHc' # YouTube 'rtsp://example.com/media.mp4' # RTSP, RTMP, HTTP stream ```Select ReID model
Some tracking methods combine appearance description and motion in the process of tracking. For those which use appearance, you can choose a ReID model based on your needs from this [ReID model zoo](https://kaiyangzhou.github.io/deep-person-reid/MODEL_ZOO). These model can be further optimized for you needs by the [reid_export.py](https://github.com/mikel-brostrom/yolo_tracking/blob/master/boxmot/deep/reid_export.py) script ```bash $ python examples/track.py --source 0 --reid-model lmbn_n_cuhk03_d.pt # lightweight osnet_x0_25_market1501.pt mobilenetv2_x1_4_msmt17.engine resnet50_msmt17.onnx osnet_x1_0_msmt17.pt clip_market1501.pt # heavy clip_vehicleid.pt ... ```Filter tracked classes
By default the tracker tracks all MS COCO classes. If you want to track a subset of the classes that you model predicts, add their corresponding index after the classes flag, ```bash python examples/track.py --source 0 --yolo-model yolov8s.pt --classes 16 17 # COCO yolov8 model. Track cats and dogs, only ``` [Here](https://tech.amikelive.com/node-718/what-object-categories-labels-are-in-coco-dataset/) is a list of all the possible objects that a Yolov8 model trained on MS COCO can detect. Notice that the indexing for the classes in this repo starts at zeroMOT compliant results
Can be saved to your experiment folder `runs/track/exp*/` by ```bash python examples/track.py --source ... --save-mot ```Evaluation
Evaluate a combination of detector, tracking method and ReID model on standard MOT dataset or you custom one by ```bash $ python3 examples/val.py --yolo-model yolo_nas_s.pt --reid-model osnetx1_0_dukemtcereid.pt --tracking-method deepocsort --benchmark MOT16 --yolo-model yolox_n.pt --reid-model osnet_ain_x1_0_msmt17.pt --tracking-method ocsort --benchmark MOT17 --yolo-model yolov8s.pt --reid-model lmbn_n_market.pt --tracking-method strongsort --benchmarkEvolution
We use a fast and elitist multiobjective genetic algorithm for tracker hyperparameter tuning. By default the objectives are: HOTA, MOTA, IDF1. Run it by ```bash $ python examples/evolve.py --tracking-method strongsort --benchmark MOT17 --n-trials 100 # tune strongsort for MOT17 --tracking-method ocsort --benchmarkCustom object detection model example
Minimalistic
```python import cv2 import numpy as np from pathlib import Path from boxmot import DeepOCSORT tracker = DeepOCSORT( model_weights=Path('osnet_x0_25_msmt17.pt'), # which ReID model to use device='cuda:0', fp16=False, ) vid = cv2.VideoCapture(0) while True: ret, im = vid.read() # substitute by your object detector, output has to be N X (x, y, x, y, conf, cls) dets = np.array([[144, 212, 578, 480, 0.82, 0], [425, 281, 576, 472, 0.56, 65]]) tracks = tracker.update(dets, im) # --> (x, y, x, y, id, conf, cls, ind) ```Complete
```python import cv2 import numpy as np from pathlib import Path from boxmot import DeepOCSORT tracker = DeepOCSORT( model_weights=Path('osnet_x0_25_msmt17.pt'), # which ReID model to use device='cuda:0', fp16=True, ) vid = cv2.VideoCapture(0) color = (0, 0, 255) # BGR thickness = 2 fontscale = 0.5 while True: ret, im = vid.read() # substitute by your object detector, input to tracker has to be N X (x, y, x, y, conf, cls) dets = np.array([[144, 212, 578, 480, 0.82, 0], [425, 281, 576, 472, 0.56, 65]]) tracks = tracker.update(dets, im) # --> (x, y, x, y, id, conf, cls, ind) xyxys = tracks[:, 0:4].astype('int') # float64 to int ids = tracks[:, 4].astype('int') # float64 to int confs = tracks[:, 5] clss = tracks[:, 6].astype('int') # float64 to int inds = tracks[:, 7].astype('int') # float64 to int # in case you have segmentations or poses alongside with your detections you can use # the ind variable in order to identify which track is associated to each seg or pose by: # segs = segs[inds] # poses = poses[inds] # you can then zip them together: zip(tracks, poses) # print bboxes with their associated id, cls and conf if tracks.shape[0] != 0: for xyxy, id, conf, cls in zip(xyxys, ids, confs, clss): im = cv2.rectangle( im, (xyxy[0], xyxy[1]), (xyxy[2], xyxy[3]), color, thickness ) cv2.putText( im, f'id: {id}, conf: {conf}, c: {cls}', (xyxy[0], xyxy[1]-10), cv2.FONT_HERSHEY_SIMPLEX, fontscale, color, thickness ) # show image with bboxes, ids, classes and confidences cv2.imshow('frame', im) # break on pressing q if cv2.waitKey(1) & 0xFF == ord('q'): break vid.release() cv2.destroyAllWindows() ```Contact
For Yolo tracking bugs and feature requests please visit GitHub Issues. For business inquiries or professional support requests please send an email to: yolov5.deepsort.pytorch@gmail.com
Owner
- Login: drkhaik
- Kind: user
- Repositories: 1
- Profile: https://github.com/drkhaik