Data

Tools

Indexes

Applications

Experiments

Open Source Science

yolo_tracking

https://github.com/lastonton/yolo_tracking

Science Score: 67.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ✓
    CITATION.cff file
    Found CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ✓
    DOI references
    Found 3 DOI reference(s) in README
  • ✓
    Academic publication links
    Links to: arxiv.org, zenodo.org
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (10.3%) to scientific vocabulary
Last synced: 6 months ago · JSON representation ·

Repository

Basic Info
  • Host: GitHub
  • Owner: lastonton
  • License: agpl-3.0
  • Language: Python
  • Default Branch: main
  • Size: 15.6 MB
Statistics
  • Stars: 0
  • Watchers: 1
  • Forks: 0
  • Open Issues: 6
  • Releases: 1
Created over 2 years ago · Last pushed about 2 years ago
Metadata Files
Readme License Citation

README.md

BoxMOT: pluggable SOTA tracking modules for segmentation, object detection and pose estimation models


CI CPU testing
Open In Colab DOI

Introduction

This repo contains a collections of pluggable state-of-the-art multi-object trackers for segmentation, object detection and pose estimation models. For the methods using appearance description, both heavy (CLIPReID) and lightweight state-of-the-art ReID models (LightMBN, OSNet and more) are available for automatic download. We provide examples on how to use this package together with popular object detection models such as: Yolov8, Yolo-NAS and YOLOX.

| Tracker | HOTA↑ | MOTA↑ | IDF1↑ | | -------- | ----- | ----- | ----- | | [BoTSORT](https://arxiv.org/pdf/2206.14651.pdf) | 77.8 | 78.9 | 88.9 | | [DeepOCSORT](https://arxiv.org/pdf/2302.11813.pdf) | 77.4 | 78.4 | 89.0 | | [OCSORT](https://arxiv.org/pdf/2203.14360.pdf) | 77.4 | 78.4 | 89.0 | | [HybridSORT](https://arxiv.org/pdf/2308.00783.pdf) | 77.3 | 77.9 | 88.8 | | [ByteTrack](https://arxiv.org/pdf/2110.06864.pdf) | 75.6 | 74.6 | 86.0 | | [StrongSORT](https://arxiv.org/pdf/2202.13514.pdf) | | | | | | | | | NOTES: performed on the 10 first frames of each MOT17 sequence. The detector used is ByteTrack's YoloXm, trained on: CrowdHuman, MOT17, Cityperson and ETHZ. Each tracker is configured with its original parameters found in their respective official repository.

Tutorials * [Yolov8 training (link to external repository)](https://docs.ultralytics.com/modes/train/)  * [Deep appearance descriptor training (link to external repository)](https://kaiyangzhou.github.io/deep-person-reid/user_guide.html)  * [ReID model export to ONNX, OpenVINO, TensorRT and TorchScript](https://github.com/mikel-brostrom/yolo_tracking/wiki/ReID-multi-framework-model-export)  * [Evaluation on custom tracking dataset](https://github.com/mikel-brostrom/yolo_tracking/wiki/How-to-evaluate-on-custom-tracking-dataset)  * [ReID inference acceleration with Nebullvm](https://colab.research.google.com/drive/1APUZ1ijCiQFBR9xD0gUvFUOC8yOJIvHm?usp=sharing) 
Experiments In inverse chronological order: * [Evaluation of the params evolved for first half of MOT17 on the complete MOT17](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/Evaluation-of-the-params-evolved-for-first-half-of-MOT17-on-the-complete-MOT17) * [Segmentation model vs object detetion model on MOT metrics](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/Segmentation-model-vs-object-detetion-model-on-MOT-metrics) * [Effect of masking objects before feature extraction](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/Masked-detection-crops-vs-regular-detection-crops-for-ReID-feature-extraction) * [conf-thres vs HOTA, MOTA and IDF1](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/conf-thres-vs-MOT-metrics) * [Effect of KF updates ahead for tracks with no associations on MOT17](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/Effect-of-KF-updates-ahead-for-tracks-with-no-associations,-on-MOT17) * [Effect of full images vs 1280 input to StrongSORT on MOT17](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/Effect-of-passing-full-image-input-vs-1280-re-scaled-to-StrongSORT-on-MOT17) * [Effect of different OSNet architectures on MOT16](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/OSNet-architecture-performances-on-MOT16) * [Yolov5 StrongSORT vs BoTSORT vs OCSORT](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/StrongSORT-vs-BoTSORT-vs-OCSORT) * Yolov5 [BoTSORT](https://arxiv.org/abs/2206.14651) branch: https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/tree/botsort * [Yolov5 StrongSORT OSNet vs other trackers MOT17](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/MOT-17-evaluation-(private-detector))  * [StrongSORT MOT16 ablation study](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/Yolov5DeepSORTwithOSNet-vs-Yolov5StrongSORTwithOSNet-ablation-study-on-MOT16)  * [Yolov5 StrongSORT OSNet vs other trackers MOT16 (deprecated)](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/MOT-16-evaluation) 
#### News * HybridSORT available (August 2023) * SOTA CLIP-ReID people and vehicle models available (August 2023) ## Why BOXMOT? Today's multi-object tracking options are heavily dependant on the computation capabilities of the underlaying hardware. BOXMOT provides a great variety of setup options that meet different hardware limitations: CPU only, low memory GPUs... Everything is designed with simplicity and flexibility in mind. If you don't get good tracking results on your custom dataset with the out-of-the-box tracker configurations, use the `examples/evolve.py` script for tracker hyperparameter tuning. ## Installation Start with [**Python>=3.8**](https://www.python.org/) environment. If you want to run the YOLOv8, YOLO-NAS or YOLOX examples: ``` git clone https://github.com/mikel-brostrom/yolo_tracking.git cd yolo_tracking pip install -v -e . ``` but if you only want to import the tracking modules you can simply: ``` pip install boxmot ``` ## YOLOv8 | YOLO-NAS | YOLOX examples
Tracking
Yolo models ```bash $ python examples/track.py --yolo-model yolov8n # bboxes only python examples/track.py --yolo-model yolo_nas_s # bboxes only python examples/track.py --yolo-model yolox_n # bboxes only yolov8n-seg # bboxes + segmentation masks yolov8n-pose # bboxes + pose estimation ```
Tracking methods ```bash $ python examples/track.py --tracking-method deepocsort strongsort ocsort bytetrack botsort ```
Tracking sources Tracking can be run on most video formats ```bash $ python examples/track.py --source 0 # webcam img.jpg # image vid.mp4 # video path/ # directory path/*.jpg # glob 'https://youtu.be/Zgi9g1ksQHc' # YouTube 'rtsp://example.com/media.mp4' # RTSP, RTMP, HTTP stream ```
Select ReID model Some tracking methods combine appearance description and motion in the process of tracking. For those which use appearance, you can choose a ReID model based on your needs from this [ReID model zoo](https://kaiyangzhou.github.io/deep-person-reid/MODEL_ZOO). These model can be further optimized for you needs by the [reid_export.py](https://github.com/mikel-brostrom/yolo_tracking/blob/master/boxmot/deep/reid_export.py) script ```bash $ python examples/track.py --source 0 --reid-model lmbn_n_cuhk03_d.pt # lightweight osnet_x0_25_market1501.pt mobilenetv2_x1_4_msmt17.engine resnet50_msmt17.onnx osnet_x1_0_msmt17.pt clip_market1501.pt # heavy clip_vehicleid.pt ... ```
Filter tracked classes By default the tracker tracks all MS COCO classes. If you want to track a subset of the classes that you model predicts, add their corresponding index after the classes flag, ```bash python examples/track.py --source 0 --yolo-model yolov8s.pt --classes 16 17 # COCO yolov8 model. Track cats and dogs, only ``` [Here](https://tech.amikelive.com/node-718/what-object-categories-labels-are-in-coco-dataset/) is a list of all the possible objects that a Yolov8 model trained on MS COCO can detect. Notice that the indexing for the classes in this repo starts at zero
MOT compliant results Can be saved to your experiment folder `runs/track/exp*/` by ```bash python examples/track.py --source ... --save-mot ```

Evaluation Evaluate a combination of detector, tracking method and ReID model on standard MOT dataset or you custom one by ```bash $ python3 examples/val.py --yolo-model yolo_nas_s.pt --reid-model osnetx1_0_dukemtcereid.pt --tracking-method deepocsort --benchmark MOT16 --yolo-model yolox_n.pt --reid-model osnet_ain_x1_0_msmt17.pt --tracking-method ocsort --benchmark MOT17 --yolo-model yolov8s.pt --reid-model lmbn_n_market.pt --tracking-method strongsort --benchmark ```
Evolution We use a fast and elitist multiobjective genetic algorithm for tracker hyperparameter tuning. By default the objectives are: HOTA, MOTA, IDF1. Run it by ```bash $ python examples/evolve.py --tracking-method strongsort --benchmark MOT17 --n-trials 100 # tune strongsort for MOT17 --tracking-method ocsort --benchmark --objective HOTA # tune ocsort for maximizing HOTA on your custom tracking dataset ``` The set of hyperparameters leading to the best HOTA result are written to the tracker's config file.

Custom object detection model example

Minimalistic ```python import cv2 import numpy as np from pathlib import Path from boxmot import DeepOCSORT tracker = DeepOCSORT( model_weights=Path('osnet_x0_25_msmt17.pt'), # which ReID model to use device='cuda:0', fp16=False, ) vid = cv2.VideoCapture(0) while True: ret, im = vid.read() # substitute by your object detector, output has to be N X (x, y, x, y, conf, cls) dets = np.array([[144, 212, 578, 480, 0.82, 0], [425, 281, 576, 472, 0.56, 65]]) tracks = tracker.update(dets, im) # --> (x, y, x, y, id, conf, cls, ind) ```
Complete ```python import cv2 import numpy as np from pathlib import Path from boxmot import DeepOCSORT tracker = DeepOCSORT( model_weights=Path('osnet_x0_25_msmt17.pt'), # which ReID model to use device='cuda:0', fp16=True, ) vid = cv2.VideoCapture(0) color = (0, 0, 255) # BGR thickness = 2 fontscale = 0.5 while True: ret, im = vid.read() # substitute by your object detector, input to tracker has to be N X (x, y, x, y, conf, cls) dets = np.array([[144, 212, 578, 480, 0.82, 0], [425, 281, 576, 472, 0.56, 65]]) tracks = tracker.update(dets, im) # --> (x, y, x, y, id, conf, cls, ind) xyxys = tracks[:, 0:4].astype('int') # float64 to int ids = tracks[:, 4].astype('int') # float64 to int confs = tracks[:, 5] clss = tracks[:, 6].astype('int') # float64 to int inds = tracks[:, 7].astype('int') # float64 to int # in case you have segmentations or poses alongside with your detections you can use # the ind variable in order to identify which track is associated to each seg or pose by: # segs = segs[inds] # poses = poses[inds] # you can then zip them together: zip(tracks, poses) # print bboxes with their associated id, cls and conf if tracks.shape[0] != 0: for xyxy, id, conf, cls in zip(xyxys, ids, confs, clss): im = cv2.rectangle( im, (xyxy[0], xyxy[1]), (xyxy[2], xyxy[3]), color, thickness ) cv2.putText( im, f'id: {id}, conf: {conf}, c: {cls}', (xyxy[0], xyxy[1]-10), cv2.FONT_HERSHEY_SIMPLEX, fontscale, color, thickness ) # show image with bboxes, ids, classes and confidences cv2.imshow('frame', im) # break on pressing q if cv2.waitKey(1) & 0xFF == ord('q'): break vid.release() cv2.destroyAllWindows() ```

Contributors

Contact

For Yolo tracking bugs and feature requests please visit GitHub Issues. For business inquiries or professional support requests please send an email to: yolov5.deepsort.pytorch@gmail.com

Owner

  • Login: lastonton
  • Kind: user

Citation (CITATION.cff) 

cff-version: 1.2.0
preferred-citation:
  type: software
  message: "If you use Yolo Tracking, please cite it as below."
  authors:
  - family-names: Broström
    given-names: Mikel
  title: "BoxMOT: pluggable SOTA tracking modules for object detection, segmentation and pose estimation models"
  version: 10.0.43
  doi: https://zenodo.org/record/7629840
  date-released: 2023-9
  license: AGPL-3.0
  url: "https://github.com/mikel-brostrom/yolo_tracking"

GitHub Events

Total
Last Year

Dependencies

.github/workflows/ci.yml actions
  • actions/checkout v4 composite
  • actions/setup-python v4 composite
.github/workflows/publish.yml actions
  • actions/checkout v4 composite
  • actions/create-release v1 composite
  • actions/setup-python v4 composite
  • pypa/gh-action-pypi-publish b7f401de30cb6434a1e19f805ff006643653240e composite
.github/workflows/stale.yml actions
  • actions/stale v8 composite
Dockerfile docker
  • nvcr.io/nvidia/pytorch 22.11-py3 build
requirements.txt pypi
  • GitPython >=3.1.0
  • PyYAML >=5.3.1
  • filterpy >=1.4.5
  • ftfy >=6.1.1
  • gdown >=4.7.1
  • lapx >=0.5.4
  • loguru >=0.7.0
  • numpy ==1.24.4
  • opencv-python >=4.6.0
  • pandas >=1.1.4
  • pre-commit >=3.3.3
  • regex >=2023.6.3
  • scikit-learn >=1.3.0
  • tensorboard >=2.13.0
  • torch >=1.7.0
  • torchvision >=0.8.1
  • yacs >=0.1.8
setup.py pypi