boxmot

BoxMOT: Pluggable SOTA multi-object tracking modules modules for segmentation, object detection and pose estimation models

https://github.com/mikel-brostrom/boxmot

Science Score: 77.0%

This score indicates how likely this project is to be science-related based on various indicators:

✓
CITATION.cff file
Found CITATION.cff file
✓
codemeta.json file
Found codemeta.json file
✓
.zenodo.json file
Found .zenodo.json file
✓
DOI references
Found 3 DOI reference(s) in README
✓
Academic publication links
Links to: arxiv.org, zenodo.org
✓
Committers with academic emails
1 of 46 committers (2.2%) from academic institutions
○
Institutional organization owner
○
JOSS paper metadata
○
Scientific vocabulary similarity
Low similarity (12.4%) to scientific vocabulary

Keywords

boosttrack botsort bytetrack clip deep-learning deepocsort improvedassociation machine-learning mot mots multi-object-tracking multi-object-tracking-segmentation ocsort oriented-bounding-box-tracking osnet segmentation strongsort tensorrt tracking-by-detection yolo

Keywords from Contributors

cryptocurrencies mesh argument-parser pypi algorithmic-trading freqtrade telegram-bot trade trading-bot dag

Scientific Fields

Artificial Intelligence and Machine Learning Computer Science - 36% confidence

Last synced: 4 months ago · JSON representation ·

Repository

BoxMOT: Pluggable SOTA multi-object tracking modules modules for segmentation, object detection and pose estimation models

Basic Info

Host: GitHub
Owner: mikel-brostrom
License: agpl-3.0
Language: Python
Default Branch: master
Homepage: https://deepwiki.com/mikel-brostrom/boxmot
Size: 127 MB

Statistics

Stars: 7,639
Watchers: 62
Forks: 1,843
Open Issues: 8
Releases: 0

Topics

Created over 5 years ago · Last pushed 4 months ago

Metadata Files

Readme Contributing Funding License Citation

README.md

BoxMOT: Pluggable SOTA multi-object tracking modules for segmentation, object detection and pose estimation models

[![CI](https://github.com/mikel-brostrom/yolov8_tracking/actions/workflows/ci.yml/badge.svg)](https://github.com/mikel-brostrom/yolov8_tracking/actions/workflows/ci.yml) [![PyPI version](https://badge.fury.io/py/boxmot.svg)](https://badge.fury.io/py/boxmot) [![downloads](https://static.pepy.tech/badge/boxmot)](https://pepy.tech/project/boxmot) [![license](https://img.shields.io/badge/license-AGPL%203.0-blue)](https://github.com/mikel-brostrom/boxmot/blob/master/LICENSE) [![python-version](https://img.shields.io/pypi/pyversions/boxmot)](https://badge.fury.io/py/boxmot) [![colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/18nIqkBr68TkK8dHdarxTco6svHUJGggY?usp=sharing) [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.8132989.svg)](https://doi.org/10.5281/zenodo.8132989) [![docker pulls](https://img.shields.io/docker/pulls/boxmot/boxmot?logo=docker)](https://hub.docker.com/r/boxmot/boxmot) [![discord](https://img.shields.io/discord/1377565354326495283?logo=discord&label=discord&labelColor=fff&color=5865f2)](https://discord.gg/tUmFEcYU4q) [![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/mikel-brostrom/boxmot)

🚀 Key Features

Pluggable Architecture
Easily swap in/out SOTA multi-object trackers.
Universal Model Support
Integrate with any segmentation, object-detection and pose-estimation models that outputs bounding boxes
Benchmark-Ready
Local evaluation pipelines for MOT17, MOT20, and DanceTrack ablation datasets with "official" ablation detectors
Performance Modes
- Motion-only: for lightweight, CPU-efficient, high-FPS performance
- Motion + Appearance: Combines motion cues with appearance embeddings (CLIPReID, LightMBN, OSNet) to maximize identity consistency and accuracy at a higher computational cost
Reusable Detections & Embeddings
Save once, run evaluations with no redundant preprocessing lightning fast.

📊 Benchmark Results (MOT17 ablation split)

| Tracker | Status | HOTA↑ | MOTA↑ | IDF1↑ | FPS | | :-----: | :-----: | :---: | :---: | :---: | :---: | | [boosttrack](https://arxiv.org/abs/2408.13003) | ✅ | 69.253 | 75.914 | 83.206 | 25 | | [botsort](https://arxiv.org/abs/2206.14651) | ✅ | 68.885 | 78.222 | 81.344 | 46 | | [strongsort](https://arxiv.org/abs/2202.13514) | ✅ | 68.05 | 76.185 | 80.763 | 17 | | [deepocsort](https://arxiv.org/abs/2302.11813) | ✅ | 67.796 | 75.868 | 80.514 | 12 | | [bytetrack](https://arxiv.org/abs/2110.06864) | ✅ | 67.68 | 78.039 | 79.157 | 1265 | | [ocsort](https://arxiv.org/abs/2203.14360) | ✅ | 66.441 | 74.548 | 77.899 | 1483 | _{NOTES: Evaluation was conducted on the second half of the MOT17 training set, as the validation set is not publicly available and the ablation detector was trained on the first half. We employed [pre-generated detections and embeddings](https://github.com/mikel-brostrom/boxmot/releases/download/v11.0.9/runs2.zip). Each tracker was configured using the default parameters from their official repositories.}

🔧 Installation

Install the boxmot package, including all requirements, in a Python>=3.9 environment:

bash pip install boxmot

If you want to contribute to this package check how to contribute here

💻 CLI

BoxMOT provides a unified CLI boxmot with the following subcommands:

```bash Usage: boxmot COMMAND [ARGS]...

Commands: track Run tracking only generate Generate detections and embeddings eval Evaluate tracking performance using the official trackeval repository tune Tune tracker hyperparameters based on selected detections and embeddings ```

🐍 PYTHON

Seamlessly integrate BoxMOT directly into your Python MOT applications with your custom model.

```python import cv2 import torch import numpy as np from pathlib import Path from boxmot import BoostTrack from torchvision.models.detection import ( fasterrcnnresnet50fpnv2, FasterRCNNResNet50FPNV2_Weights as Weights )

Set device

device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

Load model with pretrained weights and preprocessing transforms

weights = Weights.DEFAULT model = fasterrcnnresnet50fpnv2(weights=weights, boxscore_thresh=0.5) model.to(device).eval() transform = weights.transforms()

Initialize tracker

tracker = BoostTrack(reidweights=Path('osnetx025msmt17.pt'), device=device, half=False)

Start video capture

cap = cv2.VideoCapture(0)

with torch.inference_mode(): while True: success, frame = cap.read() if not success: break

    # Convert frame to RGB and prepare for model
    rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
    tensor = torch.from_numpy(rgb).permute(2, 0, 1).to(torch.uint8)
    input_tensor = transform(tensor).to(device)

    # Run detection
    output = model([input_tensor])[0]
    scores = output['scores'].cpu().numpy()
    keep = scores >= 0.5

    # Prepare detections for tracking
    boxes = output['boxes'][keep].cpu().numpy()
    labels = output['labels'][keep].cpu().numpy()
    filtered_scores = scores[keep]
    detections = np.concatenate([boxes, filtered_scores[:, None], labels[:, None]], axis=1)

    # Update tracker and draw results
    #   INPUT:  M X (x, y, x, y, conf, cls)
    #   OUTPUT: M X (x, y, x, y, id, conf, cls, ind)
    res = tracker.update(detections, frame)
    tracker.plot_results(frame, show_trajectories=True)

    # Show output
    cv2.imshow('BoXMOT + Torchvision', frame)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

Clean up

cap.release() cv2.destroyAllWindows() ```

📝 Code Examples & Tutorials

Tracking

```bash $ boxmot track --yolo-model rf-detr-base.pt # bboxes only boxmot track --yolo-model yolox_s.pt # bboxes only boxmot track --yolo-model yolo12n.pt # bboxes only boxmot track --yolo-model yolo11n.pt # bboxes only boxmot track --yolo-model yolov10n.pt # bboxes only boxmot track --yolo-model yolov9c.pt # bboxes only boxmot track --yolo-model yolov8n.pt # bboxes only yolov8n-seg.pt # bboxes + segmentation masks yolov8n-pose.pt # bboxes + pose estimation ```

Tracking methods

```bash $ boxmot track --tracking-method deepocsort strongsort ocsort bytetrack botsort boosttrack ```

Tracking sources

Tracking can be run on most video formats ```bash $ boxmot track --source 0 # webcam img.jpg # image vid.mp4 # video path/ # directory path/*.jpg # glob 'https://youtu.be/Zgi9g1ksQHc' # YouTube 'rtsp://example.com/media.mp4' # RTSP, RTMP, HTTP stream ```

Select ReID model

Some tracking methods combine appearance description and motion in the process of tracking. For those which use appearance, you can choose a ReID model based on your needs from this [ReID model zoo](https://kaiyangzhou.github.io/deep-person-reid/MODEL_ZOO). These model can be further optimized for you needs by the [reid_export.py](https://github.com/mikel-brostrom/yolo_tracking/blob/master/boxmot/appearance/reid_export.py) script ```bash $ boxmot track --source 0 --reid-model lmbn_n_cuhk03_d.pt # lightweight osnet_x0_25_market1501.pt mobilenetv2_x1_4_msmt17.engine resnet50_msmt17.onnx osnet_x1_0_msmt17.pt clip_market1501.pt # heavy clip_vehicleid.pt ... ```

Filter tracked classes

By default the tracker tracks all MS COCO classes. If you want to track a subset of the classes that you model predicts, add their corresponding index after the classes flag, ```bash boxmot track --source 0 --yolo-model yolov8s.pt --classes 16 17 # COCO yolov8 model. Track cats and dogs, only ``` [Here](https://tech.amikelive.com/node-718/what-object-categories-labels-are-in-coco-dataset/) is a list of all the possible objects that a Yolov8 model trained on MS COCO can detect. Notice that the indexing for the classes in this repo starts at zero

Evaluation

Evaluate a combination of detector, tracking method and ReID model on standard MOT dataset or you custom one by ```bash # reproduce MOT17 README results $ boxmot eval --yolo-model yolox_x_MOT17_ablation.pt --reid-model lmbn_n_duke.pt --tracking-method boosttrack --source MOT17-ablation --verbose # MOT20 results $ boxmot eval --yolo-model yolox_x_MOT20_ablation.pt --reid-model lmbn_n_duke.pt --tracking-method boosttrack --source MOT20-ablation --verbose # Dancetrack results $ boxmot eval --yolo-model yolox_x_dancetrack_ablation.pt --reid-model lmbn_n_duke.pt --tracking-method boosttrack --source dancetrack-ablation --verbose # metrics on custom dataset $ boxmot eval --yolo-model yolov8n.pt --reid-model osnet_x0_25_msmt17.pt --tracking-method deepocsort --source ./assets/MOT17-mini/train --verbose ``` add `--gsi` to your command for postprocessing the MOT results by gaussian smoothed interpolation. Detections and embeddings are stored for the selected YOLO and ReID model respectively. They can then be loaded into any tracking algorithm. Avoiding the overhead of repeatedly generating this data.

Evolution

We use a fast and elitist multiobjective genetic algorithm for tracker hyperparameter tuning. By default the objectives are: HOTA, MOTA, IDF1. Run it by ```bash # saves dets and embs under ./runs/dets_n_embs separately for each selected yolo and reid model $ boxmot generate --source ./assets/MOT17-mini/train --yolo-model yolov8n.pt yolov8s.pt --reid-model weights/osnet_x0_25_msmt17.pt # evolve parameters for specified tracking method using the selected detections and embeddings generated in the previous step $ boxmot tune --yolo-model yolov8n.pt --reid-model osnet_x0_25_msmt17.pt --n-trials 9 --tracking-method botsort --source ./assets/MOT17-mini/train ``` The set of hyperparameters leading to the best HOTA result are written to the tracker's config file.

Export

We support ReID model export to ONNX, OpenVINO, TorchScript and TensorRT ```bash # export to ONNX $ python3 boxmot/appearance/reid_export.py --include onnx --device cpu # export to OpenVINO $ python3 boxmot/appearance/reid_export.py --include openvino --device cpu # export to TensorRT with dynamic input $ python3 boxmot/appearance/reid_export.py --include engine --device 0 --dynamic ```

| Example Description | Notebook | |---------------------|----------| | Torchvision bounding box tracking with BoxMOT | [![Notebook](https://img.shields.io/badge/Notebook-torchvision_det_boxmot.ipynb-blue)](examples/det/torchvision_boxmot.ipynb) | | Torchvision pose tracking with BoxMOT | [![Notebook](https://img.shields.io/badge/Notebook-torchvision_pose_boxmot.ipynb-blue)](examples/pose/torchvision_boxmot.ipynb) | | Torchvision segmentation tracking with BoxMOT | [![Notebook](https://img.shields.io/badge/Notebook-torchvision_seg_boxmot.ipynb-blue)](examples/seg/torchvision_boxmot.ipynb) |

Contributors

Contact

For BoxMOT bugs and feature requests please visit GitHub Issues. For business inquiries or professional support requests please send an email to: box-mot@outlook.com

Owner

Name: Mike
Login: mikel-brostrom
Kind: user
Location: Sweden ⇄ Spain

Repositories: 38
Profile: https://github.com/mikel-brostrom

Applied and R&D ML

Citation (CITATION.cff)

cff-version: 15.0.2
preferred-citation:
  type: software
  message: "If you use Yolo Tracking, please cite it as below."
  authors:
  - family-names: Broström
    given-names: Mikel
  title: "BoxMOT: pluggable SOTA tracking modules for object detection, segmentation and pose estimation models"
  version: 15.0.2
  doi: https://zenodo.org/record/7629840
  date-released: 2024-6
  license: AGPL-3.0
  url: "https://github.com/mikel-brostrom/boxmot"

Committers

Last synced: 8 months ago

All Time

Total Commits: 3,047
Total Committers: 46
Avg Commits per committer: 66.239
Development Distribution Score (DDS): 0.356

Past Year

Commits: 969
Committers: 19
Avg Commits per committer: 51.0
Development Distribution Score (DDS): 0.42

Top Committers

Name	Email	Commits
mikel.brostrom	m**m@c**e	1,961
Mike	y**h@g**m	650
Mikel Broström	m**m@z**u	134
mikel-brostrom	m**0@g**m	43
Henrik	h**d@h**m	39
Fleyderer	k**v@u**u	38
renovate[bot]	2****]	37
Mike	m**m@o**m	26
LilBabines	a**r@o**r	21
dependabot[bot]	4****]	17
edblu1	e**l@g**e	9
rolson24	r**4@d**u	8
Justin Ruan	j**9@g**m	5
Johnny	j**3@g**m	5
mohit.gaikwad@wobot.ai	m**d@w**i	4
mikel	m**l@T**t	4
Saurabh Khanduja	s**a@i**i	4
beykun18	6****8	3
Shanliang Yao	1**6@q**m	3
Chanwut (Mick) Kittivorawong	3****k	3
AshwinSakhare	a**e@z**m	2
Armin	m**n@g**m	2
Usama Imdad	u**n@g**m	2
florian-fischer-swarm	f**r@s**m	2
jjaegii	h**8@g**m	2
rslim	r**m@g**m	2
Henry	h**v@g**m	2
youngjae-avikus	y**e@k**m	1
mohamedsamirx	9****x	1
liuweiqing	l**7@g**m	1
and 16 more...

Committer Domains (Top 20 + Academic)

qq.com: 2 cevt.se: 1 zeekrtech.eu: 1 ugpa.ru: 1 orange.fr: 1 gmx.de: 1 dartmouth.edu: 1 wobot.ai: 1 t4-3.bkcgtfw3lw3urakoekcqtzjyhh.ax.internal.cloudapp.net: 1 ikara.ai: 1 zdatainc.com: 1 swarm-analytics.com: 1 kakao.com: 1 fv-az1111-68.s3eip0qwctaupit5o3ahr5xzra.dx.internal.cloudapp.net: 1 fv-az448-180.gd0zyr0xj03ufix5kzlzvjjcld.gx.internal.cloudapp.net: 1 fv-az586-63.032bdyo3y4renostddu1d2ndmc.jx.internal.cloudapp.net: 1 scenera.net: 1 sngular.com: 1

Issues and Pull Requests

Last synced: 4 months ago

All Time

Total issues: 294
Total pull requests: 591
Average time to close issues: 19 days
Average time to close pull requests: 4 days
Total issue authors: 198
Total pull request authors: 39
Average comments per issue: 3.02
Average comments per pull request: 0.63
Merged pull requests: 310
Bot issues: 2
Bot pull requests: 186

Past Year

Issues: 153
Pull requests: 464
Average time to close issues: 11 days
Average time to close pull requests: 3 days
Issue authors: 97
Pull request authors: 21
Average comments per issue: 2.43
Average comments per pull request: 0.67
Merged pull requests: 229
Bot issues: 0
Bot pull requests: 154

View more stats

Top Authors

Issue Authors

mikel-brostrom (11)
microchila (9)
wk565 (8)
17860421876 (7)
fcu52005505 (6)
starsky68 (5)
Fleyderer (5)
eeric (4)
ghost (4)
22xxlin (4)
Jackson-leo (4)
alaxahml (4)
MorningHero92 (4)
Mirshal (4)
davide-bontempelli-zupit (3)

Pull Request Authors

mikel-brostrom (334)
renovate[bot] (97)
dependabot[bot] (54)
github-actions[bot] (47)
Fleyderer (20)
ahmadmughees (9)
usamaimdadsian (4)
emirhanbayar (4)
huanghany (4)
minicom365 (3)
marvel2010 (2)
14790897 (2)
MlaikahImdad (2)
asakhare-zdata (2)
LilBabines (2)

Top Labels

Issue Labels

question (123) Stale (65) bug (41) enhancement (34) help wanted (7) triage (6) tracking (6) engine (5) good first issue (3) reid (2) botsort (1) kf (1) strongsort (1) tuning (1) bytetrack (1) evaluation (1) boosttrack (1)

Pull Request Labels

Stale (74) dependencies (54) python (53) codex (6) question (3) github_actions (1) bug (1)

Packages

Total packages: 1
Total downloads:
- pypi 15,244 last-month

Total dependent packages: 1
Total dependent repositories: 1
Total versions: 122
Total maintainers: 1

pypi.org: boxmot

BoxMOT: pluggable SOTA tracking modules for segmentation, object detection and pose estimation models

Documentation: https://boxmot.readthedocs.io/
License: AGPL-3.0
Latest release: 15.0.2
published 5 months ago

Versions: 122
Dependent Packages: 1
Dependent Repositories: 1
Downloads: 15,244 Last month
Docker Downloads: 0

Rankings

Stargazers count: 0.4%

Forks count: 1.1%

Downloads: 4.3%

Docker downloads count: 4.4%

Dependent packages count: 4.7%

Average: 6.1%

Dependent repos count: 21.8%

Maintainers (1)

mike-b

Last synced: 4 months ago

Dependencies

.github/workflows/ci.yml actions

actions/checkout v4 composite
actions/setup-python v4 composite

.github/workflows/publish.yml actions

actions/checkout v4 composite
actions/create-release v1 composite
actions/setup-python v4 composite
pypa/gh-action-pypi-publish b7f401de30cb6434a1e19f805ff006643653240e composite

.github/workflows/stale.yml actions

actions/stale v8 composite

Dockerfile docker

nvcr.io/nvidia/pytorch 22.11-py3 build

requirements.txt pypi

GitPython >=3.1.0
PyYAML >=5.3.1
filterpy >=1.4.5
ftfy >=6.1.1
gdown >=4.7.1
lapx >=0.5.4
loguru >=0.7.0
numpy ==1.24.4
opencv-python >=4.6.0
pandas >=1.1.4
pre-commit >=3.3.3
regex >=2023.6.3
scikit-learn >=1.3.0
tensorboard >=2.13.0
torch >=1.7.0
torchvision >=0.8.1
yacs >=0.1.8

setup.py pypi