Data

Tools

Indexes

Applications

Experiments

Open Source Science

ada-track

Offical implementation of CVPR2024 paper ADA-Track: End-to-End Multi-Camera 3D Multi-Object Tracking with Alternating Detection and Association.

https://github.com/dsx0511/ada-track

Science Score: 54.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ✓
    CITATION.cff file
    Found CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ✓
    Academic publication links
    Links to: arxiv.org
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (9.5%) to scientific vocabulary
Last synced: 6 months ago · JSON representation ·

Repository

Offical implementation of CVPR2024 paper ADA-Track: End-to-End Multi-Camera 3D Multi-Object Tracking with Alternating Detection and Association.

Basic Info
  • Host: GitHub
  • Owner: dsx0511
  • License: other
  • Language: Python
  • Default Branch: main
  • Size: 2.39 MB
Statistics
  • Stars: 93
  • Watchers: 2
  • Forks: 12
  • Open Issues: 8
  • Releases: 0
Created about 2 years ago · Last pushed about 1 year ago
Metadata Files
Readme License Citation

README.md

[CVPR2024] ADA-Track: End-to-End Multi-Camera 3D Multi-Object Tracking with Alternating Detection and Association

This is the official implementation of the CVPR'24 paper

ADA-Track: End-to-End Multi-Camera 3D Multi-Object Tracking with Alternating Detection and Association
Shuxiao Ding, Lukas Schneider, Marius Cordts, Juergen Gall

Arxiv (CVPR version) | CVF Open Access | Arxiv (Extended version)

https://github.com/dsx0511/ADA-Track/assets/33251274/35a46ef9-a2c9-4c26-beb8-3246438004a0

Abstract

Query-based 3D Multi-Object Tracking (MOT) facilitates seamless integration into end-to-end frameworks. Many existing methods adopt the tracking-by-attention paradigm, utilizing track queries for identity-consistent detection and object queries for identity-agnostic track spawning. Tracking-by-attention, however, entangles detection and tracking queries in one embedding for both the detection and tracking task, which is sub-optimal. Other approaches resemble the tracking-by-detection paradigm, detecting objects using decoupled track and detection queries followed by a subsequent association. These methods, however, do not leverage synergies between the detection and association task. Combining the strengths of both paradigms, we introduce ADA-Track, a novel end-to-end framework for 3D MOT from multi-view cameras. We introduce a learnable data association module based on edge-augmented cross-attention, leveraging appearance and geometric features. Furthermore, we integrate this association module into the decoder layer of a DETR-based 3D detector, enabling simultaneous DETR-like query-to-image cross-attention for detection and query-to-query cross-attention for data association. By stacking these decoder layers, queries are refined for the detection and association task alternately, effectively harnessing the task dependencies. We evaluate our method on the nuScenes dataset and demonstrate the advantage of our approach compared to the two previous paradigms.

News

  • [2024/12/13] We release an extended version, ADA-Track++. The arxiv paper can be accessed here.
  • [2024/02/26] Our paper is accepted by CVPR 2024! 🎉🎉
  • [2024/05/14] We release our source code.

Getting started

  1. Installation
  2. Data preparation
  3. Training & inference

Experimental results

| Detector | Backbone | N_GPU | AMOTA | AMOTP | Logfile | Checkpoint | |---|---|---|---|---|---|---| | DETR3D | ResNet-101 | 4 | 38.4 | 1.378 | log | Download | | PETR | V2-99 | 8 | 48.3 | 1.242 | log | Download |

NOTE: After fixing a small issue in the code, we achieved slightly better performance than reported in the paper ;-)

License

See LICENSE for more details.

Citation

If you find ADA-Track is helpful for your research, please give us a star and cite it by: @inproceedings{ding2024adatrack, title={ADA-Track: End-to-End Multi-Camera 3D Multi-Object Tracking with Alternating Detection and Association}, author={Ding, Shuxiao and Schneider, Lukas and Cordts, Marius and Gall, Juergen}, booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}, year={2024} }

Acknowledgement

Our implementation is a plugin to mmdetection3d, and it is highly dependent on MUTR3D. We build our tracker based on the implementation of the detectors DETR3D and PETR. The success of this work is impossible without these aforementioned projects as well as many other high-quality open-source projects. Many thanks to them for enabling our research!

Owner

  • Name: Shuxiao Ding
  • Login: dsx0511
  • Kind: user
  • Location: Stuttgart & Nanjing
  • Company: Mercedes-Benz & University of Bonn

PhD student at Mercedes-Benz and University of Bonn. Focus on Graph Neural Network applications in traffic scenes.

Citation (CITATION.cff) 

cff-version: 1.2.0
message: "If you use this software, please cite it as below."
authors:
  - family-names: Ding
    given-names: Shuxiao
    orcid: https://orcid.org/0000-0002-4040-5585
title: "ADA-Track: End-to-End Multi-Camera 3D Multi-Object Tracking with Alternating Detection and Association"
date-released: 2024-05-14
url: "https://github.com/dsx0511/ADA-Track"

GitHub Events

Total
  • Issues event: 9
  • Watch event: 40
  • Delete event: 1
  • Issue comment event: 9
  • Push event: 1
  • Fork event: 5
  • Create event: 1
Last Year
  • Issues event: 9
  • Watch event: 40
  • Delete event: 1
  • Issue comment event: 9
  • Push event: 1
  • Fork event: 5
  • Create event: 1

Issues and Pull Requests

Last synced: 6 months ago

All Time
  • Total issues: 6
  • Total pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Total issue authors: 5
  • Total pull request authors: 0
  • Average comments per issue: 0.0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 6
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 5
  • Pull request authors: 0
  • Average comments per issue: 0.0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • CMSC740Student (2)
  • prasadpr09 (2)
  • pertain1ng (1)
  • 504058568 (1)
  • zeno0119 (1)
  • hibernationT (1)
  • Yibin122 (1)
  • PG-Wang (1)
  • zhao913 (1)
  • yiyi-2002-1225 (1)
  • tmdwo8814 (1)
  • jingyanliao (1)
  • xiao10ma (1)
  • sieafa (1)
  • JingweiZhang12 (1)
Pull Request Authors
Top Labels
Issue Labels
Pull Request Labels

Dependencies

docker/Dockerfile docker
  • pytorch/pytorch ${PYTORCH}-cuda${CUDA}-cudnn${CUDNN}-devel build
setup.py pypi
  • mlflow *
  • mmsegmentation *
  • pytorch3d *