mmaction2

OpenMMLab's Next Generation Video Understanding Toolbox and Benchmark

Science Score: 64.0%

This score indicates how likely this project is to be science-related based on various indicators:

✓
CITATION.cff file
Found CITATION.cff file
✓
codemeta.json file
Found codemeta.json file
✓
.zenodo.json file
Found .zenodo.json file
○
DOI references
✓
Academic publication links
Links to: arxiv.org
✓
Committers with academic emails
3 of 76 committers (3.9%) from academic institutions
○
Institutional organization owner
○
JOSS paper metadata
○
Scientific vocabulary similarity
Low similarity (9.9%) to scientific vocabulary

Keywords

action-recognition ava benchmark deep-learning i3d non-local openmmlab posec3d pytorch slowfast spatial-temporal-action-detection temporal-action-localization tsm tsn uniformerv2 video-classification video-understanding x3d

Keywords from Contributors

swin-transformer image-editing aigc clip multimodal pretrained-models video-frame-interpolation text2image super-resolution matting

Last synced: 6 months ago · JSON representation ·

Repository

OpenMMLab's Next Generation Video Understanding Toolbox and Benchmark

Basic Info

Host: GitHub
Owner: open-mmlab
License: apache-2.0
Language: Python
Default Branch: main
Homepage: https://mmaction2.readthedocs.io
Size: 68.2 MB

Statistics

Stars: 4,731
Watchers: 41
Forks: 1,306
Open Issues: 312
Releases: 28

Topics

Created over 5 years ago · Last pushed over 1 year ago

Metadata Files

Readme License Code of conduct Citation

README.md

OpenMMLab website ^HOT OpenMMLab platform ^{TRY IT OUT}

[![Documentation](https://readthedocs.org/projects/mmaction2/badge/?version=latest)](https://mmaction2.readthedocs.io/en/latest/) [![actions](https://github.com/open-mmlab/mmaction2/workflows/build/badge.svg)](https://github.com/open-mmlab/mmaction2/actions) [![codecov](https://codecov.io/gh/open-mmlab/mmaction2/branch/main/graph/badge.svg)](https://codecov.io/gh/open-mmlab/mmaction2) [![PyPI](https://img.shields.io/pypi/v/mmaction2)](https://pypi.org/project/mmaction2/) [![LICENSE](https://img.shields.io/github/license/open-mmlab/mmaction2.svg)](https://github.com/open-mmlab/mmaction2/blob/main/LICENSE) [![Average time to resolve an issue](https://isitmaintained.com/badge/resolution/open-mmlab/mmaction2.svg)](https://github.com/open-mmlab/mmaction2/issues) [![Percentage of issues still open](https://isitmaintained.com/badge/open/open-mmlab/mmaction2.svg)](https://github.com/open-mmlab/mmaction2/issues) [📘Documentation](https://mmaction2.readthedocs.io/en/latest/) | [🛠️Installation](https://mmaction2.readthedocs.io/en/latest/get_started/installation.html) | [👀Model Zoo](https://mmaction2.readthedocs.io/en/latest/modelzoo_statistics.html) | [🆕Update News](https://mmaction2.readthedocs.io/en/latest/notes/changelog.html) | [🚀Ongoing Projects](https://github.com/open-mmlab/mmaction2/projects) | [🤔Reporting Issues](https://github.com/open-mmlab/mmaction2/issues/new/choose)

English | 简体中文

🥳 🚀 What's New 🔝

The default branch has been switched to main(previous 1.x) from master(current 0.x), and we encourage users to migrate to the latest version with more supported models, stronger pre-training checkpoints and simpler coding. Please refer to Migration Guide for more details.

Release (2023.10.12): v1.2.0 with the following new features:

Support VindLU multi-modality algorithm and the Training of ActionClip
Support lightweight model MobileOne TSN/TSM
Support video retrieval dataset MSVD
Support SlowOnly K700 feature to train localization models
Support Video and Audio Demos

📖 Introduction 🔝

MMAction2 is an open-source toolbox for video understanding based on PyTorch. It is a part of the OpenMMLab project.

Action Recognition on Kinetics-400 (left) and Skeleton-based Action Recognition on NTU-RGB+D-120 (right)

Skeleton-based Spatio-Temporal Action Detection and Action Recognition Results on Kinetics-400

Spatio-Temporal Action Detection Results on AVA-2.1

🎁 Major Features 🔝

Modular design: We decompose a video understanding framework into different components. One can easily construct a customized video understanding framework by combining different modules.
Support five major video understanding tasks: MMAction2 implements various algorithms for multiple video understanding tasks, including action recognition, action localization, spatio-temporal action detection, skeleton-based action detection and video retrieval.
Well tested and documented: We provide detailed documentation and API reference, as well as unit tests.

🛠️ Installation 🔝

MMAction2 depends on PyTorch, MMCV, MMEngine, MMDetection (optional) and MMPose (optional).

Please refer to install.md for detailed instructions.

Quick instructions

```shell conda create --name openmmlab python=3.8 -y conda activate openmmlab conda install pytorch torchvision -c pytorch # This command will automatically install the latest version PyTorch and cudatoolkit, please check whether they match your environment. pip install -U openmim mim install mmengine mim install mmcv mim install mmdet # optional mim install mmpose # optional git clone https://github.com/open-mmlab/mmaction2.git cd mmaction2 pip install -v -e . ```

👀 Model Zoo 🔝

Results and models are available in the model zoo.

Supported model

Action Recognition
C3D (CVPR'2014)	TSN (ECCV'2016)	I3D (CVPR'2017)	C2D (CVPR'2018)	I3D Non-Local (CVPR'2018)
R(2+1)D (CVPR'2018)	TRN (ECCV'2018)	TSM (ICCV'2019)	TSM Non-Local (ICCV'2019)	SlowOnly (ICCV'2019)
SlowFast (ICCV'2019)	CSN (ICCV'2019)	TIN (AAAI'2020)	TPN (CVPR'2020)	X3D (CVPR'2020)
MultiModality: Audio (ArXiv'2020)	TANet (ArXiv'2020)	TimeSformer (ICML'2021)	ActionCLIP (ArXiv'2021)	VideoSwin (CVPR'2022)
VideoMAE (NeurIPS'2022)	MViT V2 (CVPR'2022)	UniFormer V1 (ICLR'2022)	UniFormer V2 (Arxiv'2022)	VideoMAE V2 (CVPR'2023)
Action Localization
BSN (ECCV'2018)	BMN (ICCV'2019)	TCANet (CVPR'2021)
Spatio-Temporal Action Detection
ACRN (ECCV'2018)	SlowOnly+Fast R-CNN (ICCV'2019)	SlowFast+Fast R-CNN (ICCV'2019)	LFB (CVPR'2019)	VideoMAE (NeurIPS'2022)
Skeleton-based Action Recognition
ST-GCN (AAAI'2018)	2s-AGCN (CVPR'2019)	PoseC3D (CVPR'2022)	STGCN++ (ArXiv'2022)	CTRGCN (CVPR'2021)
MSG3D (CVPR'2020)
Video Retrieval
CLIP4Clip (ArXiv'2022)

Supported dataset

Action Recognition
HMDB51 (Homepage) (ICCV'2011)	UCF101 (Homepage) (CRCV-IR-12-01)	ActivityNet (Homepage) (CVPR'2015)	Kinetics-[400/600/700] (Homepage) (CVPR'2017)
SthV1 (ICCV'2017)	SthV2 (Homepage) (ICCV'2017)	Diving48 (Homepage) (ECCV'2018)	Jester (Homepage) (ICCV'2019)
Moments in Time (Homepage) (TPAMI'2019)	Multi-Moments in Time (Homepage) (ArXiv'2019)	HVU (Homepage) (ECCV'2020)	OmniSource (Homepage) (ECCV'2020)
FineGYM (Homepage) (CVPR'2020)	Kinetics-710 (Homepage) (Arxiv'2022)
Action Localization
THUMOS14 (Homepage) (THUMOS Challenge 2014)	ActivityNet (Homepage) (CVPR'2015)	HACS (Homepage) (ICCV'2019)
Spatio-Temporal Action Detection
UCF101-24* (Homepage) (CRCV-IR-12-01)	JHMDB* (Homepage) (ICCV'2015)	AVA (Homepage) (CVPR'2018)	AVA-Kinetics (Homepage) (Arxiv'2020)
MultiSports (Homepage) (ICCV'2021)
Skeleton-based Action Recognition
PoseC3D-FineGYM (Homepage) (ArXiv'2021)	PoseC3D-NTURGB+D (Homepage) (ArXiv'2021)	PoseC3D-UCF101 (Homepage) (ArXiv'2021)	PoseC3D-HMDB51 (Homepage) (ArXiv'2021)
Video Retrieval
MSRVTT (Homepage) (CVPR'2016)

👨‍🏫 Get Started 🔝

For tutorials, we provide the following user guides for basic usage:

Research works built on MMAction2 by users from community

- Video Swin Transformer. [\[paper\]](https://arxiv.org/abs/2106.13230)[\[github\]](https://github.com/SwinTransformer/Video-Swin-Transformer) - Evidential Deep Learning for Open Set Action Recognition, ICCV 2021 **Oral**. [\[paper\]](https://arxiv.org/abs/2107.10161)[\[github\]](https://github.com/Cogito2012/DEAR) - Rethinking Self-supervised Correspondence Learning: A Video Frame-level Similarity Perspective, ICCV 2021 **Oral**. [\[paper\]](https://arxiv.org/abs/2103.17263)[\[github\]](https://github.com/xvjiarui/VFS)

🎫 License 🔝

This project is released under the Apache 2.0 license.

🖊️ Citation 🔝

If you find this project useful in your research, please consider cite:

BibTeX @misc{2020mmaction2, title={OpenMMLab's Next Generation Video Understanding Toolbox and Benchmark}, author={MMAction2 Contributors}, howpublished = {\url{https://github.com/open-mmlab/mmaction2}}, year={2020} }

🙌 Contributing 🔝

We appreciate all contributions to improve MMAction2. Please refer to CONTRIBUTING.md in MMCV for more details about the contributing guideline.

🤝 Acknowledgement 🔝

MMAction2 is an open-source project that is contributed by researchers and engineers from various colleges and companies. We appreciate all the contributors who implement their methods or add new features and users who give valuable feedback. We wish that the toolbox and benchmark could serve the growing research community by providing a flexible toolkit to reimplement existing methods and develop their new models.

🏗️ Projects in OpenMMLab 🔝

MMEngine: OpenMMLab foundational library for training deep learning models.
MMCV: OpenMMLab foundational library for computer vision.
MIM: MIM installs OpenMMLab packages.
MMEval: A unified evaluation library for multiple machine learning libraries.
MMPreTrain: OpenMMLab pre-training toolbox and benchmark.
MMDetection: OpenMMLab detection toolbox and benchmark.
MMDetection3D: OpenMMLab's next-generation platform for general 3D object detection.
MMRotate: OpenMMLab rotated object detection toolbox and benchmark.
MMYOLO: OpenMMLab YOLO series toolbox and benchmark.
MMSegmentation: OpenMMLab semantic segmentation toolbox and benchmark.
MMOCR: OpenMMLab text detection, recognition, and understanding toolbox.
MMPose: OpenMMLab pose estimation toolbox and benchmark.
MMHuman3D: OpenMMLab 3D human parametric model toolbox and benchmark.
MMSelfSup: OpenMMLab self-supervised learning toolbox and benchmark.
MMRazor: OpenMMLab model compression toolbox and benchmark.
MMFewShot: OpenMMLab fewshot learning toolbox and benchmark.
MMAction2: OpenMMLab's next-generation action understanding toolbox and benchmark.
MMTracking: OpenMMLab video perception toolbox and benchmark.
MMFlow: OpenMMLab optical flow toolbox and benchmark.
MMagic: OpenMMLab Advanced, Generative and Intelligent Creation toolbox.
MMGeneration: OpenMMLab image and video generative models toolbox.
MMDeploy: OpenMMLab model deployment framework.
Playground: A central hub for gathering and showcasing amazing projects built upon OpenMMLab.

Owner

Name: OpenMMLab
Login: open-mmlab
Kind: organization
Location: China

Website: https://openmmlab.com
Twitter: OpenMMLab
Repositories: 53
Profile: https://github.com/open-mmlab

Citation (CITATION.cff)

cff-version: 1.2.0
message: "If you use this software, please cite it as below."
authors:
  - name: "MMAction2 Contributors"
title: "OpenMMLab's Next Generation Video Understanding Toolbox and Benchmark"
date-released: 2020-07-21
url: "https://github.com/open-mmlab/mmaction2"
license: Apache-2.0

GitHub Events

Total

Issues event: 13
Watch event: 487
Issue comment event: 63
Pull request event: 7
Fork event: 80

Last Year

Issues event: 13
Watch event: 487
Issue comment event: 63
Pull request event: 7
Fork event: 80

Committers

Last synced: 9 months ago

All Time

Total Commits: 1,623
Total Committers: 76
Avg Commits per committer: 21.355
Development Distribution Score (DDS): 0.712

Past Year

Commits: 0
Committers: 0
Avg Commits per committer: 0.0
Development Distribution Score (DDS): 0.0

Top Committers

Name	Email	Commits
linjintao	l**o@s**m	467
Haodong Duan	d**z@g**m	268
cir7	3****7	134
wxDai	d**n@p**n	128
Kai Hu	k**u@g**m	82
lixuanyi	l**i@s**m	74
xusu	x**u@s**m	72
lizz	l**z@s**m	55
congee	3****4	47
irvingzhang0512	i**2@g**m	45
duanhaodong	d**g@s**m	41
gengenkai	3****i	38
Jiaqi Tang	6**8@q**m	22
chenkai	c**i@s**m	21
jiaomenglei	j**i@s**m	15
wangshiguang	w**g@s**m	11
Rejnald Lleshi	4****i	9
Kai Chen	c**v@g**m	9
LinXiaoZheng	9****o	4
makecent	4****t	4
Yining Li	l**2@g**m	4
Wang Xiao	3****o	4
zhaowangbo.vendor	z**o@p**n	3
wangruohui	1****i	3
Michael P. Camilleri	m**i@e**k	3
Jamie	j****5	3
Shoufa Chen	s**n@1**m	2
Jas	j**g@s**m	2
Ycr	3****i	2
yrqUni	3****i	2
and 46 more...

Committer Domains (Top 20 + Academic)

sensetime.com: 9 163.com: 4 qq.com: 4 pjlab.org.cn: 3 linkernetworks.com: 1 siat.ac.cn: 1 student.liu.se: 1 126.com: 1 smail.nju.edu.cn: 1 sjtu.edu.cn: 1 g105e1900907.domain.sensetime.com: 1 ed.ac.uk: 1

Issues and Pull Requests

Last synced: 6 months ago

All Time

Total issues: 308
Total pull requests: 151
Average time to close issues: 21 days
Average time to close pull requests: 21 days
Total issue authors: 215
Total pull request authors: 51
Average comments per issue: 2.7
Average comments per pull request: 1.17
Merged pull requests: 75
Bot issues: 0
Bot pull requests: 0

Past Year

Issues: 18
Pull requests: 7
Average time to close issues: 6 days
Average time to close pull requests: 4 minutes
Issue authors: 18
Pull request authors: 4
Average comments per issue: 0.56
Average comments per pull request: 0.71
Merged pull requests: 0
Bot issues: 0
Bot pull requests: 0

View more stats

Top Authors

Issue Authors

Ishihara-Masabumi (16)
KunLiam (7)
Sunshulong (5)
ya-92626 (5)
MABatin (5)
aixiaodewugege (5)
rlleshi (5)
xqc-qc (4)
Aidenxu0305 (4)
JasseurHadded1 (4)
makecent (3)
rouge012 (3)
commandermsv (3)
wwdok (3)
Better369 (3)

Pull Request Authors

cir7 (45)
Dai-Wenxun (23)
hukkai (7)
Zheng-LinXiao (4)
chaos1203 (4)
jts250 (4)
Aierhaimian (3)
nardellimar25 (3)
rlleshi (3)
snake-head (3)
HoBeom (2)
makecent (2)
rjfc (2)
kennymckormick (2)
minhhoang2705 (2)

Top Labels

Issue Labels

1.x (3) enhancement (2) onnx (1) help wanted (1) good first issue (1) question (1) upstream (1) call for webcam demo (1) awaiting response (1)

Pull Request Labels

docker (1) stale (1) help wanted (1) Blocked (1)

Packages

Total packages: 4
Total downloads:
- pypi 4,851 last-month

Total dependent packages: 1
(may contain duplicates)
Total dependent repositories: 20
(may contain duplicates)
Total versions: 54
Total maintainers: 2

pypi.org: mmaction2

OpenMMLab Video Understanding Toolbox and Benchmark

Homepage: https://github.com/open-mmlab/mmaction2
Documentation: https://mmaction2.readthedocs.io/
License: Apache License 2.0
Latest release: 1.2.0
published over 2 years ago

Versions: 28
Dependent Packages: 1
Dependent Repositories: 19
Downloads: 4,661 Last month

Rankings

Forks count: 1.2%

Stargazers count: 1.3%

Average: 3.1%

Dependent repos count: 3.3%

Dependent packages count: 4.8%

Downloads: 4.8%

Maintainers (1)

openmmlab

Last synced: 6 months ago

proxy.golang.org: github.com/open-mmlab/mmaction2

Documentation: https://pkg.go.dev/github.com/open-mmlab/mmaction2#section-documentation
License: apache-2.0
Latest release: v1.2.0
published over 2 years ago

Versions: 24
Dependent Packages: 0
Dependent Repositories: 0

Rankings

Dependent packages count: 6.5%

Average: 6.7%

Dependent repos count: 7.0%

Last synced: 6 months ago

pypi.org: mmaction

OpenMMLab Action Understanding Toolbox and Benchmark

Homepage: https://github.com/open-mmlab/mmaction2
Documentation: https://mmaction.readthedocs.io/
License: Apache License 2.0
Latest release: 0.5.0
published over 5 years ago

Versions: 1
Dependent Packages: 0
Dependent Repositories: 1
Downloads: 180 Last month

Rankings

Forks count: 1.2%

Stargazers count: 1.3%

Dependent packages count: 7.3%

Average: 9.8%

Downloads: 17.0%

Dependent repos count: 22.1%

Maintainers (1)

openmmlab

Last synced: 6 months ago

pypi.org: otxmmaction2

OpenMMLab Video Understanding Toolbox and Benchmark

Homepage: https://github.com/open-mmlab/mmaction2
Documentation: https://otxmmaction2.readthedocs.io/
License: Apache License 2.0
Latest release: 1.2.0
published about 2 years ago

Versions: 1
Dependent Packages: 0
Dependent Repositories: 0
Downloads: 10 Last month

Rankings

Dependent packages count: 10.1%

Average: 38.3%

Dependent repos count: 66.5%

Maintainers (1)

JGHyun

Last synced: 6 months ago

Dependencies

.github/workflows/lint.yml actions

actions/checkout v2 composite
actions/setup-python v2 composite

.github/workflows/merge_stage_test.yml actions

actions/checkout v3 composite
actions/setup-python v4 composite
codecov/codecov-action v1.0.14 composite

.github/workflows/pr_stage_test.yml actions

actions/checkout v3 composite
actions/setup-python v4 composite
codecov/codecov-action v1.0.14 composite

.github/workflows/publish-to-pypi.yml actions

actions/checkout v2 composite
actions/setup-python v2 composite

.circleci/docker/Dockerfile docker

pytorch/pytorch ${PYTORCH}-cuda${CUDA}-cudnn${CUDNN}-devel build

docker/Dockerfile docker

pytorch/pytorch ${PYTORCH}-cuda${CUDA}-cudnn${CUDNN}-devel build

docker/serve/Dockerfile docker

pytorch/pytorch ${PYTORCH}-cuda${CUDA}-cudnn${CUDNN}-devel build

requirements/build.txt pypi

Pillow *
decord >=0.4.1
einops *
matplotlib *
numpy *
opencv-contrib-python *
scipy *
torch >=1.3

requirements/docs.txt pypi

docutils ==0.18.1
einops *
modelindex *
myst-parser *
opencv-python *
scipy *
sphinx ==6.1.3
sphinx-notfound-page *
sphinx-tabs *
sphinx_copybutton *
sphinx_markdown_tables *
sphinxcontrib-jquery *
tabulate *

requirements/mminstall.txt pypi

mmcv >=2.0.0rc4,<2.2.0
mmengine >=0.7.1,<1.0.0

requirements/multimodal.txt pypi

transformers >=4.28.0

requirements/optional.txt pypi

PyTurboJPEG *
av >=9.0
future *
imgaug *
librosa *
lmdb *
moviepy *
openai-clip *
packaging *
pims *
soundfile *
tensorboard *
wandb *

requirements/readthedocs.txt pypi

mmcv *
titlecase *
torch *
torchvision *

requirements/tests.txt pypi

coverage * test
flake8 * test
interrogate * test
isort ==4.3.21 test
parameterized * test
pytest * test
pytest-runner * test
xdoctest >=0.10.0 test
yapf * test

requirements.txt pypi

setup.py pypi

tools/data/activitynet/environment.yml conda

ca-certificates 2020.1.1.*
certifi 2020.4.5.1.*
ffmpeg 2.8.6.*
libcxx 10.0.0.*
libedit 3.1.20181209.*
libffi 3.3.*
ncurses 6.2.*
openssl 1.1.1g.*
pip 20.0.2.*
python 3.7.7.*
readline 8.0.*
setuptools 46.4.0.*
sqlite 3.31.1.*
tk 8.6.8.*
wheel 0.34.2.*
xz 5.2.5.*
zlib 1.2.11.*

tools/data/gym/environment.yml conda

ca-certificates 2020.1.1.*
certifi 2020.4.5.1.*
ffmpeg 2.8.6.*
libcxx 10.0.0.*
libedit 3.1.20181209.*
libffi 3.3.*
ncurses 6.2.*
openssl 1.1.1g.*
pip 20.0.2.*
python 3.7.7.*
readline 8.0.*
setuptools 46.4.0.*
sqlite 3.31.1.*
tk 8.6.8.*
wheel 0.34.2.*
xz 5.2.5.*
zlib 1.2.11.*

tools/data/hvu/environment.yml conda

ca-certificates 2020.1.1.*
certifi 2020.4.5.1.*
ffmpeg 2.8.6.*
libcxx 10.0.0.*
libedit 3.1.20181209.*
libffi 3.3.*
ncurses 6.2.*
openssl 1.1.1g.*
pip 20.0.2.*
python 3.7.7.*
readline 8.0.*
setuptools 46.4.0.*
sqlite 3.31.1.*
tk 8.6.8.*
wheel 0.34.2.*
xz 5.2.5.*
zlib 1.2.11.*

tools/data/kinetics/environment.yml conda

ca-certificates 2020.1.1.*
certifi 2020.4.5.1.*
ffmpeg 2.8.6.*
libcxx 10.0.0.*
libedit 3.1.20181209.*
libffi 3.3.*
ncurses 6.2.*
openssl 1.1.1g.*
pip 20.0.2.*
python 3.7.7.*
readline 8.0.*
setuptools 46.4.0.*
sqlite 3.31.1.*
tk 8.6.8.*
wheel 0.34.2.*
xz 5.2.5.*
zlib 1.2.11.*