mmaction2 - MMAction2 V1.2.0 Release

Highlights

Support the Training of ActionClip
Support VindLU multi-modality algorithm
Support MobileOne TSN/TSM

New Features

Support the Training of ActionClip (2620)
Support video retrieval dataset MSVD (2622)
Support VindLU multi-modality algorithm (2667)
Support Dense Regression Network for Video Grounding (2668)

Improvements

Support Video Demos (2602)
Support Audio Demos (2603)
Add README_zh-CN.md for Swin and VideoMAE (2621)
Support MobileOne TSN/TSM (2656)
Support SlowOnly K700 feature to train localization models (2673)

Bug Fixes

Refine ActionDataSample structure (2658)
Fix MPS device (2619)

- Python
Published by ly015 over 2 years ago

mmaction2 - MMAction2 V1.1.0 Release

New Direction: Multi-Modal Video Understanding

We support two novel models for video recognition and retrieval based on open-domain text: ActionCLIP and CLIP4Clip. These models mark the first step of MMAction2's journey towards multi-modal video understanding. Furthermore, we also introduce a new video retrieval dataset, MSR-VTT.

img_v2_e882ffb4-84c9-4b3a-9ab6-38c251e7d95g

For more details, please refer to ActionCLIP, CLIP4Clip and MSR-VTT.

Supported by @Dai-Wenxun in #2470 and #2489.

New Config Type

MMEngine introduced the pure Python style configuration file:

Support navigating to base configuration file in IDE
Support navigating to base variable in IDE
Support navigating to source code of class in IDE
Support inheriting two configuration files containing the same field
Load the configuration file without other third-party requirements

Refer to the tutorial for more detailed usages.

img_v2_e882ffb4-84c9-4b3a-9ab6-38c251e7d95g

New Datasets

We are glad to support 3 new datasets:

(ICCV2019) HACS
(ICCV2021) MultiSports
(Arxiv2022) Kinetics-710

(ICCV2019) HACS

HACS is a new large-scale dataset for recognition and temporal localization of human actions collected from Web videos.

https://github.com/open-mmlab/mmaction2/assets/58767402/7b7407e3-994a-4523-975c-5bdee3b54998

For more details, please refer to HACS.

Supported by @hukkai in #2224

(ICCV2021) MultiSports

MultiSports is a multi-person video dataset of spatio-temporally localized sports actions.

https://github.com/open-mmlab/mmaction2/assets/58767402/1f94668a-823b-46a0-9ea7-eedf0f29d1d1

For more details, please refer to MultiSports.

Supported by @cir7 in #2280

(Arxiv2022) Kinetics-710

For more details, please refer to Kinetics710.

Supported by @cir7 in #2534

Other New Features

Support rich projects: Gesture Recognition, Spatio-Temporal Action Detection Tutorial, and Knowledge Distillation
Support TCANet(CVPR'2021)
Support VideoMAE V2(CVPR'2023) and VideoMAE(NeurIPS'2022) on action detection

What's Changed

[Doc] Fix document links in readme by @cir7 in https://github.com/open-mmlab/mmaction2/pull/2358
[doc] fix installation doc by @cir7 in https://github.com/open-mmlab/mmaction2/pull/2362
[Enhance] Support automatically assigning issues by @cir7 in https://github.com/open-mmlab/mmaction2/pull/2368
[Doc] Fix model links in README by @cir7 in https://github.com/open-mmlab/mmaction2/pull/2372
[Fix] Restore the wrongly modified config by @cir7 in https://github.com/open-mmlab/mmaction2/pull/2375
[Doc] Fix readme links by @cir7 in https://github.com/open-mmlab/mmaction2/pull/2376
[Fix] update skeleton demo by @WILLOSCAR in https://github.com/open-mmlab/mmaction2/pull/2381
[Fix] Fix a bug in demo_skeleton.py by @Dai-Wenxun in https://github.com/open-mmlab/mmaction2/pull/2380
[Update] Update version requirements by @Dai-Wenxun in https://github.com/open-mmlab/mmaction2/pull/2383
[Doc] update readme by @cir7 in https://github.com/open-mmlab/mmaction2/pull/2382
[Doc] Update Installation Related Doc by @Dai-Wenxun in https://github.com/open-mmlab/mmaction2/pull/2379
[Fix] Fix colab tutorial by @cir7 in https://github.com/open-mmlab/mmaction2/pull/2384
[Fix] update colab link in tutorial by @cir7 in https://github.com/open-mmlab/mmaction2/pull/2391
[Doc] Refine Docs by @Dai-Wenxun in https://github.com/open-mmlab/mmaction2/pull/2404
[CI] fix github ci (main) by @cir7 in https://github.com/open-mmlab/mmaction2/pull/2421
[Fix] fix a bug in multi-label classification by @Dai-Wenxun in https://github.com/open-mmlab/mmaction2/pull/2425
[Fix] Fix issue template by @cir7 in https://github.com/open-mmlab/mmaction2/pull/2399
[Doc] Update repo list by @cir7 in https://github.com/open-mmlab/mmaction2/pull/2429
[Fix] Fix a warning caused by torch.div by @Dai-Wenxun in https://github.com/open-mmlab/mmaction2/pull/2449
[Fix] Fix readthedoc error raised by incompatible OpenSSL version by @cir7 in https://github.com/open-mmlab/mmaction2/pull/2455
[Fix] Fix incompatibility of ImgAug and latest Numpy by @cir7 in https://github.com/open-mmlab/mmaction2/pull/2451
[Fix] Update branch in dockerfile by @cir7 in https://github.com/open-mmlab/mmaction2/pull/2397
[Doc] Update outdated config in readme by @cir7 in https://github.com/open-mmlab/mmaction2/pull/2419
[Fix] Fix tutorial by @cir7 in https://github.com/open-mmlab/mmaction2/pull/2475
[fix] Fix batch blending bug when use multi-label classification by @cir7 in https://github.com/open-mmlab/mmaction2/pull/2466
[Fix] Fix UniFormer README and metafile by @cir7 in https://github.com/open-mmlab/mmaction2/pull/2450
[Doc] update faq by @cir7 in https://github.com/open-mmlab/mmaction2/pull/2476
[Fix] Fix a bug of MViT when set withclstoken to False by @KeepLost in https://github.com/open-mmlab/mmaction2/pull/2480
[Fix] Update outdated dependencies of mmcv for downloading fine-gym dataset by @yhZhai in https://github.com/open-mmlab/mmaction2/pull/2495
[Doc] add finetune doc by @cir7 in https://github.com/open-mmlab/mmaction2/pull/2453
[Doc] Update faq doc by @cir7 in https://github.com/open-mmlab/mmaction2/pull/2482
[Doc] Fix document link by @cir7 in https://github.com/open-mmlab/mmaction2/pull/2457
Merge dev-1.x to main by @cir7 in https://github.com/open-mmlab/mmaction2/pull/2551

New Contributors

@WILLOSCAR made their first contribution in https://github.com/open-mmlab/mmaction2/pull/2381
@KeepLost made their first contribution in https://github.com/open-mmlab/mmaction2/pull/2480
@yhZhai made their first contribution in https://github.com/open-mmlab/mmaction2/pull/2495

Full Changelog: https://github.com/open-mmlab/mmaction2/compare/v1.0.0...v1.1.0

- Python
Published by cir7 almost 3 years ago

mmaction2 - MMAction2 V1.0.0 Release

Highlights

We are excited to announce the release of MMAction2 1.0.0 as a part of the OpenMMLab 2.0 project! MMAction2 1.0.0 introduces an updated framework structure for the core package and a new section called Projects. This section showcases various engaging and versatile applications built upon the MMAction2 foundation.

In this latest release, we have significantly refactored the core package's code to make it clearer, more comprehensible, and disentangled. This has resulted in improved performance for several existing algorithms, ensuring that they now outperform their previous versions. Additionally, we have incorporated some cutting-edge algorithms, such as VideoSwin and VideoMAE, to further enhance the capabilities of MMAction2 and provide users with a more comprehensive and powerful toolkit. The new Projects section serves as an essential addition to MMAction2, created to foster innovation and collaboration among users. This section offers the following attractive features:

Flexible code contribution: Unlike the core package, the Projects section allows for a more flexible environment for code contributions, enabling faster integration of state-of-the-art models and features.
Showcase of diverse applications: Explore various projects built upon the MMAction2 foundation, such as deployment examples and combinations of video recognition with other tasks.
Fostering creativity and collaboration: Encourages users to experiment, build upon the MMAction2 platform, and share their innovative applications and techniques, creating an active community of developers and researchers. Discover the possibilities within the "Projects" section and join the vibrant MMAction2 community in pushing the boundaries of video understanding applications!

Exciting Features

RGBPoseConv3D

RGBPoseConv3D is a framework that jointly uses 2D human skeletons and RGB appearance for human action recognition. It is a 3D CNN with two streams, with the architecture borrowed from SlowFast. In RGBPoseConv3D:

The RGB stream corresponds to the slow stream in SlowFast; The Skeleton stream corresponds to the fast stream in SlowFast.
The input resolution of RGB frames is 4x larger than the pseudo heatmaps.
Bilateral connections are used for early feature fusion between the two modalities.

Supported by @Dai-Wenxun in https://github.com/open-mmlab/mmaction2/pull/2182

Inferencer

In this release, we introduce the MMAction2Inferencer, which is a versatile API for the inference that supports multiple input types. The API enables users to easily specify and customize action recognition models, streamlining the process of performing video prediction using MMAction2.

Usage:

shell python demo/demo_inferencer.py ${INPUTS} [OPTIONS]

The INPUTS can be a video path or rawframes folder. For more detailed information on OPTIONS, please refer to Inferencer.

Example:

shell python demo/demo_inferencer.py zelda.mp4 --rec tsn --vid-out-dir zelda_out --label-file tools/data/kinetics/label_map_k400.txt

You can find the zelda.mp4 here. The output video is displayed below:

https://user-images.githubusercontent.com/58767402/232312742-f5eb2e8c-f015-459c-8a4d-99c331a65735.mp4

Supported by @cir7 in https://github.com/open-mmlab/mmaction2/pull/2164

List of Novel Features

MMAction2 V1.0 introduces support for new models and datasets in the field of video understanding, including MSG3D [Project] (CVPR'2020), CTRGCN [Project] (CVPR'2021), STGCN++ (Arxiv'2022), Video Swin Transformer (CVPR'2022), VideoMAE (NeurIPS'2022), C2D (CVPR'2018), MViT V2 (CVPR'2022), UniFormer V1 (ICLR'2022), and UniFormer V2 (Arxiv'2022), as well as the spatiotemporal action detection dataset AVA-Kinetics (Arxiv'2022).

Enhanced Omni-Source: We enhanced the original omni-source technique by dynamically adjusting 3D convolutional network architecture to simultaneously utilize videos and images for training. Taking the SlowOnlyR50 8x8 as an example, the Top-1 accuracy comparison of the three training methods illustrates that our omni-source training effectively employs the additional ImageNet dataset, significantly boosting performance on Kinetics400.

Mulit-Stream Skeleton Pipeline: In light of MMAction2's prior support for only joint and bone modalities, we have extended support to joint motion and bone motion modalities in MMAction2 V1.0. Furthermore, we have conducted training and evaluation for these four modalities using NTU60 2D and 3D keypoint data on STGCN, 2s-AGCN, and STGCN++.

Repeat Augment was initially proposed as a data augmentation method for ImageNet training and has been employed in recent Video Transformer works. Whenever a video is read during training, we use multiple (typically 2-4) random samples from the video for training. This approach not only enhances the model's generalization capability but also reduces the IO pressure of video reading. We support Repeat Augment in MMAction2 V1.0 and utilize this technique in MViT V2 training. The table below compares the Top-1 accuracy on Kinetics400 before and after employing Repeat Augment:

Bug Fixes

[Fix] Fix flip config of TSM for sth2sth v1/v2 dataset by @cir7 in https://github.com/open-mmlab/mmaction2/pull/2247
[Fix] Fix circle ci by @cir7 in https://github.com/open-mmlab/mmaction2/pull/2336 and https://github.com/open-mmlab/mmaction2/pull/2334
[Fix] Fix accepting an unexpected argument local-rank in PyTorch 2.0 by @cir7 in https://github.com/open-mmlab/mmaction2/pull/2320
[Fix] Fix TSM config link by @zyx-cv in https://github.com/open-mmlab/mmaction2/pull/2315
[Fix] Fix numpy version requirement in CI by @hukkai in https://github.com/open-mmlab/mmaction2/pull/2284
[Fix] Fix NTU pose extraction script by @cir7 in https://github.com/open-mmlab/mmaction2/pull/2246
[Fix] Fix TSM-MobileNet V2 by @cir7 in https://github.com/open-mmlab/mmaction2/pull/2332
[Fix] Fix command bugs in localization tasks' README by @hukkai in https://github.com/open-mmlab/mmaction2/pull/2244
[Fix] Fix duplicate name in DecordInit and SampleAVAFrame by @cir7 in https://github.com/open-mmlab/mmaction2/pull/2251
[Fix] Fix channel order when showing video by @cir7 in https://github.com/open-mmlab/mmaction2/pull/2308
[Fix] Specify maplocation to cpu when using _loadcheckpoint by @Zheng-LinXiao in https://github.com/open-mmlab/mmaction2/pull/2252

New Contributors

@Andy1621 made their first contribution in https://github.com/open-mmlab/mmaction2/pull/2153
@zoe08 made their first contribution in https://github.com/open-mmlab/mmaction2/pull/2188
@vansin made their first contribution in https://github.com/open-mmlab/mmaction2/pull/2228
@Zheng-LinXiao made their first contribution in https://github.com/open-mmlab/mmaction2/pull/2252

Full Changelog: https://github.com/open-mmlab/mmaction2/compare/v0.24.0...v1.0.0

- Python
Published by cir7 about 3 years ago

mmaction2 - MMAction2 V1.0.0rc3 Release

Highlights

Support Action Recognition model UniFormer V1(ICLR'2022), UniFormer V2(Arxiv'2022).
Support training MViT V2(CVPR'2022), and MaskFeat(CVPR'2022) fine-tuning.

New Features

Support UniFormer V1/V2 (#2153)
Support training MViT, and MaskFeat fine-tuning (#2186)
Support a unified inference interface: Inferencer (#2164)

Improvements

Support load data list from multi-backends (#2176)

Bug Fixes

Upgrade isort to fix CI (#2198)
Fix bug in skeleton demo (#2214)

Documentation

Add Chinese documentation for config.md (#2188)
Add readme for Omnisource (#2205)

- Python
Published by ly015 over 3 years ago

mmaction2 - MMAction2 V1.0.0rc2 Release

Highlights

Support Action Recognition model VideoMAE(NeurIPS'2022), MVit V2(CVPR'2022), C2D and skeleton-based action recognition model STGCN++
Support Omni-Source training on ImageNet and Kinetics datasets
Support exporting spatial-temporal detection models to ONNX

New Features

Support VideoMAE (#1942)
Support MViT V2 (#2007)
Supoort C2D (#2022)
Support AVA-Kinetics dataset (#2080)
Support STGCN++ (#2156)
Support exporting spatial-temporal detection models to ONNX (#2148)
Support Omni-Source training on ImageNet and Kinetics datasets (#2143)

Improvements

Support repeat batch data augmentation (#2170)
Support calculating FLOPs tool powered by fvcore (#1997)
Support Spatial-temporal detection demo (#2019)
Add SyncBufferHook and add randomness config in train.py (#2044)
Refactor gradcam (#2049)
Support init_cfg in Swin and ViTMAE (#2055)
Refactor STGCN and related pipelines (#2087)
Refactor visualization tools (#2092)
Update SampleFrames transform and improve most models' performance (#1942)
Support real-time webcam demo (#2152)
Refactor and enhance 2s-AGCN (#2130)
Support adjusting fps in SampleFrame (#2157)

Bug Fixes

Fix CI upstream library dependency (#2000)
Fix SlowOnly readme typos and results (#2006)
Fix VideoSwin readme (#2010)
Fix tools and mim error (#2028)
Fix Imgaug wrapper (#2024)
Remove useless scripts (#2032)
Fix multi-view inference (#2045)
Update mmcv maximum version to 1.8.0 (#2047)
Fix torchserver dependency (#2053)
Fix gen_ntu_rgbd_raw script (#2076)
Update AVA-Kinetics experiment configs and results (#2099)
Add joint.pkl and bone.pkl used in multi-stream fusion tool (#2106)
Fix lint CI config (#2110)
Update testing accuracy for modified SampleFrames (#2117), (#2121), (#2122), (#2124), (#2125), (#2126), (#2129), (#2128)
Fix timm related bug (#1976)
Fix check_videos.py script (#2134)
Update CI maximum torch version to 1.13.0 (#2118)

Documentation

Add MMYOLO description in README (#2011)
Add v1.x introduction in README (#2023)
Fix link in README (#2035)
Refine some docs (#2038), (#2040), (#2058)
Update TSN/TSM Readme (#2082)
Add chinese document (#2083)
Adjust docment structure (#2088)
Fix Sth-Sth and Jester dataset links (#2103)
Fix doc link (#2131)

- Python
Published by ly015 over 3 years ago

mmaction2 - MMAction2 V1.0.0rc1 Release

Highlights

Support Video Swin Transformer

New Features

Support Video Swin Transformer (#1939)

Improvements

Add colab tutorial for 1.x (#1956)
Support skeleton-based action recognition demo (#1920)

Bug Fixes

Fix link in doc (#1986, #1967, #1951, #1926,#1944, #1944, #1927, #1925)
Fix CI (#1987, #1930, #1923)
Fix pre-commit hook config (#1971)
Fix TIN config (#1912)
Fix UT for BMN and BSN (#1966)
Fix UT for Recognizer2D (#1937)
Fix BSN and BMN configs for localization (#1913)
Modeify ST-GCN configs (#1913)
Fix typo in migration doc (#1931)
Remove Onnx related tools (#1928)
Update TANet readme (#1916, #1890)
Update 2S-AGCN readme (#1915)
Fix TSN configs (#1905)
Fix configs for detection (#1903)
Fix typo in TIN config (#1904)
Fix PoseC3D readme (#1899)
Fix ST-GCN configs (#1891)
Fix audio recognition readme (#1898)
Fix TSM readme (#1887)
Fix SlowOnly readme (#1889)
Fix TRN readme (#1888)
Fix typo in get_started doc (#1895)

- Python
Published by ly015 over 3 years ago

mmaction2 - MMAction2 V1.0.0rc0 Release

We are excited to announce the release of MMAction2 v1.0.0rc0. MMAction2 1.0.0beta is the first version of MMAction2 1.x, a part of the OpenMMLab 2.0 projects. Built upon the new training engine.

Highlights

New engines. MMAction2 1.x is based on MMEngine](https://github.com/open-mmlab/mmengine), which provides a general and powerful runner that allows more flexible customizations and significantly simplifies the entrypoints of high-level interfaces.
Unified interfaces. As a part of the OpenMMLab 2.0 projects, MMAction2 1.x unifies and refactors the interfaces and internal logics of train, testing, datasets, models, evaluation, and visualization. All the OpenMMLab 2.0 projects share the same design in those interfaces and logics to allow the emergence of multi-task/modality algorithms.
More documentation and tutorials. We add a bunch of documentation and tutorials to help users get started more smoothly. Read it here.

Breaking Changes

In this release, we made lots of major refactoring and modifications. Please refer to the migration guide for details and migration instructions.

- Python
Published by ly015 almost 4 years ago

mmaction2 - MMAction2 V0.24.1 Release

This release is meant to fix the compatibility with the latest mmcv v1.6.1

- Python
Published by ly015 almost 4 years ago

mmaction2 - MMAction2 V0.24.0 Release

Highlights

Support different seeds

New Features

Add lateral norm in multigrid config (#1567)
Add openpose 25 joints in graph config (#1578)
Support MLU Backend (#1608)

Bug and Typo Fixes

Fix local_rank (#1558)
Fix install typo (#1571)
Fix the inference API doc (#1580)
Fix zh-CN demo.md and getting_started.md (#1587)
Remove Recommonmark (#1595)
Fix inference with ndarray (#1603)
Fix the log error when IterBasedRunner is used (#1606)

- Python
Published by kennymckormick about 4 years ago

mmaction2 - MMAction2 V0.23.0 Release

Highlights

Support different seeds
Provide multi-node training & testing script
Update error log

New Features

Support different seeds(#1502)
Provide multi-node training & testing script(#1521)
Update error log(#1546)

Documentations

Update gpus in Slowfast readme(#1497)
Fix work_dir in multigrid config(#1498)
Add sub bn docs(#1503)
Add shortcycle sampler docs(#1513)
Update Windows Declaration(#1520)
Update the link for ST-GCN(#1544)
Update install commands(#1549)

Bug and Typo Fixes

Update colab tutorial install cmds(#1522)
Fix numitersperepoch in analyzelogs.py(#1530)
Fix distributed_sampler(#1532)
Fix cd dir error(#1545)
Update arg names(#1548)

- Python
Published by kennymckormick about 4 years ago

mmaction2 - MMAction2 V0.22.0 Release

0.22.0 (03/05/2022)

Highlights

Support Multigrid training strategy
Support CPU training
Support audio demo
Support topk customizing in models/heads/base.py

New Features

Support Multigrid training strategy(#1378)
Support STGCN in demo_skeleton.py(#1391)
Support CPU training(#1407)
Support audio demo(#1425)
Support topk customizing in models/heads/base.py(#1452)

Documentations

Add OpenMMLab platform(#1393)
Update links(#1394)
Update readme in configs(#1404)
Update instructions to install mmcv-full(#1426)
Add shortcut(#1433)
Update modelzoo(#1439)
add video_structuralize in readme(#1455)
Update OpenMMLab repo information(#1482)

Bug and Typo Fixes

Update train.py(#1375)
Fix printout bug(#1382)
Update multi processing setting(#1395)
Setup multi processing both in train and test(#1405)
Fix bug in nondistributed multi-gpu training(#1406)
Add variable fps in ava_dataset.py(#1409)
Only support distributed training(#1414)
Set test_mode for AVA configs(#1432)
Support single label(#1434)
Add check copyright(#1447)
Support Windows CI(#1448)
Fix wrong device of classweight in models/losses/crossentropy_loss.py(#1457)
Fix bug caused by distributed(#1459)
Update readme(#1460)
Fix lint caused by colab automatic upload(#1461)
Refine CI(#1471)
Update pre-commit(#1474)
Add deprecation message for deploy tool(#1483)

ModelZoo

Support slowfast_steplr(#1421)

- Python
Published by kennymckormick over 4 years ago

mmaction2 - MMAction2 V0.21.0 Release

Highlights

Support 2s-AGCN
Support publish models in Windows
Improve some sthv1 related models
Support BABEL

New Features

Support 2s-AGCN(#1248)
Support skip postproc in ntuposeextraction(#1295)
Support publish models in Windows(#1325)
Add copyright checkhook in pre-commit-config(#1344)

Documentations

Add MMFlow (#1273)
Revise README.md and add projects.md (#1286)
Add 2s-AGCN in Updates(#1289)
Add MMFewShot(#1300)
Add MMHuman3d(#1304)
Update pre-commit(#1313)
Use share menu from the theme instead(#1328)
Update installation command(#1340)

Bug and Typo Fixes

Update the inference part in notebooks(#1256)
Update the map_location(#1262)
Fix bug that start_index is not used in RawFrameDecode(#1278)
Fix bug in initrandomseed(#1282)
Fix bug in setup.py(#1303)
Fix interrogate error in workflows(#1305)
Fix typo in slowfast config(#1309)
Cancel previous runs that are not completed(#1327)
Fix missing skip_postproc parameter(#1347)
Update ssn.py(#1355)
Use latest youtube-dl(#1357)
Fix test-best(#1362)

ModelZoo

Improve some sthv1 related models(#1306)
Support BABEL(#1332)

- Python
Published by kennymckormick over 4 years ago

mmaction2 - MMAction2 V0.20.0 Release

Highlights

Support TorchServe
Add video structuralize demo
Support using 3D skeletons for skeleton-based action recognition
Benchmark PoseC3D on UCF and HMDB

New Features

Support TorchServe (#1212)
Support 3D skeletons pre-processing (#1218)
Support video structuralize demo (#1197)

Documentations

Revise README.md and add projects.md (#1214)
Add CN docs for Skeleton dataset, PoseC3D and ST-GCN (#1228, #1237, #1236)
Add tutorial for custom dataset training for skeleton-based action recognition (#1234)

Bug and Typo Fixes

Fix tutorial link (#1219)
Fix GYM links (#1224)

ModelZoo

Benchmark PoseC3D on UCF and HMDB (#1223)
Add ST-GCN + 3D skeleton model for NTU60-XSub (#1236)

New Contributors

@bit-scientist made their first contribution in https://github.com/open-mmlab/mmaction2/pull/1234

Full Changelog: https://github.com/open-mmlab/mmaction2/compare/v0.19.0...v0.20.0

- Python
Published by kennymckormick over 4 years ago

mmaction2 - MMAction2 V0.19.0 Release

Highlights

Support ST-GCN
Refactor the inference API
Add code spell check hook

New Features

Support ST-GCN (#1123)

Improvement

Add label maps for every dataset (#1127)
Remove useless code MultiGroupCrop (#1180)
Refactor Inference API (#1191)
Add code spell check hook (#1208)
Use docker in CI (#1159)

Documentations

Update metafiles to new OpenMMLAB protocols (#1134)
Switch to new doc style (#1160)
Improve the ERROR message (#1203)
Fix invalid URL in getting_started (#1169)

Bug and Typo Fixes

Compatible with new MMClassification (#1139)
Add missing runtime dependencies (#1144)
Fix THUMOS tag proposals path (#1156)
Fix LoadHVULabel (#1194)
Switch the default value of persistent_workers to False (#1202)
Fix _freeze_stages for MobileNetV2 (#1193)
Fix resume when building rawframes (#1150)
Fix device bug for class weight (#1188)
Correct Arg names in extract_audio.py (#1148)

ModelZoo

Add TSM-MobileNetV2 ported from TSM (#1163)
Add ST-GCN for NTURGB+D-XSub-60 (#1123)

- Python
Published by kennymckormick over 4 years ago

mmaction2 - MMAction2 V0.18.0 Release

Improvement

Add CopyRight (#1099)
Support NTU Pose Extraction (#1076)
Support Caching in RawFrameDecode (#1078)
Add citations & Support python3.9 CI & Use fixed-version sphinx (#1125)

Documentations

Add Descriptions of PoseC3D dataset (#1053)

Bug and Typo Fixes

Fix SSV2 checkpoints (#1101)
Fix CSN normalization (#1116)
Fix typo (#1121)
Fix newcropquadruple bug (#1108)

- Python
Published by kennymckormick almost 5 years ago

mmaction2 - MMAction2 V0.17.0 Release

Highlights

Support PyTorch 1.9
Support Pytorchvideo Transforms
Support PreciseBN

New Features

Support Pytorchvideo Transforms (#1008)
Support PreciseBN (#1038)

Improvements

Remove redundant augmentations in config files (#996)
Make resource directory to hold common resource pictures (#1011)
Remove deperecated FrameSelector (#1010)
Support Concat Dataset (#1000)
Add to-mp4 option to resize_videos.py (#1021)
Add option to keep tail frames (#1050)
Update MIM support (#1061)
Calculate Top-K accurate and inaccurate classes (#1047)

Bug and Typo Fixes

Fix bug in PoseC3D demo (#1009)
Fix some problems in resize_videos.py (#1012)
Support torch1.9 (#1015)
Remove redundant code in CI (#1046)
Fix bug about persistent_workers (#1044)
Support TimeSformer feature extraction (#1035)
Fix ColorJitter (#1025)

ModelZoo

Add TSM-R50 sthv1 models trained by PytorchVideo RandAugment and AugMix (#1008)
Update SlowOnly SthV1 checkpoints (#1034)
Add SlowOnly Kinetics400 checkpoints trained with Precise-BN (#1038)
Add CSN-R50 from scratch checkpoints (#1045)
TPN Kinetics-400 Checkpoints trained with the new ColorJitter (#1025)

Documentation

Add Chinese translation of feature_extraction.md (#1020)
Fix the code snippet in getting_started.md (#1023)
Fix TANet config table (#1028)
Add description to PoseC3D dataset (#1053)

- Python
Published by kennymckormick almost 5 years ago

mmaction2 - MMAction2 V0.16.0 Release

Highlights

Support using backbone from pytorch-image-models(timm)
Support PIMS Decoder
Demo for skeleton-based action recognition
Support Timesformer

New Features

Support using backbones from pytorch-image-models(timm) for TSN (#880)
Support torchvision transformations in preprocessing pipelines (#972)
Demo for skeleton-based action recognition (#972)
Support Timesformer (#839)

Improvements

Add a tool to find invalid videos (#907, #950)
Add an option to specify spectrogram_type (#909)
Add json output to video demo (#906)
Add MIM related docs (#918)
Rename lr to scheduler (#916)
Support --cfg-options for demos (#911)
Support number counting for flow-wise filename template (#922)
Add Chinese tutorial (#941)
Change ResNet3D default values (#939)
Adjust script structure (#935)
Add font color to args in longvideodemo (#947)
Polish code style with Pylint (#908)
Support PIMS Decoder (#946)
Improve Metafiles (#956, #979, #966)
Add links to download Kinetics400 validation (#920)
Audit the usage of shutil.rmtree (#943)
Polish localizer related codes(#913)

Bug and Typo Fixes

Fix spatiotemporal detection demo (#899)
Fix docstring for 3D inflate (#925)
Fix bug of writing text to video with TextClip (#952)
Fix mmcv install in CI (#977)

ModelZoo

Add TSN with Swin Transformer backbone as an example for using pytorch-image-models(timm) backbones (#880)
Port CSN checkpoints from VMZ (#945)
Release various checkpoints for UCF101, HMDB51 and Sthv1 (#938)
Support Timesformer (#839)
Update TSM modelzoo (#981)

- Python
Published by kennymckormick almost 5 years ago

mmaction2 - MMAction2 V0.15.0 Release

Highlights

Support PoseC3D
Support ACRN
Support MIM

New Features

Support PoseC3D (#786, #890)
Support MIM (#870)
Support ACRN and Focal Loss (#891)
Support Jester dataset (#864)

Improvements

Add metric_options for evaluation to docs (#873)
Support creating a new label map based on custom classes for demos about spatio temporal demo (#879)
Improve document about AVA dataset preparation (#878)
Provide a script to extract clip-level feature (#856)

Bug and Typo Fixes

Fix issues about resume (#877, #878)
Correct the key name of eval_results dictionary for metric 'mmitmeanaverage_precision' (#885)

ModelZoo

Support Jester dataset (#864)
Support ACRN and Focal Loss (#891)

- Python
Published by dreamerlin about 5 years ago

mmaction2 - MMAction2 V0.14.0 Release

Highlights

Support TRN
Support Diving48

New Features

Support TRN (#755)
Support Diving48 (#835)
Support Webcam Demo for Spatio-temporal Action Detection Models (#795)

Improvements

Add softmax option for pytorch2onnx tool (#781)
Support TRN (#755)
Test with onnx models and TensorRT engines (#758)
Speed up AVA Testing (#784)
Add self.with_neck attribute (#796)
Update installation document (#798)
Use a random master port (#809)
Update AVA processing data document (#801)
Refactor spatio-temporal augmentation (#782)
Add QR code in CN README (#812)
Add Alternative way to download Kinetics (#817, #822)
Refactor Sampler (#790)
Use EvalHook in MMCV with backward compatibility (#793)
Use MMCV Model Registry (#843)

Bug and Typo Fixes

Fix a bug in pytorch2onnx.py when num_classes <= 4 (#800, #824)
Fix demo_spatiotemporal_det.py error (#803, #805)
Fix loading config bugs when resume (#820)
Make HMDB51 annotation generation more robust (#811)

ModelZoo

Update checkpoint for 256 height in something-V2 (#789)
Support Diving48 (#835)

- Python
Published by dreamerlin about 5 years ago

mmaction2 - MMAction2 V0.13.0 Release

Highlights

Support LFB
Support using backbone from MMCls/TorchVision
Add Chinese documentation

New Features

Support LFB (#553)
Support using backbones from MMCls for TSN (#679)
Support using backbones from TorchVision for TSN (#720)
Support Mixup and Cutmix for recognizers (#681)
Support Chinese documentation (#665, #680, #689, #701, #702, #703, #706, #716, #717, #731, #733, #735, #736, #737, #738, #739, #740, #742, #752, #759, #761, #772, #775)

Improvements

Add slowfast config/json/log/ckpt for training custom classes of AVA (#678)
Set RandAugment as Imgaug default transforms (#585)
Add --test-last & --test-best for tools/train.py to test checkpoints after training (#608)
Add fcn_testing in TPN (#684)
Remove redundant recall functions (#741)
Recursively remove pretrained step for testing (#695)
Improve demo by limiting inference fps (#668)

Bug and Typo Fixes

Fix a bug about multi-class in VideoDataset (#723)
Reverse key-value in anet filelist generation (#686)
Fix flow norm cfg typo (#693)

ModelZoo

Add LFB for AVA2.1 (#553)
Add TSN with ResNeXt-101-32x4d backbone as an example for using MMCls backbones (#679)
Add TSN with Densenet161 backbone as an example for using TorchVision backbones (#720)
Add slowonlynlembeddedgaussianr504x16x1150ekinetics400rgb (#690)
Add slowonlynlembeddedgaussianr508x8x1150ekinetics400rgb (#704)
Add slowonlynlkineticspretrainedr504x16x1(8x8x1)20eavargb (#730)

- Python
Published by dreamerlin about 5 years ago

mmaction2 - MMAction2 V0.12.0 Release

Highlights

Support TSM-MobileNetV2
Support TANet
Support GPU Normalize

New Features

Support TSM-MobileNetV2 (#415)
Support flip with label mapping (#591)
Add seed option for sampler (#642)
Support GPU Normalize (#586)
Support TANet (#595)

Improvements

Training custom classes of ava dataset (#555)
Add CN README in homepage (#592, #594)
Support soft label for CrossEntropyLoss (#625)
Refactor config: Specify train_cfg and test_cfg in model (#629)
Provide an alternative way to download older kinetics annotations (#597)
Update FAQ for
- 1). data pipeline about video and frames (#598)
- 2). how to show results (#598)
- 3). batch size setting for batchnorm (#657)
- 4). how to fix stages of backbone when finetuning models (#658)
Modify default value of save_best (#600)
Use BibTex rather than latex in markdown (#607)
Add warnings of uninstalling mmdet and supplementary documents (#624)
Support soft label for CrossEntropyLoss (#625)

Bug and Typo Fixes

Fix value of pem_low_temporal_iou_threshold in BSN (#556)
Fix ActivityNet download script (#601)

ModelZoo

Add TSM-MobileNetV2 for Kinetics400 (#415)
Add deeper SlowFast models (#605)

- Python
Published by innerlee over 5 years ago

mmaction2 - MMAction2 V0.11.0 Release

Highlights

Support imgaug
Support spatial temporal demo
Refactor EvalHook, config structure, unittest structure

New Features

Support imgaug for augmentations in the data pipeline (#492)
Support setting max_testing_views for extremely large models to save GPU memory used (#511)
Add spatial temporal demo (#547, #566)

Improvements

Refactor EvalHook (#395)
Refactor AVA hook (#567)
Add repo citation (#545)
Add dataset size of Kinetics400 (#503)
Add lazy operation docs (#504)
Add class_weight for CrossEntropyLoss and BCELossWithLogits (#509)
add some explanation about the resampling in slowfast (#502)
Modify paper title in README.md (#512)
Add alternative ways to download Kinetics (#521)
Add OpenMMLab projects link in README (#530)
Change default preprocessing to shortedge to 256 (#538)
Add config tag in dataset README (#540)
Add solution for markdownlint installation issue (#497)
Add dataset overview in readthedocs (#548)
Modify the trigger mode of the warnings of missing mmdet (583)
Refactor config structure (#488, #572)
Refactor unittest structure (#433)

Bug and Typo Fixes

Fix a bug about ava dataset validation (#527)
Fix a bug about ResNet pretrain weight initialization (#582)
Fix a bug in CI due to MMCV index (#495)
Remove invalid links of MiT and MMiT (#516)
Fix frame rate bug for AVA preparation (#576)

- Python
Published by innerlee over 5 years ago

mmaction2 - MMAction2 V0.10.0 Release

Highlights

Support Spatio-Temporal Action Detection (AVA)
Support precise BN

New Features

Support precise BN (#501)
Support Spatio-Temporal Action Detection (AVA) (#351)
Support to return feature maps in inference_recognizer (#458)

Improvements

Add arg stride to longvideodemo.py, to make inference faster (#468)
Support training and testing for Spatio-Temporal Action Detection (#351)
Fix CI due to pip upgrade (#454)
Add markdown lint in pre-commit hook (#255)
Speed up confusion matrix calculation (#465)
Use title case in modelzoo statistics (#456)
Add FAQ documents for easy troubleshooting. (#413, #420, #439)
Support Spatio-Temporal Action Detection with context (#471)
Add class weight for CrossEntropyLoss and BCELossWithLogits (#509)
Add Lazy OPs docs (#504)

Bug and Typo Fixes

Fix typo in default argument of BaseHead (#446)
Fix potential bug about output_config overwrite (#463)

ModelZoo

Add SlowOnly, SlowFast for AVA2.1 (#351)

- Python
Published by kennymckormick over 5 years ago

mmaction2 - MMAction2 V0.9.0 Release

Highlights - Support GradCAM utils for recognizers - Support ResNet Audio model

New Features - Automatically add modelzoo statistics to readthedocs (#327) - Support GYM99 data preparation (#331) - Add AudioOnly Pathway from AVSlowFast. (#355) - Add GradCAM utils for recognizer (#324) - Add print config script (#345) - Add online motion vector decoder (#291)

Improvements - Support PyTorch 1.7 in CI (#312) - Support to predict different labels in a long video (#274) - Update docs bout test crops (#359) - Polish code format using pylint manually (#338) - Update unittest coverage (#358, #322, #325) - Add random seed for building filelists (#323) - Update colab tutorial (#367) - set default batch_size of evaluation and testing to 1 (#250) - Rename the preparation docs to README.md (#388) - Move docs about demo to demo/README.md (#329) - Remove redundant code in tools/test.py (#310) - Automatically calculate number of test clips for Recognizer2D (#359)

Bug and Typo Fixes - Fix rename Kinetics classnames bug (#384) - Fix a bug in BaseDataset when data_prefix is None (#314) - Fix a bug about tmp_folder in OpenCVInit (#357) - Fix get_thread_id when not using disk as backend (#354, #357) - Fix the bug of HVU object num_classes from 1679 to 1678 (#307) - Fix typo in export_model.md (#399) - Fix OmniSource training configs (#321) - Fix Issue #306: Bug of SampleAVAFrames (#317)

ModelZoo - Add SlowOnly model for GYM99, both RGB and Flow (#336) - Add auto modelzoo statistics in readthedocs (#327) - Add TSN for HMDB51 pretrained on Kinetics400, Moments in Time and ImageNet. (#372)

- Python
Published by innerlee over 5 years ago

mmaction2 - MMAction2 V0.8.0 Release

v0.8.0 (31/10/2020)

Highlights - Support OmniSource - Support C3D - Support video recognition with audio modality - Support HVU - Support X3D

New Features - Support AVA dataset preparation (#266) - Support the training of video recognition dataset with multiple tag categories (#235) - Support joint training with multiple training datasets of multiple formats, including images, untrimmed videos, etc. (#242) - Support to specify a start epoch to conduct evaluation (#216) - Implement X3D models, support testing with model weights converted from SlowFast (#288)

Improvements - Set default values of 'averageclips' in each config file so that there is no need to set it explicitly during testing in most cases (#232) - Extend HVU datatools to generate individual file list for each tag category (#258) - Support data preparation for Kinetics-600 and Kinetics-700 (#254) - Add cfg-options in arguments to override some settings in the used config for convenience (#212) - Rename the old evaluating protocol `meanaverageprecisionasmmitmeanaverageprecisionsince it is only used on MMIT and is not themAPwe usually talk about. Addmeanaverageprecision, which is the realmAP([#235](https://github.com/open-mmlab/mmaction2/pull/235)) - Add accurate setting (Three crop * 2 clip) and report corresponding performance for TSM model ([#241](https://github.com/open-mmlab/mmaction2/pull/241)) - Add citations in each preparing_dataset.md intools/data/dataset([#289](https://github.com/open-mmlab/mmaction2/pull/289)) - Update the performance of audio-visual fusion on Kinetics-400 ([#281](https://github.com/open-mmlab/mmaction2/pull/281)) - Support data preparation of OmniSource web datasets, including GoogleImage, InsImage, InsVideo and KineticsRawVideo ([#294](https://github.com/open-mmlab/mmaction2/pull/294)) - Usemetric_optionsdict to provide metric args inevaluate` (#286)

Bug Fixes - Register FrameSelector in PIPELINES (#268) - Fix the potential bug for default value in datasetsetting (#245) - Fix the data preparation bug for something-something dataset (#278) - Fix the invalid config url in slowonly README data benchmark (#249) - Validate that the performance of models trained with videos have no significant difference comparing to the performance of models trained with rawframes (#256) - Correct the `imgnorm_cfg` used by TSN-3seg-R50 UCF-101 model, improve the Top-1 accuracy by 3% (#273)

ModelZoo - Add Baselines for Kinetics-600 and Kinetics-700, including TSN-R50-8seg and SlowOnly-R50-8x8 (#259) - Add OmniSource benchmark on MiniKineitcs (#296) - Add Baselines for HVU, including TSN-R18-8seg on 6 tag categories of HVU (#287) - Add X3D models ported from SlowFast (#288)

- Python
Published by innerlee over 5 years ago

mmaction2 - MMAction2 V0.7.0 Release

Highlights

Support TPN
Support JHMDB, UCF101-24, HVU dataset preparation
support onnx model conversion

New Features

Support the data pre-processing pipeline for the HVU Dataset (#277)
Support real-time action recognition from web camera (#171)
Support onnx (#160)
Support UCF101-24 preparation (#219)
Support evaluating mAP for ActivityNet with CUHK17activitynetpred (#176)
Add the data pipeline for ActivityNet, including downloading videos, extracting RGB and Flow frames, finetuning TSN and extracting feature (#190)
Support JHMDB preparation (#220)

ModelZoo

Add finetuning setting for SlowOnly (#173)
Add TSN and SlowOnly models trained with OmniSource, which achieve 75.7% Top-1 with TSN-R50-3seg and 80.4% Top-1 with SlowOnly-R101-8x8 (#215)

Improvements

Support demo with video url (#165)
Support multi-batch when testing (#184)
Add tutorial for adding a new learning rate updater (#181)
Add config name in meta info (#183)
Remove git hash in __version__ (#189)
Check mmcv version (#189)
Update url with 'https://download.openmmlab.com' (#208)
Update Docker file to support PyTorch 1.6 and update install.md (#209)
Polish readsthedocs display (#217, #229)

Bug Fixes

Fix the bug when using OpenCV to extract only RGB frames with original shape (#184)
Fix the bug of sthv2 num_classes from 339 to 174 (#174, #207)

- Python
Published by innerlee over 5 years ago

mmaction2 - MMAction2 V0.6.0 Release

Highlights

Support TIN, CSN, SSN, NonLocal
Support FP16 training

New Features

Support NonLocal module and provide ckpt in TSM and I3D (#41)
Support SSN (#33, #37, #52, #55)
Support CSN (#87)
Support TIN (#53)
Support HMDB51 dataset preparation (#60)
Support encoding videos from frames (#84)
Support FP16 training (#25)
Enhance demo by supporting rawframe inference (#59), output video/gif (#72)

ModelZoo

Update Slowfast modelzoo (#51)
Update TSN, TSM video checkpoints (#50)
Add data benchmark for TSN (#57)
Add data benchmark for SlowOnly (#77)
Add BSN/BMN performance results with feature extracted by our codebase (#99)

Improvements

Polish data preparation codes (#70)
Improve data preparation scripts (#58)
Improve unittest coverage and minor fix (#62)
Support PyTorch 1.6 in CI (#117)
Support with_offset for rawframe dataset (#48)
Support json annotation files (#119)
Support multi-class in TSMHead (#104)
Support using val_step() to validate data for each val workflow (#123)
Use xxInit() method to get total_frames and make total_frames a required key (#90)
Add paper introduction in model readme (#140)
Adjust the directory structure of tools/ and rename some scripts files (#142)

Bug Fixes

Fix configs for localization test (#67)
Fix configs of SlowOnly by fixing lr to 8 gpus (#136)
Fix the bug in analyze_log (#54)
Fix the bug of generating HMDB51 class index file (#69)
Fix the bug of using load_checkpoint() in ResNet (#93)
Fix the bug of --work-dir when using slurm training script (#110)
Correct the sthv1/sthv2 rawframes filelist generate command (#71)
CosineAnnealing typo (#47)

- Python
Published by hellock almost 6 years ago

mmaction2 - MMAction2 V0.5.0 Release

The first release of MMAction2.

- Python
Published by hellock almost 6 years ago

Recent Releases of mmaction2

mmaction2 - MMAction2 V1.2.0 Release

mmaction2 - MMAction2 V1.1.0 Release

New Direction: Multi-Modal Video Understanding

New Config Type

New Datasets

(ICCV2019) HACS

(ICCV2021) MultiSports

(Arxiv2022) Kinetics-710

Other New Features

What's Changed

New Contributors

mmaction2 - MMAction2 V1.0.0 Release

Highlights

Exciting Features

RGBPoseConv3D

Inferencer

List of Novel Features

Bug Fixes

New Contributors

mmaction2 - MMAction2 V1.0.0rc3 Release

mmaction2 - MMAction2 V1.0.0rc2 Release

mmaction2 - MMAction2 V1.0.0rc1 Release

mmaction2 - MMAction2 V1.0.0rc0 Release

mmaction2 - MMAction2 V0.24.1 Release

mmaction2 - MMAction2 V0.24.0 Release

mmaction2 - MMAction2 V0.23.0 Release

mmaction2 - MMAction2 V0.22.0 Release

0.22.0 (03/05/2022)

mmaction2 - MMAction2 V0.21.0 Release

mmaction2 - MMAction2 V0.20.0 Release

New Contributors

mmaction2 - MMAction2 V0.19.0 Release

mmaction2 - MMAction2 V0.18.0 Release

mmaction2 - MMAction2 V0.17.0 Release

mmaction2 - MMAction2 V0.16.0 Release

mmaction2 - MMAction2 V0.15.0 Release

mmaction2 - MMAction2 V0.14.0 Release

mmaction2 - MMAction2 V0.13.0 Release

mmaction2 - MMAction2 V0.12.0 Release

mmaction2 - MMAction2 V0.11.0 Release

mmaction2 - MMAction2 V0.10.0 Release

mmaction2 - MMAction2 V0.9.0 Release

mmaction2 - MMAction2 V0.8.0 Release

v0.8.0 (31/10/2020)

mmaction2 - MMAction2 V0.7.0 Release

Highlights

New Features

ModelZoo

Improvements

Bug Fixes

mmaction2 - MMAction2 V0.6.0 Release

Highlights

New Features

ModelZoo

Improvements

Bug Fixes

mmaction2 - MMAction2 V0.5.0 Release