Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/bytedance/portrait-mode-video

Video dataset dedicated to portrait-mode video recognition.

https://github.com/bytedance/portrait-mode-video

Science Score: 36.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ✓
    Academic publication links
    Links to: arxiv.org
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (13.1%) to scientific vocabulary

Keywords

benchmark portrait-mode portrait-mode-video social-media-video vertical-orientation vertical-orientation-video video-recognition
Last synced: 7 months ago · JSON representation

Repository

Video dataset dedicated to portrait-mode video recognition.

Basic Info
Statistics
  • Stars: 48
  • Watchers: 4
  • Forks: 1
  • Open Issues: 2
  • Releases: 0
Topics
benchmark portrait-mode portrait-mode-video social-media-video vertical-orientation vertical-orientation-video video-recognition
Created over 2 years ago · Last pushed over 1 year ago
Metadata Files
Readme

README.md

Portrait-Mode Video Recognition

We are releasing our code and dataset regarding Portrait-Mode Video Recognition research. The videos are sourced from Douyin platform. We distribute video content through the provision of links. Users are responsible for downloading the videos independently.

Videos

The high-quality videos are filtered by humans, with human activities across wide-spread categories.

🚀🚀 Thanks for the support from the community. Please check the issue here for cached videos on OneDrive.

🚀🚀 Please check the annotation at Uniformer/data_list/PMV/ and text description of the categories at data/class_name_mapping.csv

Taxonomy

Please check our released taxonomy here. There is also an interactive demo of the taxonomy here.

Usage

We assume two directories for this project. {CODE_DIR} for the code respository; {PROJ_DIR} for the model logs, checkpoints and dataset.

To start with, please clone our code from Github bash git clone https://github.com/bytedance/Portrait-Mode-Video.git {CODE_DIR}

Python environment

We train our model with Python 3.7.3 and Pytorch 1.10.0. Please use the following command to install the packages used for our project. First install pytorch following the official instructions. Then install other packages by bash pip3 install -r requirements.txt

Data downloading

Please refer to DATA.md for data downloading. We assume the videos are stored under {PROJ_DIR}/PMV_dataset. Category IDs for the released videos are under {CODE_DIR}/MViT/data_list/PMV and {CODE_DIR}/Uniformer/data_list/PMV.

Training

We provide bash scripts for training models using our PMV-400 data, as in exps/PMV/. A demo running script is bash bash exps/PMV/run_MViT_PMV.sh For each model, e.g., MViT, we provide the scripts for different training recipes in a single bash scripts, e.g., exps/PMV/run_MViT_PMV.sh. Please choose the one suiting your purpose.

Note that you should set some environment variables in the bash scripts, such as WORKER_0_HOST, WORKER_NUM and WORKER_ID in run_SlowFast_MViTv2_S_16x4_PMV_release.sh; PROJ_DIR in run_{model}_PMV.sh.

Inference

We provide inference scripts for obtaining the report results in our paper. We also provide the trained model checkpoints.

License

Our code is licensed under an Apache 2.0 License. Our data is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0) License. The data is released for non-commercial research purposes only.

By engaging in the downloading process, users are considered to have agreed to comply with our distribution license terms and conditions.

We would like to extend our thanks to the teams behind SlowFast code repository, 3Massiv, Kinetics and Uniformer. Our work builds upon their valuable contributions. Please acknowledge these resources in your work.

Owner

  • Name: Bytedance Inc.
  • Login: bytedance
  • Kind: organization
  • Location: Singapore

GitHub Events

Total
  • Issues event: 3
  • Watch event: 12
  • Issue comment event: 3
  • Push event: 1
Last Year
  • Issues event: 3
  • Watch event: 12
  • Issue comment event: 3
  • Push event: 1

Issues and Pull Requests

Last synced: about 1 year ago

All Time
  • Total issues: 6
  • Total pull requests: 0
  • Average time to close issues: about 1 month
  • Average time to close pull requests: N/A
  • Total issue authors: 5
  • Total pull request authors: 0
  • Average comments per issue: 1.5
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 2
  • Pull requests: 0
  • Average time to close issues: about 2 months
  • Average time to close pull requests: N/A
  • Issue authors: 2
  • Pull request authors: 0
  • Average comments per issue: 1.0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • zhimin-z (2)
  • Kerin637 (1)
  • jmgrn61 (1)
  • pear-blossoms (1)
  • saicoco (1)
Pull Request Authors
Top Labels
Issue Labels
Pull Request Labels

Dependencies

MViT/setup.py pypi
  • av *
  • detectron2 *
  • fairscale *
  • matplotlib *
  • opencv-python *
  • pandas *
  • pillow *
  • psutil *
  • pyyaml >=5.1
  • simplejson *
  • sklearn *
  • tensorboard *
  • termcolor >=1.1
  • torchvision >=0.4.2
  • tqdm *
  • yacs >=0.1.6
MViT/slowfast.egg-info/requires.txt pypi
  • av *
  • detectron2 *
  • fairscale *
  • matplotlib *
  • moviepy *
  • opencv-python *
  • pandas *
  • pillow *
  • psutil *
  • pyyaml >=5.1
  • simplejson *
  • sklearn *
  • tensorboard *
  • termcolor >=1.1
  • torchvision >=0.4.2
  • tqdm *
  • yacs >=0.1.6
Uniformer/setup.py pypi
  • av *
  • detectron2 *
  • matplotlib *
  • opencv-python *
  • pandas *
  • pillow *
  • psutil *
  • pyyaml >=5.1
  • simplejson *
  • sklearn *
  • tensorboard *
  • termcolor >=1.1
  • timm *
  • torchvision >=0.4.2
  • tqdm *
  • yacs >=0.1.6
requirements.txt pypi
  • Babel ==2.10.3
  • Cython ==0.29.32
  • Deprecated ==1.2.13
  • GitPython ==3.1.27
  • Jinja2 ==3.1.2
  • Markdown ==3.4.1
  • MarkupSafe ==2.1.1
  • Pillow ==9.0.1
  • PyGObject ==3.30.4
  • PyJWT ==1.7.1
  • PyYAML ==6.0
  • Pygments ==2.13.0
  • QtPy ==2.2.0
  • Send2Trash ==1.8.0
  • Werkzeug ==2.2.2
  • absl-py ==1.2.0
  • addict ==2.4.0
  • aiohttp ==3.8.1
  • aiosignal ==1.2.0
  • antlr4-python3-runtime ==4.9.3
  • anyio ==3.6.1
  • argon2-cffi ==21.3.0
  • argon2-cffi-bindings ==21.2.0
  • asn1crypto ==1.4.0
  • async-timeout ==4.0.2
  • asynctest ==0.13.0
  • attributee ==0.1.5
  • attrs ==21.2.0
  • av ==9.2.0
  • backcall ==0.2.0
  • beautifulsoup4 ==4.11.1
  • bidict ==0.22.1
  • black ==22.3.0
  • bleach ==5.0.1
  • braceexpand ==0.1.7
  • bypy ==1.7.12
  • cachetools ==5.2.0
  • certifi ==2021.10.8
  • cffi ==1.15.0
  • chardet ==5.2.0
  • charset-normalizer ==2.0.12
  • chumpy ==0.70
  • click ==7.1.2
  • cloudpickle ==2.2.1
  • colorama ==0.4.5
  • commonmark ==0.9.1
  • coverage ==6.4.4
  • cryptography ==37.0.4
  • cycler ==0.11.0
  • datasets ==1.15.1
  • debugpy ==1.6.3
  • decorator ==4.4.2
  • decord ==0.6.0
  • defusedxml ==0.7.1
  • dill ==0.3.5.1
  • distlib ==0.3.8
  • distro-info ==0.21
  • dnspython ==2.2.1
  • docker-pycreds ==0.4.0
  • dotty-dict ==1.3.1
  • ecdsa ==0.14.1
  • einops ==0.4.1
  • entrypoints ==0.4
  • fairscale ==0.4.6
  • fastjsonschema ==2.16.1
  • filelock ==3.12.2
  • findspark ==2.0.1
  • fonttools ==4.35.0
  • frozenlist ==1.3.1
  • fsspec ==2021.11.1
  • ftfy ==6.1.1
  • future ==0.18.2
  • gevent ==21.12.0
  • gitdb ==4.0.9
  • google-auth ==2.10.0
  • google-auth-oauthlib ==0.4.6
  • greenlet ==1.1.2
  • grpcio ==1.47.0
  • gunicorn ==20.1.0
  • h11 ==0.14.0
  • horovod ==0.24.2
  • huggingface-hub ==0.9.0
  • hydra-core ==1.2.0
  • idna ==3.3
  • imageio ==2.21.1
  • imageio-ffmpeg ==0.4.7
  • importlib-metadata ==6.7.0
  • importlib-resources ==5.9.0
  • iniconfig ==1.1.1
  • iopath ==0.1.9
  • iotop ==0.6
  • ipaddress ==1.0.23
  • ipykernel ==5.4.3
  • ipython ==7.19.0
  • ipython-genutils ==0.2.0
  • ipywidgets ==7.7.1
  • jedi ==0.18.1
  • joblib ==1.1.0
  • json-tricks ==3.15.5
  • json5 ==0.9.9
  • jsonschema ==4.9.1
  • jupyter ==1.0.0
  • jupyter-client ==6.1.12
  • jupyter-console ==6.4.4
  • jupyter-core ==4.11.1
  • jupyter-kernel-gateway ==2.5.1
  • jupyter-server ==1.18.1
  • jupyterlab ==3.4.5
  • jupyterlab-pygments ==0.2.2
  • jupyterlab-server ==2.15.0
  • jupyterlab-widgets ==1.1.1
  • kiwisolver ==1.4.4
  • lap ==0.4.0
  • line-profiler ==3.5.1
  • lxml ==4.9.1
  • matplotlib ==3.5.3
  • matplotlib-inline ==0.1.3
  • memory-profiler ==0.60.0
  • mistune ==0.8.4
  • mmcls ==0.23.1
  • mmcv-full ==1.3.18
  • mmdet ==2.25.1
  • mmpose ==0.28.1
  • mmtrack ==0.13.0
  • mock ==4.0.3
  • model-index ==0.1.11
  • motmetrics ==1.2.5
  • moviepy ==1.0.3
  • multidict ==6.0.2
  • multiprocess ==0.70.13
  • munkres ==1.1.4
  • mypy-extensions ==0.4.3
  • namesgenerator ==0.3
  • nbclassic ==0.4.3
  • nbclient ==0.6.6
  • nbconvert ==6.5.3
  • nbformat ==5.4.0
  • nest-asyncio ==1.5.5
  • networkx ==2.6.3
  • notebook ==6.4.12
  • notebook-shim ==0.1.0
  • numpy ==1.21.5
  • oauthlib ==3.2.0
  • omegaconf ==2.2.2
  • opencv-contrib-python ==4.6.0.66
  • opencv-python ==4.6.0.66
  • openmim ==0.2.1
  • ordered-set ==4.1.0
  • packaging ==21.3
  • pandas ==1.3.5
  • pandocfilters ==1.5.0
  • parameterized ==0.8.1
  • parso ==0.8.3
  • pathlib2 ==2.3.7.post1
  • pathspec ==0.9.0
  • pathtools ==0.1.2
  • pexpect ==4.8.0
  • pickleshare ==0.7.5
  • pkgutil_resolve_name ==1.3.10
  • platformdirs ==4.0.0
  • pluggy ==1.0.0
  • ply ==3.11
  • portalocker ==2.5.1
  • proglog ==0.1.10
  • prometheus-client ==0.14.1
  • promise ==2.3
  • prompt-toolkit ==3.0.30
  • protobuf ==3.19.4
  • psutil ==5.9.0
  • ptyprocess ==0.7.0
  • py ==1.11.0
  • py-spy ==0.3.14
  • pyOpenSSL ==22.0.0
  • pyarrow ==6.0.1
  • pyasn1 ==0.4.8
  • pyasn1-modules ==0.2.8
  • pycocotools ==2.0.4
  • pycparser ==2.21
  • pycryptodomex ==3.19.1
  • pycurl ==7.43.0.2
  • pydot ==1.4.2
  • pyparsing ==3.0.9
  • pyrsistent ==0.18.1
  • pytest ==6.2.5
  • pytest-cov ==3.0.0
  • pytest-cover ==3.0.0
  • pytest-coverage ==0.0
  • python-apt ==1.8.4.3
  • python-consul ==1.1.0
  • python-dateutil ==2.8.2
  • python-engineio ==4.8.1
  • python-etcd ==0.4.5
  • python-jose ==3.2.0
  • python-snappy ==0.6.1
  • python-socketio ==5.10.0
  • pytorchvideo ==0.1.5
  • pytz ==2022.2.1
  • pyzmq ==23.2.1
  • qtconsole ==5.3.1
  • regex ==2022.7.25
  • requests ==2.27.1
  • requests-oauthlib ==1.3.1
  • requests-toolbelt ==0.9.1
  • rich ==12.5.1
  • rsa ==4.9
  • schedule ==0.6.0
  • scikit-learn ==1.0.2
  • scipy ==1.7.3
  • seaborn ==0.11.2
  • sentry-sdk ==1.9.5
  • setproctitle ==1.3.2
  • shortuuid ==1.0.9
  • simple-websocket ==1.0.0
  • simplejson ==3.17.6
  • six ==1.16.0
  • sklearn ==0.0
  • smmap ==5.0.0
  • sniffio ==1.2.0
  • soupsieve ==2.3.2.post1
  • tabulate ==0.8.10
  • tensorboard ==2.10.0
  • tensorboard-data-server ==0.6.1
  • tensorboard-plugin-wit ==1.8.1
  • termcolor ==1.1.0
  • terminado ==0.15.0
  • terminaltables ==3.1.10
  • threadpoolctl ==3.1.0
  • thriftpy2 ==0.4.14
  • timm ==0.6.7
  • tinycss2 ==1.1.1
  • toml ==0.10.2
  • tomli ==2.0.1
  • tornado ==6.2
  • tox ==3.28.0
  • tqdm ==4.64.0
  • traitlets ==5.3.0
  • typed-ast ==1.5.4
  • typing_extensions ==4.7.1
  • unattended-upgrades ==0.1
  • urllib3 ==1.26.12
  • virtualenv ==20.25.0
  • watchdog ==3.0.0
  • wcwidth ==0.2.5
  • webdataset ==0.1.103
  • webencodings ==0.5.1
  • websocket-client ==1.3.3
  • widgetsnbextension ==3.6.1
  • wrapt ==1.14.1
  • wsproto ==1.2.0
  • xmltodict ==0.13.0
  • xtcocotools ==1.12
  • xxhash ==3.0.0
  • yacs ==0.1.8
  • yapf ==0.32.0
  • yarl ==1.8.1
  • zipp ==3.8.1
  • zope.event ==4.5.0
  • zope.interface ==5.4.0
  • zstandard ==0.18.0