Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/bytedance/music_source_separation

https://github.com/bytedance/music_source_separation

Science Score: 23.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ✓
    Academic publication links
    Links to: zenodo.org
  • ○
    Committers with academic emails
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (10.2%) to scientific vocabulary

Keywords

research
Last synced: 10 months ago · JSON representation

Repository

Basic Info
  • Host: GitHub
  • Owner: bytedance
  • License: other
  • Language: Python
  • Default Branch: master
  • Homepage:
  • Size: 1.25 MB
Statistics
  • Stars: 1,355
  • Watchers: 27
  • Forks: 200
  • Open Issues: 50
  • Releases: 0
Topics
research
Created about 5 years ago · Last pushed over 2 years ago
Metadata Files
Readme License

README.md

Music Source Separation

Music source separation is a task to separate audio recordings into individual sources. This repository is an PyTorch implmementation of music source separation. Users can separate their favorite songs into different sources by installing this repository. Users can also train their own source separation systems. This repository can also be used for training speech enhancement, instruments separation, and any separation systems.

Demos

Vocals and accompaniment separation: https://www.youtube.com/watch?v=WH4m5HYzHsg

Installation

Install Python 3.7.

Sometimes the installation of bytesep will fail when using higher versions of Python. Suggest to use conda (or other environment manage tools) to manage the packages.

bash pip install bytesep==0.1.1

Separation

After installation, to separate your favorite song is easy. Users can execute the following commands in any directory.

bash python3 -m bytesep download_checkpoints

bash python3 -m bytesep separate \ --source_type="vocals" \ --audio_path="./resources/vocals_accompaniment_10s.mp3" \ --output_path="separated_results/output.mp3"

Users can also put many audio files into a directory and separate them all.

bash python3 -m bytesep separate \ --source_type="vocals" \ --audio_path="audios_directory" \ --output_path="outputs_directory"

The currently supported source types include "vocals" and "accompaniment". Users could also plug this MSS system into their own programmes. See example.py for examples.

The separation models are trained ONLY on the Musdb18 dataset (100 songs). Trained checkpoints can be downloaded at: https://zenodo.org/record/5804160.

Train a music source separation system from scratch

0. Download dataset

Users could train on the MUSDB18 dataset to reproduce our music source separation systems. Execute the following script to download and unzip the MUSDB18 dataset:

bash ./scripts/0_download_datasets/musdb18.sh

The dataset looks like: 

./datasets/musdb18
├── train (100 files)
│   ├── 'A Classic Education - NightOwl.stem.mp4'
│   └── ...
├── test (50 files)
│   ├── 'Al James - Schoolboy Facination.stem.mp4'
│   └── ...
└── README.md

1. Pack audio files into hdf5 files

Pack audio waveforms into hdf5 files to speed up training. bash ./scripts/1_pack_audios_to_hdf5s/musdb18/sr=44100,chn=2.sh

2. Create indexes for training

bash ./scripts/2_create_indexes/musdb18/create_indexes.sh

3. Create evaluation audios

bash ./scripts/3_create_evaluation_audios/musdb18/create_evaluation_audios.sh

4. Train & evaluate & save checkpoints

bash ./scripts/4_train/musdb18/train.sh

5. Separate using user trained checkpoint

bash ./scripts/5_separate/musdb18/separate.sh

Results

1. Separation Metrics

The following table shows the signal to noise ratio (SDR) metrics of vocals and accompaniment. The MSS systems are only trained with 100 songs from the MUSDB18 dataset. The metrics are calculated on the 50 test songs. It is highly suggest to use the subband version because it is faster to train and inference.

| Model | vocals (dB) | accompaniment (dB) | |---------------------------|-------------|--------------------| | ResUNet143 vocals | 8.9 | 16.8 | | ResUNet143 Subband vocals | 8.7 | 16.4 | | MobileNet Subband vocals | 7.2 | 14.6 |

2. Parameters number & speed

The following table shows the number of parameters and inference time of a 1-min audio clip.

| Model | Trainable params. num | process 1-min time (GPU Tesla V100) | process 1-min time (CPU Core i7) | |--------------------|------------------------|-------------------------------------|----------------------------------| | ResUNet143 ISMIR | 102 million | 2.24 s | 53.00 s | | ResUNet143 Subband | 102 million | 0.56 s | 13.68 s | | MobileNet Subband | 0.306 million | 0.33 s | 9.84 |

3. Metrics over step

The evaluation metrics over different stesp is shown below.

Finetune on new datasets

Users can finetuen pretrained checkpoints on new datasets. The following script is a template showing how to finetune pretrained a MSS system to the VCTK dataset for speech enhancement. (This is just an example. There is no problem if users do not have the VCTK dataset.) Users can also resume the training from a checkpoint by modifying the following script.

bash ./scripts/4_train/vctk-musdb18/finetune.sh

Cite

[1] Qiuqiang Kong, Yin Cao, Haohe Liu, Keunwoo Choi, Yuxuan Wang, Decoupling Magnitude and Phase Estimation with Deep ResUNet for Music Source Separation, International Society for Music Information Retrieval (ISMIR), 2021. @inproceedings{kong2021decoupling, title={Decoupling Magnitude and Phase Estimation with Deep ResUNet for Music Source Separation.}, author={Kong, Qiuqiang and Cao, Yin and Liu, Haohe and Choi, Keunwoo and Wang, Yuxuan }, booktitle={ISMIR}, year={2021}, organization={Citeseer} }

Contact

Qiquiang Kong

Frequent Asked Questions (FAQ)

FAQ.md

External Links

Other open sourced music source separation projects include but not limited to:

Subband ResUNet: https://github.com/haoheliu/Subband-Music-Separation

Demucs: https://github.com/facebookresearch/demucs

Spleeter: https://github.com/deezer/spleeter

Asteroid: https://github.com/asteroid-team/asteroid

Open-Unmix: https://github.com/sigsep/open-unmix-pytorch

Owner

  • Name: Bytedance Inc.
  • Login: bytedance
  • Kind: organization
  • Location: Singapore

GitHub Events

Total
  • Watch event: 85
  • Issue comment event: 4
  • Fork event: 7
Last Year
  • Watch event: 85
  • Issue comment event: 4
  • Fork event: 7

Committers

Last synced: about 1 year ago

All Time
  • Total Commits: 45
  • Total Committers: 4
  • Avg Commits per committer: 11.25
  • Development Distribution Score (DDS): 0.356
Past Year
  • Commits: 0
  • Committers: 0
  • Avg Commits per committer: 0.0
  • Development Distribution Score (DDS): 0.0
Top Committers
Name Email Commits
kongqiuqiang k****g@b****m 29
QiuqiangKong q****g@g****m 10
Christian Clauss c****s@m****m 5
song s****g@s****n 1
Committer Domains (Top 20 + Academic)
sensorsdata.cn: 1 me.com: 1 bytedance.com: 1

Issues and Pull Requests

Last synced: 11 months ago

All Time
  • Total issues: 64
  • Total pull requests: 11
  • Average time to close issues: 7 days
  • Average time to close pull requests: 3 months
  • Total issue authors: 54
  • Total pull request authors: 11
  • Average comments per issue: 1.81
  • Average comments per pull request: 0.64
  • Merged pull requests: 2
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 2
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 2
  • Pull request authors: 0
  • Average comments per issue: 0.0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • mrlihellohorld (3)
  • labellmg (2)
  • Heporis (2)
  • Blakey-Gavin (2)
  • AK391 (2)
  • lanyuemingning (2)
  • sophia1488 (2)
  • lucasbr15 (2)
  • jinleileiking (2)
  • Akarinnnnnnn (1)
  • seq2193 (1)
  • yeyuanzheng177 (1)
  • BigCat20196 (1)
  • hihei (1)
  • poor1017 (1)
Pull Request Authors
  • jinleileiking (2)
  • songminger (1)
  • 007xiaoliang (1)
  • dingjibang (1)
  • cclauss (1)
  • Freddd13 (1)
  • ctrlcctrlv (1)
  • AK391 (1)
  • sophia1488 (1)
  • xalanq (1)
  • leminhnguyen (1)
Top Labels
Issue Labels
Pull Request Labels

Packages

  • Total packages: 1
  • Total downloads:
    • pypi 110 last-month
  • Total docker downloads: 42
  • Total dependent packages: 0
  • Total dependent repositories: 1
  • Total versions: 6
  • Total maintainers: 1
pypi.org: bytesep

Music source separation

  • Versions: 6
  • Dependent Packages: 0
  • Dependent Repositories: 1
  • Downloads: 110 Last month
  • Docker Downloads: 42
Rankings
Stargazers count: 1.9%
Docker downloads count: 3.4%
Forks count: 3.7%
Average: 9.4%
Dependent packages count: 10.1%
Downloads: 15.6%
Dependent repos count: 21.6%
Maintainers (1)
qiuqiangkong
Last synced: 11 months ago

Dependencies

requirements.txt pypi
  • h5py ==2.10.0
  • librosa ==0.8.0
  • matplotlib ==3.3.4
  • musdb ==0.4.0
  • museval ==0.4.0
  • numpy ==1.18.5
  • pytorch_lightning ==1.2.1
  • samplerate ==0.1.0
  • torch ==1.7.1
  • torchlibrosa ==0.0.9
setup.py pypi
  • h5py ==2.10.0
  • librosa ==0.8.0
  • matplotlib ==3.3.4
  • musdb ==0.4.0
  • museval ==0.4.0
  • numpy ==1.18.5
  • pytorch_lightning ==1.2.1
  • samplerate ==0.1.0
  • torch ==1.7.1
  • torchlibrosa ==0.0.9
pyproject.toml pypi