Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/gabotechs/musicgpt

Generate music based on natural language prompts using LLMs running locally

https://github.com/gabotechs/musicgpt

Science Score: 13.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Committers with academic emails
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (15.5%) to scientific vocabulary

Keywords

ai gpt llm machine-learning music
Last synced: 9 months ago · JSON representation

Repository

Generate music based on natural language prompts using LLMs running locally

Basic Info
  • Host: GitHub
  • Owner: gabotechs
  • License: mit
  • Language: Rust
  • Default Branch: main
  • Homepage:
  • Size: 1.78 MB
Statistics
  • Stars: 1,134
  • Watchers: 15
  • Forks: 122
  • Open Issues: 13
  • Releases: 58
Topics
ai gpt llm machine-learning music
Created about 2 years ago · Last pushed over 1 year ago
Metadata Files
Readme License

README.md

MusicGPT Signway logo

Generate music based on natural language prompts using LLMs running locally.

https://github.com/gabotechs/MusicGPT/assets/45515538/f0276e7c-70e5-42fc-817a-4d9ee9095b4c

☝️ Turn up the volume!

Overview

MusicGPT is an application that allows running the latest music generation AI models locally in a performant way, in any platform and without installing heavy dependencies like Python or machine learning frameworks.

Right now it only supports MusicGen by Meta, but the plan is to support different music generation models transparently to the user.

The main milestones for the project are: - [x] Text conditioned music generation - [ ] Melody conditioned music generation - [ ] Indeterminately long / infinite music streams

Install

Mac and Linux

MusicGPT can be installed on Mac and Linux using brew:

shell brew install gabotechs/taps/musicgpt

Windows

Download and install MusicGPT's executable file following this link.

All platforms

Precompiled binaries are available for the following platforms: - macOS Apple Silicon - Linux x86_64 - Windows

Just downloading them and executing them should be enough.

Docker (Recommend for running with CUDA)

If you want to run MusicGPT with a CUDA enabled GPU, this is the best way, as you only need to have the basic NVIDIA drivers installed in your system.

shell docker pull gabotechs/musicgpt

Once the image is downloaded, you can run it with:

shell docker run -it --gpus all -p 8642:8642 -v ~/.musicgpt:/root/.local/share/musicgpt gabotechs/musicgpt --gpu --ui-expose

With cargo

If you have the Rust toolchain installed in your system, you can install it with cargo.

shell cargo install musicgpt

Usage

There are two ways of interacting with MusicGPT: the UI mode and the CLI mode.

UI mode

This mode will display a chat-like web application for exchanging prompts with the LLM. It will: - store your chat history - allow you to play the generated music samples whenever you want - generate music samples in the background - allow you to use the UI in a device different from the one executing the LLMs

You can run the UI by just executing the following command:

shell musicgpt

You can also choose different models for running inference, and whether to use a GPU or not, for example:

shell musicgpt --gpu --model medium

[!WARNING]
Most models require really powerful hardware for running inference

If you want to use a CUDA enabled GPU, it's recommended that you run MusicGPT with Docker:

shell docker run -it --gpus all -p 8642:8642 -v ~/.musicgpt:/root/.local/share/musicgpt gabotechs/musicgpt --ui-expose --gpu

CLI mode

This mode will generate and play music directly in the terminal, allowing you to provide multiple prompts and playing audio as soon as it's generated. You can generate audio based on a prompt with the following command:

shell musicgpt "Create a relaxing LoFi song"

By default, it produces a sample of 10s, which can be configured up to 30s:

shell musicgpt "Create a relaxing LoFi song" --secs 30

There's multiple models available, it will use the smallest one by default, but you can opt into a bigger model:

shell musicgpt "Create a relaxing LoFi song" --model medium

[!WARNING]
Most models require really powerful hardware for running inference

If you want to use a CUDA enabled GPU, it's recommended that you run MusicGPT with Docker:

shell docker run -it --gpus all -v ~/.musicgpt:/root/.local/share/musicgpt gabotechs/musicgpt --gpu "Create a relaxing LoFi song"

You can review all the options available running:

shell musicgpt --help

Benchmarks

The following graph shows the inference time taken for generating 10 seconds of audio using different models on a Mac M1 Pro. For comparison, it's Python equivalent using https://github.com/huggingface/transformers is shown.

The command used for generating the 10 seconds of audio was:

shell musicgpt '80s pop track with bassy drums and synth'

This is the Python script used for generating the 10 seconds of audio ```python import scipy import time from transformers import AutoProcessor, MusicgenForConditionalGeneration processor = AutoProcessor.from_pretrained("facebook/musicgen-small") model = MusicgenForConditionalGeneration.from_pretrained("facebook/musicgen-small") inputs = processor( text=["80s pop track with bassy drums and synth"], padding=True, return_tensors="pt", ) start = time.time() audio_values = model.generate(**inputs, max_new_tokens=500) print(time.time() - start) # Log time taken in generation sampling_rate = model.config.audio_encoder.sampling_rate scipy.io.wavfile.write("musicgen_out.wav", rate=sampling_rate, data=audio_values[0, 0].numpy()) ```

Storage

MusicGPT needs access to your storage in order to save downloaded models and generated audios along with some metadata needed for the application to work properly. Assuming your username is foo, it will store the data in the following locations:

  • Windows: C:\Users\foo\AppData\Roaming\gabotechs\musicgpt
  • MacOS: /Users/foo/Library/Application\ Support/com.gabotechs.musicgpt
  • Linux: /home/foo/.config/musicgpt

License

The code is licensed under a MIT License, but the AI model weights that get downloaded at application startup are licensed under the CC-BY-NC-4.0 License as they are generated based on the following repositories:

  • https://huggingface.co/facebook/musicgen-small
  • https://huggingface.co/facebook/musicgen-medium
  • https://huggingface.co/facebook/musicgen-large
  • https://huggingface.co/facebook/musicgen-melody

Owner

  • Name: Gabriel
  • Login: gabotechs
  • Kind: user
  • Location: Madrid, Spain

Passionate developer and new technology lover aiming to make the world a better place with software 💪

GitHub Events

Total
  • Create event: 26
  • Release event: 21
  • Issues event: 25
  • Watch event: 429
  • Delete event: 6
  • Issue comment event: 50
  • Push event: 90
  • Pull request event: 12
  • Fork event: 61
Last Year
  • Create event: 26
  • Release event: 21
  • Issues event: 25
  • Watch event: 429
  • Delete event: 6
  • Issue comment event: 50
  • Push event: 90
  • Pull request event: 12
  • Fork event: 61

Committers

Last synced: about 1 year ago

All Time
  • Total Commits: 286
  • Total Committers: 5
  • Avg Commits per committer: 57.2
  • Development Distribution Score (DDS): 0.255
Past Year
  • Commits: 114
  • Committers: 3
  • Avg Commits per committer: 38.0
  • Development Distribution Score (DDS): 0.254
Top Committers
Name Email Commits
gabotechs g****e@g****m 213
github-actions g****s@g****m 65
arantxa a****a@h****m 5
GabrielMusat g****e@g****m 2
Robert Vitonsky r****y@p****m 1
Committer Domains (Top 20 + Academic)
github.com: 1

Issues and Pull Requests

Last synced: 10 months ago

All Time
  • Total issues: 26
  • Total pull requests: 26
  • Average time to close issues: 2 months
  • Average time to close pull requests: about 20 hours
  • Total issue authors: 24
  • Total pull request authors: 4
  • Average comments per issue: 2.31
  • Average comments per pull request: 0.08
  • Merged pull requests: 22
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 17
  • Pull requests: 11
  • Average time to close issues: about 2 months
  • Average time to close pull requests: about 5 hours
  • Issue authors: 15
  • Pull request authors: 2
  • Average comments per issue: 1.76
  • Average comments per pull request: 0.09
  • Merged pull requests: 9
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • DjagbleyEmmanuel (3)
  • mcoyle1960 (1)
  • mattcaron (1)
  • MrSteelRat (1)
  • Elwag (1)
  • Phrixus2023 (1)
  • appeal2audacity (1)
  • javadevmtl (1)
  • googlebox007 (1)
  • pheonixfirewingz (1)
  • ZoeyKL (1)
  • gitfabianmeyer (1)
  • rubeniskov (1)
  • Tidusle (1)
  • airtonix (1)
Pull Request Authors
  • gabotechs (22)
  • rubeniskov (2)
  • szabgab (2)
  • vitonsky (2)
Top Labels
Issue Labels
Pull Request Labels

Packages

  • Total packages: 1
  • Total downloads:
    • cargo 34,348 total
  • Total dependent packages: 0
  • Total dependent repositories: 0
  • Total versions: 34
  • Total maintainers: 1
crates.io: musicgpt

Generate music based on natural language prompts using LLMs running locally

  • Versions: 34
  • Dependent Packages: 0
  • Dependent Repositories: 0
  • Downloads: 34,348 Total
Rankings
Dependent repos count: 27.7%
Dependent packages count: 32.7%
Average: 52.3%
Downloads: 96.5%
Maintainers (1)
gabotechs
Last synced: 10 months ago

Dependencies

.github/actions/setup/action.yml actions
  • Swatinem/rust-cache v2 composite
  • actions/cache v4 composite
  • awalsh128/cache-apt-pkgs-action latest composite
  • dtolnay/rust-toolchain stable composite
Dockerfile docker
  • nvidia/cuda 11.6.1-cudnn8-devel-ubuntu20.04 build
web/package-lock.json npm
  • 294 dependencies
web/package.json npm
  • @dep-tree/cli ^0.19.9 development
  • @types/react ^18.2.66 development
  • @types/react-dom ^18.2.22 development
  • @types/uuid ^9.0.8 development
  • @typescript-eslint/eslint-plugin ^7.8.0 development
  • @typescript-eslint/parser ^7.2.0 development
  • @vitejs/plugin-react-swc ^3.5.0 development
  • autoprefixer ^10.4.19 development
  • eslint ^8.57.0 development
  • eslint-plugin-react-hooks ^4.6.0 development
  • eslint-plugin-react-refresh ^0.4.6 development
  • postcss ^8.4.38 development
  • tailwindcss ^3.4.3 development
  • typescript ^5.2.2 development
  • vite ^5.2.0 development
  • vite-plugin-singlefile ^2.0.1 development
  • @rspc/client ^0.2.0
  • react ^18.2.0
  • react-dom ^18.2.0
  • react-h5-audio-player ^3.9.1
  • react-router-dom ^6.23.1
  • react-use-websocket ^4.8.1
  • uuid ^9.0.1
.github/workflows/ci.yml actions
  • Swatinem/rust-cache v2 composite
  • actions/cache v4 composite
  • actions/checkout v4 composite
  • awalsh128/cache-apt-pkgs-action latest composite
  • dtolnay/rust-toolchain stable composite
Cargo.lock cargo
  • 296 dependencies
Cargo.toml cargo