trainer

🐸 - A general purpose model trainer, as flexible as it gets

https://github.com/coqui-ai/trainer

Science Score: 13.0%

This score indicates how likely this project is to be science-related based on various indicators:

○
CITATION.cff file
✓
codemeta.json file
Found codemeta.json file
○
.zenodo.json file
○
DOI references
○
Academic publication links
○
Committers with academic emails
○
Institutional organization owner
○
JOSS paper metadata
○
Scientific vocabulary similarity
Low similarity (11.2%) to scientific vocabulary

Keywords

ai data-science deep-learning machine-learning pytorch

Keywords from Contributors

speaker-encoder glow-tts hifigan melgan multi-speaker-tts speech speech-synthesis tacotron text-to-speech tts

Last synced: 11 months ago · JSON representation

Repository

🐸 - A general purpose model trainer, as flexible as it gets

Basic Info

Host: GitHub
Owner: coqui-ai
Language: Python
Default Branch: main
Homepage:
Size: 263 KB

Statistics

Stars: 224
Watchers: 12
Forks: 145
Open Issues: 21
Releases: 32

Topics

ai data-science deep-learning machine-learning pytorch

Created almost 5 years ago · Last pushed over 2 years ago

Metadata Files

Readme Contributing Code of conduct

👟 Trainer

An opinionated general purpose model trainer on PyTorch with a simple code base.

Installation

From Github:

console git clone https://github.com/coqui-ai/Trainer cd Trainer make install

From PyPI:

console pip install trainer

Prefer installing from Github as it is more stable.

Implementing a model

Subclass and overload the functions in the TrainerModel()

Training a model with auto-optimization

See the MNIST example.

Training a model with advanced optimization

With 👟 you can define the whole optimization cycle as you want as the in GAN example below. It enables more under-the-hood control and flexibility for more advanced training loops.

You just have to use the scaled_backward() function to handle mixed precision training.

```python ...

def optimize(self, batch, trainer): imgs, _ = batch

# sample noise
z = torch.randn(imgs.shape[0], 100)
z = z.type_as(imgs)

# train discriminator
imgs_gen = self.generator(z)
logits = self.discriminator(imgs_gen.detach())
fake = torch.zeros(imgs.size(0), 1)
fake = fake.type_as(imgs)
loss_fake = trainer.criterion(logits, fake)

valid = torch.ones(imgs.size(0), 1)
valid = valid.type_as(imgs)
logits = self.discriminator(imgs)
loss_real = trainer.criterion(logits, valid)
loss_disc = (loss_real + loss_fake) / 2

# step dicriminator
_, _ = self.scaled_backward(loss_disc, None, trainer, trainer.optimizer[0])

if trainer.total_steps_done % trainer.grad_accum_steps == 0:
    trainer.optimizer[0].step()
    trainer.optimizer[0].zero_grad()

# train generator
imgs_gen = self.generator(z)

valid = torch.ones(imgs.size(0), 1)
valid = valid.type_as(imgs)

logits = self.discriminator(imgs_gen)
loss_gen = trainer.criterion(logits, valid)

# step generator
_, _ = self.scaled_backward(loss_gen, None, trainer, trainer.optimizer[1])
if trainer.total_steps_done % trainer.grad_accum_steps == 0:
    trainer.optimizer[1].step()
    trainer.optimizer[1].zero_grad()
return {"model_outputs": logits}, {"loss_gen": loss_gen, "loss_disc": loss_disc}

... ```

See the GAN training example with Gradient Accumulation

Training with Batch Size Finder

see the test script here for training with batch size finder.

The batch size finder starts at a default BS(defaults to 2048 but can also be user defined) and searches for the largest batch size that can fit on your hardware. you should expect for it to run multiple trainings until it finds it. to use it instead of calling trainer.fit() youll call trainer.fit_with_largest_batch_size(starting_batch_size=2048) with starting_batch_size being the batch the size you want to start the search with. very useful if you are wanting to use as much gpu mem as possible.

Training with DDP

console $ python -m trainer.distribute --script path/to/your/train.py --gpus "0,1"

We don't use .spawn() to initiate multi-gpu training since it causes certain limitations.

Everything must the pickable.
.spawn() trains the model in subprocesses and the model in the main process is not updated.
DataLoader with N processes gets really slow when the N is large.

Training with Accelerate

Setting use_accelerate in TrainingArgs to True will enable training with Accelerate.

You can also use it for multi-gpu or distributed training.

console CUDA_VISIBLE_DEVICES="0,1,2" accelerate launch --multi_gpu --num_processes 3 train_recipe_autoregressive_prompt.py

See the Accelerate docs.

Adding a callback

👟 Supports callbacks to customize your runs. You can either set callbacks in your model implementations or give them explicitly to the Trainer.

Please check trainer.utils.callbacks to see available callbacks.

Here is how you provide an explicit call back to a 👟Trainer object for weight reinitialization.

```python def my_callback(trainer): print(" > My callback was called.")

trainer = Trainer(..., callbacks={"oninitend": my_callback}) trainer.fit() ```

Profiling example

Create the torch profiler as you like and pass it to the trainer. python import torch profiler = torch.profiler.profile( activities=[ torch.profiler.ProfilerActivity.CPU, torch.profiler.ProfilerActivity.CUDA, ], schedule=torch.profiler.schedule(wait=1, warmup=1, active=3, repeat=2), on_trace_ready=torch.profiler.tensorboard_trace_handler("./profiler/"), record_shapes=True, profile_memory=True, with_stack=True, ) prof = trainer.profile_fit(profiler, epochs=1, small_run=64) then run Tensorboard
Run the tensorboard. console tensorboard --logdir="./profiler/"

Supported Experiment Loggers

Tensorboard - actively maintained
ClearML - actively maintained
MLFlow
Aim
WandDB

To add a new logger, you must subclass BaseDashboardLogger and overload its functions.

Anonymized Telemetry

We constantly seek to improve 🐸 for the community. To understand the community's needs better and address them accordingly, we collect stripped-down anonymized usage stats when you run the trainer.

Of course, if you don't want, you can opt out by setting the environment variable TRAINER_TELEMETRY=0.

Owner

Name: coqui
Login: coqui-ai
Kind: organization
Email: info@coqui.ai

Website: https://coqui.ai
Twitter: coqui_ai
Repositories: 17
Profile: https://github.com/coqui-ai

Coqui, a startup providing open speech tech for everyone 🐸

GitHub Events

Total

Issues event: 1
Watch event: 26
Issue comment event: 3
Pull request event: 1
Fork event: 25

Last Year

Issues event: 1
Watch event: 26
Issue comment event: 3
Pull request event: 1
Fork event: 25

Committers

Last synced: about 1 year ago

All Time

Total Commits: 199
Total Committers: 14
Avg Commits per committer: 14.214
Development Distribution Score (DDS): 0.216

Past Year

Commits: 0
Committers: 0
Avg Commits per committer: 0.0
Development Distribution Score (DDS): 0.0

Top Committers

Name	Email	Commits
Eren Gölge	e**e@c**i	156
WeberJulian	j**r@h**r	10
Edresson Casanova	e**1@g**m	9
Adam BB	a**r@g**m	8
EC2 Default User	e**r@i**l	3
logan hart	l**l@g**m	2
ivan provalov	i**o@y**m	2
Roee Shenberg	s**g@g**m	2
Enno Hermann	E****d	2
manmay nakhashi	m**i@g**m	1
bitnom	1****m	1
Zutatensuppe	o**a@g**m	1
Peter Pisljar	p**r@g**m	1
AlexanderAbugaliev	a**8@g**m	1

Committer Domains (Top 20 + Academic)

ip-172-31-69-14.ec2.internal: 1 coqui.ai: 1

Issues and Pull Requests

Last synced: 11 months ago

All Time

Total issues: 41
Total pull requests: 78
Average time to close issues: 3 months
Average time to close pull requests: 19 days
Total issue authors: 26
Total pull request authors: 22
Average comments per issue: 1.27
Average comments per pull request: 0.77
Merged pull requests: 58
Bot issues: 0
Bot pull requests: 0

Past Year

Issues: 6
Pull requests: 1
Average time to close issues: 10 days
Average time to close pull requests: N/A
Issue authors: 6
Pull request authors: 1
Average comments per issue: 0.83
Average comments per pull request: 2.0
Merged pull requests: 0
Bot issues: 0
Bot pull requests: 0

View more stats

Top Authors

Issue Authors

erogol (11)
mweinelt (4)
Ca-ressemble-a-du-fake (3)
loganhart420 (1)
mmcc1 (1)
Edresson (1)
mushahid-intesum (1)
kunibald413 (1)
iamkhalidbashir (1)
Eyalm321 (1)
DManowitz (1)
devops724 (1)
xyyimian (1)
mueller91 (1)
alpoktem (1)

Pull Request Authors

erogol (34)
loganhart420 (5)
Edresson (5)
shenberg (3)
eginhard (3)
iprovalo (2)
WeberJulian (2)
thinhlpg (2)
EniddeallA (2)
a-froghyar (2)
Squire-tomsk (1)
tarasfrompir (1)
bitnom (1)
wokalski (1)
gullabi (1)

Top Labels

Issue Labels

bug (26) feature request (9)

Pull Request Labels

enhancement (2)

Packages

Total packages: 4
Total downloads:
- pypi 169,691 last-month
Total docker downloads: 638

Total dependent packages: 5
(may contain duplicates)
Total dependent repositories: 295
(may contain duplicates)
Total versions: 38
Total maintainers: 2

pypi.org: trainer

General purpose model trainer for PyTorch that is more flexible than it should be, by 🐸Coqui.

Homepage: https://github.com/coqui-ai/Trainer
Documentation: https://github.com/coqui-ai/Trainer/
License: Apache2
Latest release: 0.0.36
published over 2 years ago

Versions: 32
Dependent Packages: 5
Dependent Repositories: 293
Downloads: 168,315 Last month
Docker Downloads: 638

Rankings

Dependent repos count: 0.9%

Downloads: 0.9%

Docker downloads count: 2.4%

Dependent packages count: 2.4%

Average: 3.1%

Forks count: 5.4%

Stargazers count: 6.7%

Maintainers (1)

coqui

Last synced: 11 months ago

pypi.org: coqui-trainer

General purpose model trainer for PyTorch that is more flexible than it should be, by 🐸Coqui.

Homepage: https://github.com/coqui-ai/Trainer
Documentation: https://github.com/coqui-ai/Trainer/
License: Apache2
Latest release: 0.0.5
published over 4 years ago

Versions: 2
Dependent Packages: 0
Dependent Repositories: 1
Downloads: 1,371 Last month

Rankings

Forks count: 5.5%

Stargazers count: 6.7%

Dependent packages count: 10.0%

Average: 15.8%

Dependent repos count: 21.7%

Downloads: 35.1%

Maintainers (1)

coqui

Last synced: 11 months ago

pypi.org: trainer4win

General purpose model trainer for PyTorch that is more flexible than it should be, by 🐸Coqui.

Homepage: https://github.com/coqui-ai/Trainer
Documentation: https://github.com/coqui-ai/Trainer/
License: Apache2
Latest release: 0.0.7
published over 4 years ago

Versions: 2
Dependent Packages: 0
Dependent Repositories: 1
Downloads: 5 Last month

Rankings

Forks count: 5.5%

Stargazers count: 6.7%

Dependent packages count: 10.0%

Average: 20.4%

Dependent repos count: 21.7%

Downloads: 58.1%

Maintainers (1)

bitnom

Last synced: 11 months ago

conda-forge.org: coqui-trainer

Homepage: https://github.com/coqui-ai/Trainer
License: Apache-2.0
Latest release: 0.0.5
published over 4 years ago

Versions: 2
Dependent Packages: 0
Dependent Repositories: 0

Rankings

Forks count: 31.5%

Dependent repos count: 34.0%

Stargazers count: 38.0%

Average: 38.7%

Dependent packages count: 51.2%

Last synced: 11 months ago

Dependencies

requirements.dev.txt pypi

black *
coverage *
isort *
pylint ==2.10.2
pytest *

requirements.test.txt pypi

torchvision *

requirements.txt pypi

coqpit *
fsspec *
soundfile *
tensorboardX *
torch >=1.7

.github/workflows/pypi-release.yml actions

actions/checkout v2 composite
actions/download-artifact v2 composite
actions/setup-python v2 composite
actions/upload-artifact v2 composite

.github/workflows/style_check.yml actions

actions/checkout v2 composite
coqui-ai/setup-python pip-cache-key-py-ver composite

.github/workflows/tests.yml actions

actions/checkout v2 composite
coqui-ai/setup-python pip-cache-key-py-ver composite

pyproject.toml pypi

setup.py pypi

trainer

Science Score: 13.0%

Keywords

Keywords from Contributors

Repository

Basic Info

Statistics

Topics

Metadata Files

README.md

👟 Trainer

Installation

Implementing a model

Training a model with auto-optimization

Training a model with advanced optimization

Training with Batch Size Finder

Training with DDP

Training with Accelerate

Adding a callback

Profiling example

Supported Experiment Loggers

Anonymized Telemetry

Owner

GitHub Events

Total

Last Year

Committers

All Time

Past Year

Top Committers

Committer Domains (Top 20 + Academic)

Issues and Pull Requests

All Time

Past Year

Top Authors

Issue Authors

Pull Request Authors

Top Labels

Issue Labels

Pull Request Labels

Packages

pypi.org: trainer

Rankings

Maintainers (1)

pypi.org: coqui-trainer

Rankings

Maintainers (1)

pypi.org: trainer4win

Rankings

Maintainers (1)

conda-forge.org: coqui-trainer

Rankings

Dependencies