pythae - Pythae 0.1.2

New features - Migration to pydantic=2.* (#105) - Supports custom collate function thanks to @fbosshard (#83) - Adds auto mixed precision to BaseTrainer thanks to @liamchalcroft (#90)

Minor changes - Unifies Gaussian likelihood for all (VAE-based) model implementations (#104) - Updates predict method in RHVAE thanks to @soumickmj (#80) - Adds clamping to SVAE model for stability thanks to @soumickmj (#79)

- Python
Published by clementchadebec over 2 years ago

pythae - Pythae 0.1.1

New features - Added the trainingcallback TrainHistoryCallback that stores the training metrics during training in #71 by @VolodyaCO ```python from pythae.trainers.trainingcallbacks import TrainHistoryCallback

trainhistory = TrainHistoryCallback() callbacks = [trainhistory] pipeline( ... traindata=traindataset, ... evaldata=evaldataset, ... callbacks=callbacks ... ) trainhistory.history ... { ... 'trainloss': [58.51896972363562, 42.15931177749049, 40.583426756017346], ... 'evalloss': [43.39408182034827, 41.45351771943888, 39.77221281209569] ... } - Added a `predict` method that encodes and decodes input data without loss computation in #75 by @soumickmj and @ravih18python out = model.predict(evaldataset[:3]) out.embedding.shape, out.reconx.shape ... (torch.Size([3, 16]), torch.Size([3, 1, 28, 28])) - Added `embed` method that returns the latent representations of the input data in #76 by @tbouchikpython out = model.embed(evaldataset[:3].to(device)) out.shape ... torch.Size([3, 16]) ```

- Python
Published by clementchadebec about 3 years ago

pythae - Pythae 0.1.0

New features :rocket: - Pythae now supports distributed training (built on top of PyTorch DDP). Launching a distributed training can be done using a training script in which all of the distributed environment variables are passed to a BaseTrainerConfig instance as follows:

python training_config = BaseTrainerConfig( num_epochs=10, learning_rate=1e-3, per_device_train_batch_size=64, per_device_eval_batch_size=64, dist_backend="nccl", # distributed backend world_size=8 # number of gpus to use (n_nodes x n_gpus_per_node), rank=0 # process/gpu id, local_rank=1 # node id, master_addr="localhost" # master address, master_port="12345" # master port, )

The script can then be launched using a launcher such a srun. This module was tested in both mono-node-multi-gpu and multi-node-multi-gpu settings.

Thanks to @ravih18, MSSSIM_VAE now supports 3D images :rocket:

Major Changes - Selection and definition of custom optimizers and schedulers changed. It is no longer needed to build the optimizer (resp. scheduler) and pass them to the Trainer. As of v0.1.0, the choice and parameters of the optimizers and schedulers can be passed directly to the TrainerConfig. See changes below:

As of v0.1.0 ```python mymodel = VAE(modelconfig=model_config)

Specify instances and params directly in Trainer config

trainingconfig = BaseTrainerConfig( ..., optimizercls="AdamW", optimizerparams={"betas": (0.91, 0.995)} schedulercls="MultiStepLR", schedulerparams={"milestones": [10, 20, 30], "gamma": 10**(-1/5)} ) trainer = BaseTrainer( model=model, traindataset=traindataset, evaldataset=evaldataset, trainingconfig=training_config )

Launch training

trainer.train() ```

Before v0.1.0 ```python mymodel = VAE(modelconfig=modelconfig) trainingconfig = BaseTrainerConfig(...)

Optimizer

optimizer = torch.optim.AdamW(model.parameters(), lr=trainingconfig.learningrate, betas=(0.91, 0.995))

Scheduler

scheduler = torch.optim.lr_scheduler.MultiStepLR(optimizer, milestones=[10, 20, 30], gamma=10**(-1/5))

Pass instances to Trainer

trainer = BaseTrainer( model=model, traindataset=traindataset, evaldataset=evaldataset, trainingconfig=trainingconfig, optimizer=optimizer, scheduler=scheduler )

Launch training

trainer.train() ```

batch_size key no longer available in the Trainer configurations. It is replaced by the keys per_device_train_batch_size and per_device_eval_batch_size where the batch size per device is specified. Please note that if you are in a distributed setting with for instance 4 GPUs and specify a per_device_eval_batch_size=64, this is equivalent to training on a single GPU using a batch_size of 4*64.

Minor changes - Added the ability to specify the desired number of workers for dataloading in the Trainer configuration under the keys `traindataloadernumworkersandevaldataloadernumworkers- Cleaned upinitofTrainersand moved *sanity* checks fromtrainmethod toinit- Moved checks onoptimizersandschedulersinTrainerConfingpostinitpostparse__`

- Python
Published by clementchadebec about 3 years ago

pythae - Release 0.0.9

New features - Integration of comet_ml through CometCallback training callbacks further to #55

Bugs fixed :bug: - Fix pickle5 compatibility with python>=3.8 - update conda-forge feedstock with correct requirements (https://github.com/conda-forge/pythae-feedstock/pull/11)