Table of content

1. Quick start
2. Warnings
3. Encoders
4. Decoders
5. Pretrained
6. Pretrained old configs fixes
7. Datasets
8. Losses
9. Metrics
10. Optimizers
11. Schedulers
12. Examples
13. Wandb

Quick start

1. Straight install.

1.1 Install torch with cuda.

bash pip install -U torch --extra-index-url https://download.pytorch.org/whl/cu113

1.2 Install opennn_pytorch.

bash pip install -U opennn_pytorch

2. Dockerfile.

bash cd docker/ docker build -t opennn:latest .

Warnings

1. Cuda is only supported for nvidia graphics cards.
2. Alexnet decoder doesn't support bce losses family.
3. Sometimes combination of dataset/encoder/decoder/loss will give bad results, try to combine others.
4. Custom cross-entropy support only mode when preds have (n, c) shape and labels have (n) shape.
5. Not all options in transform.yaml and config.yaml are required.
6. Mean and std in datasets section must be used in transform.yaml, for example [mean=[0.2859], std=[0.3530]] -> normalize: [[0.2859], [0.3530]]

Encoders

• LeNet [paper] [code]
• AlexNet [paper] [code]
• GoogleNet [paper] [code]
• ResNet18 [paper] [code]
• ResNet34 [paper] [code]
• ResNet50 [paper] [code]
• ResNet101 [paper] [code]
• ResNet152 [paper] [code]
• MobileNet [paper] [code]
• VGG-11 [paper] [code]
• VGG-16 [paper] [code]
• VGG-19 [paper] [code]

Decoders

• LeNet [paper] [code]
• AlexNet [paper] [code]
• Linear [docs] [code]

Pretrained

Pretrained configs issues

Config file changed, check configs folder!!! 1. Config must include testpart value, (trainpart + validpart + testpart) value can be < 1.0. 2. You will able to add wandb structure for logging in wandb. 3. Full restructure into branches structure.

Datasets

• Any torchvision dataset [torchvision] [docs] [code]
• Custom dataset [custom] [example] [code]

Dataset parameters: - MNIST [classes=10] [mean=[0.1307], std=[0.3801]] - FASHION-MNIST [classes=10] [mean=[0.2859], std=[0.3530]] - CIFAR-10 [classes=10] [mean=[0.491, 0.482, 0.446], std=[0.247, 0.243, 0.261]] - CIFAR-100 [classes=100] [mean=[0.5071, 0.4867, 0.4408], std=[0.2675, 0.2565, 0.2761]]

Losses

• Any pytorch loss [pytorch] [docs] [code]
• Custom loss [custom] [docs] [code]

Metrics

• Any sklearn metric [sklearn] [docs] [code]

Optimizers

• Any pytorch optimizer [pytorch] [docs] [code]

Schedulers

• Any pytorch scheduler [pytorch] [docs] [code]
• PolynomialLRDecay [custom] [docs] [code]

Examples

1. Run from yaml config. ```python from opennn_pytorch import run

config = 'path to yaml config' # check configs folder run(config) ```

1. Get encoder and decoder. ```python from opennnpytorch.encoder import getencoder from opennnpytorch.decoder import getdecoder

encodername = 'ResNet18' decodername = 'AlexNet' decodermode = 'Single' inputchannels = 1 number_classes = 10 device = 'cuda:0'

encoder = getencoder(encodername, input_channels).to(device)

model = getdecoder(decodername, encoder, numberclasses, decodermode, device).to(device) ```

3.1 Get dataset. ```python from opennnpytorch.dataset import getdataset from torchvision import transforms

transformconfig = 'path to transform yaml config' datasetname = 'MNIST' datafiles = None trainpart = 0.7 validpart = 0.2 test_part = 0.05

transformlst = opennnpytorch.transformslst(transformconfig) transform = transforms.Compose(transform_lst)

traindata, validdata, testdata = getdataset(datasetname, trainpart, validpart, testpart, transform, datafiles) ```

3.2 Get custom dataset. ```python from opennnpytorch.dataset import getdataset from torchvision import transforms

transformconfig = 'path to transform yaml config' datasetname = 'CUSTOM' images = 'path to folder with images' annotation = 'path to annotation yaml file with image: class structure' datafiles = (images, annotation) trainpart = 0.7 validpart = 0.2 test_part = 0.05

transformlst = opennnpytorch.transformslst(transformconfig) transform = transforms.Compose(transform_lst)

traindata, validdata, testdata = getdataset(datasetname, trainpart, validpart, testpart, transform, datafiles) ```

1. Get optimizer. ```python from opennnpytorch.optimizer import getoptimizer

optimizername = 'RAdam' lr = 1e-3 weightdecay = 1e-5 optimizerparams = {'lr': lr, 'weightdecay': weight_decay}

optimizer = getoptimizer(optimizername, model, optimizer_params) ```

1. Get scheduler. ```python from opennnpytorch.scheduler import getscheduler

schedulername = 'PolynomialLRDecay' schedulertype = 'custom' schedulerparams = {'maxdecaysteps': 20, 'endlearning_rate': 0.0005, 'power': 0.9}

scheduler = getscheduler(schedulername, optimizer, schedulerparams, schedulertype) ```

1. Get loss function. ```python from opennnpytorch.loss import getloss

lossfn = 'L1Loss' lossfn, onehot = getloss(loss_fn) ```

1. Get metrics functions. ```python from opennnpytorch.metric import getmetric

metricsnames = ['accuracy', 'precision', 'recall', 'f1score'] numberclasses = 10 metricsfn = getmetric(metricsnames, nc=number_classes) ```

1. Train/Test. ```python from opennn_pytorch.algo import train, test, prediction import random

algorithm = 'train' batchsize = 16 classnames = ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9'] numberclasses = 10 saveevery = 5 epochs = 20 wandbflag = True wandbmetrics = ['accuracy', 'f1_score']

traindataloader = torch.utils.data.DataLoader(traindata, batchsize=batchsize, shuffle=True)

validdataloader = torch.utils.data.DataLoader(validdata, batchsize=batchsize, shuffle=False)

testdataloader = torch.utils.data.DataLoader(testdata, batch_size=1, shuffle=False)

if algorithm == 'train': train(traindataloader, validdataloader, model, optimizer, scheduler, lossfn, metricsfn, epochs, checkpoints, logs, device, saveevery, onehot, numberclasses, wandbflag, wandbmetrics) elif algorithm == 'test': testlogs = test(testdataloader, model, lossfn, metricsfn, logs, device, onehot, numberclasses, wandbflag, wandbmetrics) if pred: indices = random.sample(range(0, len(testdata)), 10) os.mkdir(testlogs + '/prediction', 0o777) for i in range(10): tmprange = range(numberclasses) tmpdct = {i: names[i] for i in tmprange} prediction(testdata, model, device, tmpdct, testlogs + f'/prediction/{i}', indices[i]) ```

Wandb

Wandb is very powerful logging tool, you will able to log metrics, hyperparamets, model hooks etc.

bash wandb login <your api token from wandb.ai>

Citation

Project citation.

License

Project is distributed under MIT License.