Implementations of recent Deep Learning tricks in Computer Vision, easily paired up with your favorite framework and model zoo.

Holocrons were information-storage datacron devices used by both the Jedi Order and the Sith that contained ancient lessons or valuable information in holographic form.

Source: Wookieepedia

Quick Tour

This project was created for quality implementations, increased developer flexibility and maximum compatibility with the PyTorch ecosystem. For instance, here is a short snippet to showcase how Holocron models are meant to be used:

```python from PIL import Image from torchvision.transforms import Compose, ConvertImageDtype, Normalize, PILToTensor, Resize from torchvision.transforms.functional import InterpolationMode from holocron.models.classification import repvgg_a0

Load your model

model = repvgg_a0(pretrained=True).eval()

Read your image

img = Image.open(pathtoan_image).convert("RGB")

Preprocessing

config = model.default_cfg transform = Compose([ Resize(config['input_shape'][1:], interpolation=InterpolationMode.BILINEAR), PILToTensor(), ConvertImageDtype(torch.float32), Normalize(config['mean'], config['std']) ])

input_tensor = transform(img).unsqueeze(0)

Inference

with torch.inferencemode(): output = model(inputtensor) print(config['classes'][output.squeeze(0).argmax().item()], output.squeeze(0).softmax(dim=0).max()) ```

Installation

Prerequisites

Python 3.9 (or higher) and pip/conda are required to install Holocron.

Latest stable release

You can install the last stable release of the package using pypi as follows:

shell pip install pylocron

or using conda:

shell conda install -c frgfm pylocron

Developer mode

Alternatively, if you wish to use the latest features of the project that haven't made their way to a release yet, you can install the package from source (install Git first):

shell git clone https://github.com/frgfm/Holocron.git pip install -e Holocron/.

Paper references

PyTorch layers for every need

• Activation: HardMish, NLReLU, FReLU
• Loss: Focal Loss, MultiLabelCrossEntropy, MixupLoss, ClassBalancedWrapper, ComplementCrossEntropy, MutualChannelLoss, DiceLoss, PolyLoss
• Convolutions: NormConv2d, Add2d, SlimConv2d, PyConv2d, Involution
• Regularization: DropBlock
• Pooling: BlurPool2d, SPP, ZPool
• Attention: SAM, LambdaLayer, TripletAttention

Models for vision tasks

• Image Classification: Res2Net (based on the great implementation from Ross Wightman), Darknet-24, Darknet-19, Darknet-53, CSPDarknet-53, ResNet, ResNeXt, TridentNet, PyConvResNet, ReXNet, SKNet, RepVGG, ConvNeXt, MobileOne.
• Object Detection: YOLOv1, YOLOv2, YOLOv4
• Semantic Segmentation: U-Net, UNet++, UNet3+

Vision-related operations

• boxes: Distance-IoU & Complete-IoU losses

Trying something else than Adam

• Optimizer: LARS, Lamb, TAdam, AdamP, AdaBelief, Adan, and customized versions (RaLars), AdEMAMix
• Optimizer wrapper: Lookahead, Scout (experimental)

More goodies

Documentation

The full package documentation is available here for detailed specifications.

Demo app

The project includes a minimal demo app using Gradio

You can check the live demo, hosted on :hugs: HuggingFace Spaces :hugs: over here :point_down:

Reference scripts

Reference scripts are provided to train your models using holocron on famous public datasets. Those scripts currently support the following vision tasks: - Image classification - Object detection - Semantic segmentation

Latency benchmark

You crave for SOTA performances, but you don't know whether it fits your needs in terms of latency?

In the table below, you will find a latency benchmark for all supported models:

| Arch | GPU mean (std) | CPU mean (std) | | ------------------------------------------------------------ | ----------------- | ------------------ | | repvgg_a0* | 3.14ms (0.87ms) | 23.28ms (1.21ms) | | repvgg_a1* | 4.13ms (1.00ms) | 29.61ms (0.46ms) | | repvgg_a2* | 7.35ms (1.11ms) | 46.87ms (1.27ms) | | repvgg_b0* | 4.23ms (1.04ms) | 33.16ms (0.58ms) | | repvgg_b1* | 12.48ms (0.96ms) | 100.66ms (1.46ms) | | repvgg_b2* | 20.12ms (0.31ms) | 155.90ms (1.59ms) | | repvgg_b3* | 24.94ms (1.70ms) | 224.68ms (14.27ms) | | rexnet1_0x | 6.01ms (0.26ms) | 13.66ms (0.21ms) | | rexnet1_3x | 6.43ms (0.10ms) | 19.13ms (2.05ms) | | rexnet1_5x | 6.46ms (0.28ms) | 21.06ms (0.24ms) | | rexnet2_0x | 6.75ms (0.21ms) | 31.77ms (3.28ms) | | rexnet2_2x | 6.92ms (0.51ms) | 33.61ms (0.60ms) | | sknet50 | 11.40ms (0.38ms) | 54.03ms (3.35ms) | | sknet101 | 23.55 ms (1.11ms) | 94.89ms (5.61ms) | | sknet152 | 69.81ms (0.60ms) | 253.07ms (3.33ms) | | tridentnet50 | 16.62ms (1.21ms) | 142.85ms (5.33ms) | | res2net5026w4s | 9.25ms (0.22ms) | 41.84ms (0.80ms) | | resnet50d | 36.97ms (3.58ms) | 36.97ms (3.58ms) | | pyconv_resnet50 | 20.03ms (0.28ms) | 178.85ms (2.35ms) | | pyconvhg_resnet50 | 38.41ms (0.33ms) | 301.03ms (12.39ms) | | darknet24 | 3.94ms (1.08ms) | 29.39ms (0.78ms) | | darknet19 | 3.17ms (0.59ms) | 26.36ms (2.80ms) | | darknet53 | 7.12ms (1.35ms) | 53.20ms (1.17ms) | | cspdarknet53 | 6.41ms (0.21ms) | 48.05ms (3.68ms) | | cspdarknet53_mish | 6.88ms (0.51ms) | 67.78ms (2.90ms) |

The reported latency for RepVGG models is the one of the reparametrized version

This benchmark was performed over 100 iterations on (224, 224) inputs, on a laptop to better reflect performances that can be expected by common users. The hardware setup includes an Intel(R) Core(TM) i7-10750H for the CPU, and a NVIDIA GeForce RTX 2070 with Max-Q Design for the GPU.

You can run this latency benchmark for any model on your hardware as follows:

bash python scripts/eval_latency.py rexnet1_0x

All script arguments can be checked using python scripts/eval_latency.py --help

Docker container

If you wish to deploy containerized environments, you can use the provided Dockerfile to build a docker image:

shell make build-api

Minimal API template

Looking for a boilerplate to deploy a model from Holocron with a REST API? Thanks to the wonderful FastAPI framework, you can do this easily.

Deploy your API locally

Run your API in a docker container as follows:

shell make start-api

In order to stop the container, use make stop-api

What you have deployed

Your API is now running on port 8080, with its documentation http://localhost:8080/redoc and requestable routes:

python import requests with open('/path/to/your/img.jpeg', 'rb') as f: data = f.read() response = requests.post("http://localhost:8080/classification", files={'file': data}).json()

Citation

If you wish to cite this project, feel free to use this BibTeX reference:

bibtex @software{Fernandez_Holocron_2020, author = {Fernandez, François-Guillaume}, month = {5}, title = {{Holocron}}, url = {https://github.com/frgfm/Holocron}, year = {2020} }

Contributing

Any sort of contribution is greatly appreciated!

You can find a short guide in CONTRIBUTING to help grow this project!

License

Distributed under the Apache 2.0 License. See LICENSE for more information.