TorchMetrics - Measuring Reproducibility in PyTorch

TorchMetrics - Measuring Reproducibility in PyTorch - Published in JOSS (2022)

https://github.com/Lightning-AI/torchmetrics

Science Score: 77.0%

This score indicates how likely this project is to be science-related based on various indicators:

✓
CITATION.cff file
Found CITATION.cff file
✓
codemeta.json file
Found codemeta.json file
✓
.zenodo.json file
Found .zenodo.json file
✓
DOI references
Found 3 DOI reference(s) in README
✓
Academic publication links
Links to: joss.theoj.org, zenodo.org
✓
Committers with academic emails
7 of 282 committers (2.5%) from academic institutions
○
Institutional organization owner
○
JOSS paper metadata
○
Scientific vocabulary similarity
Low similarity (13.5%) to scientific vocabulary

Keywords

analyses data-science deep-learning machine-learning metrics python pytorch

Keywords from Contributors

exoplanet energy-system hydrology mesh neural-style-transfer optimade-python optimade-specification interpretability agent-based-modeling agent-based-simulation

Scientific Fields

Mathematics Computer Science - 63% confidence

Last synced: 4 months ago · JSON representation ·

Repository

Machine learning metrics for distributed, scalable PyTorch applications.

Basic Info

Host: GitHub
Owner: Lightning-AI
License: apache-2.0
Language: Python
Default Branch: master
Homepage: https://lightning.ai/docs/torchmetrics/
Size: 13 MB

Statistics

Stars: 2,328
Watchers: 33
Forks: 455
Open Issues: 81
Releases: 65

Topics

analyses data-science deep-learning machine-learning metrics python pytorch

Created about 5 years ago · Last pushed 4 months ago

Metadata Files

Readme Changelog Contributing License Code of conduct Citation Codeowners

README.md

**Machine learning metrics for distributed, scalable PyTorch applications.** ______________________________________________________________________

What is Torchmetrics • Implementing a metric • Built-in metrics • Docs • Community • License

______________________________________________________________________ [![PyPI - Python Version](https://img.shields.io/pypi/pyversions/torchmetrics)](https://pypi.org/project/torchmetrics/) [![PyPI Status](https://badge.fury.io/py/torchmetrics.svg)](https://badge.fury.io/py/torchmetrics) [![PyPI - Downloads](https://img.shields.io/pypi/dm/torchmetrics) ](https://pepy.tech/project/torchmetrics) [![Conda](https://img.shields.io/conda/v/conda-forge/torchmetrics?label=conda&color=success)](https://anaconda.org/conda-forge/torchmetrics) [![license](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://github.com/Lightning-AI/torchmetrics/blob/master/LICENSE) [![CI testing | CPU](https://github.com/Lightning-AI/torchmetrics/actions/workflows/ci-tests.yml/badge.svg?event=push)](https://github.com/Lightning-AI/torchmetrics/actions/workflows/ci-tests.yml) [![Build Status](https://dev.azure.com/Lightning-AI/Metrics/_apis/build/status%2FTM.unittests?branchName=master)](https://dev.azure.com/Lightning-AI/Metrics/_build/latest?definitionId=54&branchName=master) [![codecov](https://codecov.io/gh/Lightning-AI/torchmetrics/branch/master/graph/badge.svg?token=NER6LPI3HS)](https://codecov.io/gh/Lightning-AI/torchmetrics) [![pre-commit.ci status](https://results.pre-commit.ci/badge/github/Lightning-AI/torchmetrics/master.svg)](https://results.pre-commit.ci/latest/github/Lightning-AI/torchmetrics/master) [![Documentation Status](https://readthedocs.org/projects/torchmetrics/badge/?version=latest)](https://torchmetrics.readthedocs.io/en/latest/?badge=latest) [![Discord](https://img.shields.io/discord/1077906959069626439?style=plastic)](https://discord.gg/VptPCZkGNa) [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.5844769.svg)](https://doi.org/10.5281/zenodo.5844769) [![JOSS status](https://joss.theoj.org/papers/561d9bb59b400158bc8204e2639dca43/status.svg)](https://joss.theoj.org/papers/561d9bb59b400158bc8204e2639dca43) ______________________________________________________________________

Installation

Simple installation from PyPI

bash pip install torchmetrics

Other installations

Install using conda ```bash conda install -c conda-forge torchmetrics ``` Install using uv ```bash uv add torchmetrics ``` Pip from source ```bash # with git pip install git+https://github.com/Lightning-AI/torchmetrics.git@release/stable ``` Pip from archive ```bash pip install https://github.com/Lightning-AI/torchmetrics/archive/refs/heads/release/stable.zip ``` Extra dependencies for specialized metrics: ```bash pip install torchmetrics[audio] pip install torchmetrics[image] pip install torchmetrics[text] pip install torchmetrics[all] # install all of the above ``` Install latest developer version ```bash pip install https://github.com/Lightning-AI/torchmetrics/archive/master.zip ```

What is TorchMetrics

TorchMetrics is a collection of 100+ PyTorch metrics implementations and an easy-to-use API to create custom metrics. It offers:

A standardized interface to increase reproducibility
Reduces boilerplate
Automatic accumulation over batches
Metrics optimized for distributed-training
Automatic synchronization between multiple devices

You can use TorchMetrics with any PyTorch model or with PyTorch Lightning to enjoy additional features such as:

Module metrics are automatically placed on the correct device.
Native support for logging metrics in Lightning to reduce even more boilerplate.

Using TorchMetrics

Module metrics

The module-based metrics contain internal metric states (similar to the parameters of the PyTorch module) that automate accumulation and synchronization across devices!

Automatic accumulation over multiple batches
Automatic synchronization between multiple devices
Metric arithmetic

This can be run on CPU, single GPU or multi-GPUs!

For the single GPU/CPU case:

```python import torch

import our library

import torchmetrics

initialize metric

metric = torchmetrics.classification.Accuracy(task="multiclass", num_classes=5)

move the metric to device you want computations to take place

device = "cuda" if torch.cuda.is_available() else "cpu" metric.to(device)

nbatches = 10 for i in range(nbatches): # simulate a classification problem preds = torch.randn(10, 5).softmax(dim=-1).to(device) target = torch.randint(5, (10,)).to(device)

# metric on current batch
acc = metric(preds, target)
print(f"Accuracy on batch {i}: {acc}")

metric on all batches using custom accumulation

acc = metric.compute() print(f"Accuracy on all data: {acc}") ```

Module metric usage remains the same when using multiple GPUs or multiple nodes.

Example using DDP

```python import os import torch import torch.distributed as dist import torch.multiprocessing as mp from torch import nn from torch.nn.parallel import DistributedDataParallel as DDP import torchmetrics def metric_ddp(rank, world_size): os.environ["MASTER_ADDR"] = "localhost" os.environ["MASTER_PORT"] = "12355" # create default process group dist.init_process_group("gloo", rank=rank, world_size=world_size) # initialize model metric = torchmetrics.classification.Accuracy(task="multiclass", num_classes=5) # define a model and append your metric to it # this allows metric states to be placed on correct accelerators when # .to(device) is called on the model model = nn.Linear(10, 10) model.metric = metric model = model.to(rank) # initialize DDP model = DDP(model, device_ids=[rank]) n_epochs = 5 # this shows iteration over multiple training epochs for n in range(n_epochs): # this will be replaced by a DataLoader with a DistributedSampler n_batches = 10 for i in range(n_batches): # simulate a classification problem preds = torch.randn(10, 5).softmax(dim=-1) target = torch.randint(5, (10,)) # metric on current batch acc = metric(preds, target) if rank == 0: # print only for rank 0 print(f"Accuracy on batch {i}: {acc}") # metric on all batches and all accelerators using custom accumulation # accuracy is same across both accelerators acc = metric.compute() print(f"Accuracy on all data: {acc}, accelerator rank: {rank}") # Resetting internal state such that metric ready for new data metric.reset() # cleanup dist.destroy_process_group() if __name__ == "__main__": world_size = 2 # number of gpus to parallelize over mp.spawn(metric_ddp, args=(world_size,), nprocs=world_size, join=True) ```

Implementing your own Module metric

Implementing your own metric is as easy as subclassing an torch.nn.Module. Simply, subclass torchmetrics.Metric and just implement the update and compute methods:

```python import torch from torchmetrics import Metric

class MyAccuracy(Metric): def init(self): # remember to call super super().init() # call self.add_statefor every internal state that is needed for the metrics computations # distreducefx indicates the function that should be used to reduce # state from multiple processes self.addstate("correct", default=torch.tensor(0), distreducefx="sum") self.addstate("total", default=torch.tensor(0), distreducefx="sum")

def update(self, preds: torch.Tensor, target: torch.Tensor) -> None:
    # extract predicted class index for computing accuracy
    preds = preds.argmax(dim=-1)
    assert preds.shape == target.shape
    # update metric states
    self.correct += torch.sum(preds == target)
    self.total += target.numel()

def compute(self) -> torch.Tensor:
    # compute final result
    return self.correct.float() / self.total

my_metric = MyAccuracy() preds = torch.randn(10, 5).softmax(dim=-1) target = torch.randint(5, (10,))

print(my_metric(preds, target)) ```

Functional metrics

Similar to torch.nn, most metrics have both a module-based and functional version. The functional versions are simple python functions that as input take torch.tensors and return the corresponding metric as a torch.tensor.

```python import torch

import our library

import torchmetrics

simulate a classification problem

preds = torch.randn(10, 5).softmax(dim=-1) target = torch.randint(5, (10,))

acc = torchmetrics.functional.classification.multiclassaccuracy( preds, target, numclasses=5 ) ```

Covered domains and example metrics

In total TorchMetrics contains 100+ metrics, which covers the following domains:

Audio
Classification
Detection
Information Retrieval
Image
Multimodal (Image-Text-3D Talking Heads)
Nominal
Regression
Segmentation
Text

Each domain may require some additional dependencies which can be installed with pip install torchmetrics[audio], pip install torchmetrics['image'] etc.

Additional features

Plotting

Visualization of metrics can be important to help understand what is going on with your machine learning algorithms. Torchmetrics have built-in plotting support (install dependencies with pip install torchmetrics[visual]) for nearly all modular metrics through the .plot method. Simply call the method to get a simple visualization of any metric!

```python import torch from torchmetrics.classification import MulticlassAccuracy, MulticlassConfusionMatrix

num_classes = 3

this will generate two distributions that comes more similar as iterations increase

w = torch.randn(num_classes) target = lambda it: torch.multinomial((it * w).softmax(dim=-1), 100, replacement=True) preds = lambda it: torch.multinomial((it * w).softmax(dim=-1), 100, replacement=True)

acc = MulticlassAccuracy(numclasses=numclasses, average="micro") accperclass = MulticlassAccuracy(numclasses=numclasses, average=None) confmat = MulticlassConfusionMatrix(numclasses=numclasses)

plot single value

for i in range(5): accperclass.update(preds(i), target(i)) confmat.update(preds(i), target(i)) fig1, ax1 = accperclass.plot() fig2, ax2 = confmat.plot()

plot multiple values

values = [] for i in range(10): values.append(acc(preds(i), target(i))) fig3, ax3 = acc.plot(values) ```

For examples of plotting different metrics try running this example file.

Contribute!

The lightning + TorchMetrics team is hard at work adding even more metrics. But we're looking for incredible contributors like you to submit new metrics and improve existing ones!

Join our Discord to get help with becoming a contributor!

Community

For help or questions, join our huge community on Discord!

Citation

We’re excited to continue the strong legacy of open source software and have been inspired over the years by Caffe, Theano, Keras, PyTorch, torchbearer, ignite, sklearn and fast.ai.

If you want to cite this framework feel free to use GitHub's built-in citation option to generate a bibtex or APA-Style citation based on this file (but only if you loved it 😊).

License

Please observe the Apache 2.0 license that is listed in this repository. In addition, the Lightning framework is Patent Pending.

Owner

Name: ⚡️ Lightning AI
Login: Lightning-AI
Kind: organization
Location: United States of America

Website: https://lightning.ai/
Twitter: LightningAI
Repositories: 38
Profile: https://github.com/Lightning-AI

Turn ideas into AI, Lightning fast. Creators of PyTorch Lightning, Lightning AI Studio, TorchMetrics, Fabric, Lit-GPT, Lit-LLaMA

Citation (CITATION.cff)

cff-version: 1.2.0
message: "If you want to cite the framework, feel free to use this (but only if you loved it 😊)"
title: "TorchMetrics - Measuring Reproducibility in PyTorch"
abstract:
  "A main problem with reproducing machine learning publications is the variance of metric implementations across papers.
  A lack of standardization leads to different behavior in mech- anisms such as checkpointing, learning rate schedulers or early stopping, that will influence the reported results.
  For example, a complex metric such as Fréchet inception distance (FID) for synthetic image quality evaluation will differ based on the specific interpolation method used.
  There have been a few attempts at tackling the reproducibility issues.
  Papers With Code links research code with its corresponding paper. Similarly, arXiv recently added a code and data section that links both official and community code to papers.
  However, these methods rely on the paper code to be made publicly accessible which is not always possible.
  Our approach is to provide the de-facto reference implementation for metrics.
  This approach enables proprietary work to still be comparable as long as they’ve used our reference implementations.
  We introduce TorchMetrics, a general-purpose metrics package that covers a wide variety of tasks and domains used in the machine learning community.
  TorchMetrics provides standard classification and regression metrics; and domain-specific metrics for audio, computer vision, natural language processing, and information retrieval.
  Our process for adding a new metric is as follows, first we integrate a well-tested and established third-party library.
  Once we’ve verified the implementations and written tests for them, we re-implement them in native PyTorch to enable hardware acceleration and remove any bottlenecks in inter-device transfer."
authors:
  - name: Nicki Skafte Detlefsen
    orcid: "https://orcid.org/0000-0002-8133-682X"
  - name: Jiri Borovec
    orcid: "https://orcid.org/0000-0001-7437-824X"
  - name: Justus Schock
    orcid: "https://orcid.org/0000-0003-0512-3053"
  - name: Ananya Harsh
  - name: Teddy Koker
  - name: Luca Di Liello
  - name: Daniel Stancl
  - name: Changsheng Quan
  - name: Maxim Grechkin
  - name: William Falcon

doi: 10.21105/joss.04101
license: "Apache-2.0"
url: "https://www.pytorchlightning.ai"
repository-code: "https://github.com/Lightning-AI/torchmetrics"
date-released: 2022-02-11
keywords:
  - machine learning
  - deep learning
  - artificial intelligence
  - metrics
  - pytorch

GitHub Events

Total

Create event: 225
Release event: 14
Issues event: 165
Watch event: 200
Delete event: 195
Issue comment event: 790
Push event: 1,059
Pull request review event: 564
Pull request review comment event: 304
Pull request event: 544
Fork event: 57

Last Year

Create event: 225
Release event: 14
Issues event: 165
Watch event: 200
Delete event: 195
Issue comment event: 790
Push event: 1,060
Pull request review event: 564
Pull request review comment event: 304
Pull request event: 544
Fork event: 57

Committers

Last synced: 5 months ago

All Time

Total Commits: 2,245
Total Committers: 282
Avg Commits per committer: 7.961
Development Distribution Score (DDS): 0.686

Past Year

Commits: 369
Committers: 45
Avg Commits per committer: 8.2
Development Distribution Score (DDS): 0.629

Top Committers

Name	Email	Commits
Jirka	j**c@s**z	705
Nicki Skafte Detlefsen	s**i@g**m	490
dependabot[bot]	4****]	383
Daniel Stancl	4****d	47
Rittik Panda	r**4@g**m	27
Changsheng Quan	q**s@q**m	22
pre-commit-ci[bot]	6****]	20
Justus Schock	1****k	18
Bas Krahmer	b**r@g**m	15
Luca Di Liello	l**o@u**t	15
Shion Matsumoto	s**7@g**m	14
Maxim Grechkin	m**2@g**m	13
deepsource-autofix[bot]	6****]	13
Santiago Castro	s**o@u**u	12
Akihiro Nitta	n**a@a**m	10
Reagan Lee	9****e	9
William Falcon	w**7@c**u	9
Ethan Harris	e**s@g**m	8
Adam J. Stewart	a**6@g**m	8
Irakli Salia	6****0	8
Teddy Koker	t**r@g**m	8
Ashutosh Kumar	a**1@g**m	7
Tobias Kupek	t****k	7
edenlightning	6****g	7
thomas chaton	t**s@g**i	7
Rohit Gupta	r**8@g**m	6
Tadej Svetina	t**a@g**m	6
twsl	4****l	6
Adrian Wälchli	a**i@g**m	6
Valérian Rey	3****y	5
and 252 more...

Committer Domains (Top 20 + Academic)

qq.com: 2 outlook.de: 2 shuai.guru: 1 student.tue.nl: 1 wisc.edu: 1 amazon.com: 1 tailanian.com: 1 claas.plus: 1 todorkov.de: 1 dhs.sg: 1 tecnico.ulisboa.pt: 1 protonmail.ch: 1 standigm.com: 1 pytorchlightning.ai: 1 grid.ai: 1 columbia.edu: 1 akihironitta.com: 1 umich.edu: 1 unitn.it: 1 seznam.cz: 1 tu-dortmund.de: 1 uni-leipzig.de: 1 aqueducthq.com: 1

Issues and Pull Requests

Last synced: 4 months ago

All Time

Total issues: 441
Total pull requests: 1,504
Average time to close issues: 2 months
Average time to close pull requests: 15 days
Total issue authors: 338
Total pull request authors: 114
Average comments per issue: 2.82
Average comments per pull request: 1.59
Merged pull requests: 1,169
Bot issues: 3
Bot pull requests: 626

Past Year

Issues: 118
Pull requests: 669
Average time to close issues: 19 days
Average time to close pull requests: 6 days
Issue authors: 97
Pull request authors: 51
Average comments per issue: 1.25
Average comments per pull request: 1.39
Merged pull requests: 478
Bot issues: 1
Bot pull requests: 290

View more stats

Top Authors

Issue Authors

Borda (12)
stancld (10)
ValerianRey (5)
Yann-CV (5)
nkaenzig (5)
robmarkcole (5)
rittik9 (4)
michael080808 (4)
OmerShubi (4)
awaelchli (4)
SkafteNicki (4)
donglihe-hub (4)
Queuecumber (4)
gratus907 (3)
ashinkajay (3)

Pull Request Authors

dependabot[bot] (613)
Borda (297)
SkafteNicki (233)
rittik9 (67)
baskrahmer (31)
matsumotosan (17)
Isalia20 (16)
pre-commit-ci[bot] (13)
adamjstewart (10)
stancld (8)
quancs (7)
martin-kokos (6)
i-aki-y (6)
gratus907 (5)
nkaenzig (4)

Top Labels

Issue Labels

help wanted (262) bug / fix (246) enhancement (136) topic: Image (36) v1.2.x (34) New metric (34) documentation (26) question (23) v1.3.x (19) v1.4.x (16) good first issue (16) v0.11.x (16) v1.1.x (12) waiting on author (12) topic: Audio (11) v0.9.x (9) topic: Text (8) v1.6.x (8) v1.0.x (7) refactoring (7) v0.8.x (6) test / CI (6) Important (5) v0.10.x (5) Priority (5) working as intended (5) v1.7.x (5) v0.3.x (3) API / design (3) wontfix (3)

Pull Request Labels

test / CI (704) ready (555) 0:] Ready-To-Go (303) documentation (190) bug / fix (178) topic: Classif (127) enhancement (126) topic: Image (112) topic: Detect (83) topic: Text (74) topic: Regress (67) topic: Audio (55) New metric (54) topic: MModal (38) topic: Cluster (35) Priority (29) topic: Retrieval (27) refactoring (24) topic: Nominal (13) has conflicts (9) waiting on author (7) Important (6) API / design (2) help wanted (1) v1.2.x (1) v1.5.x (1)

Packages

Total packages: 1
Total downloads:
- pypi 9,347,314 last-month
Total docker downloads: 26,112,751

Total dependent packages: 340
Total dependent repositories: 7,449
Total versions: 72
Total maintainers: 3

pypi.org: torchmetrics

PyTorch native Metrics

Homepage: https://github.com/Lightning-AI/torchmetrics
Documentation: https://torchmetrics.rtfd.io/en/latest/
License: Apache-2.0
Latest release: 1.8.2
published 4 months ago

Versions: 72
Dependent Packages: 340
Dependent Repositories: 7,449
Downloads: 9,347,314 Last month
Docker Downloads: 26,112,751

Rankings

Dependent packages count: 0.1%

Dependent repos count: 0.1%

Downloads: 0.1%

Docker downloads count: 0.6%

Average: 0.9%

Stargazers count: 1.7%

Forks count: 2.8%

Maintainers (3)

Borda justusschock lantiga

Last synced: 4 months ago

Dependencies

.github/actions/unittesting/action.yml actions

actions/upload-artifact v2 composite

.github/workflows/ci-integrate.yml actions

./.github/actions/caching * composite
actions/checkout v3 composite
actions/setup-python v4 composite

.github/workflows/focus-diff.yml actions

actions/checkout v3 composite
actions/setup-python v4 composite

.github/workflows/greetings.yml actions

actions/first-interaction v1 composite

.devcontainer/Dockerfile docker

mcr.microsoft.com/vscode/devcontainers/python ${VARIANT} build

requirements/audio.txt pypi

pystoi *

requirements/audio_test.txt pypi

fast-bss-eval >=0.1.0 test
mir_eval >=0.6 test
pypesq >1.2 test
torch_complex * test

requirements/classification_test.txt pypi

netcal * test

requirements/detection.txt pypi

pycocotools *
torchvision >=0.8

requirements/detection_test.txt pypi

pycocotools * test

requirements/docs.txt pypi

docutils >=0.16
myst-parser *
nbsphinx >=0.8
pandoc >=1.0
pt-lightning-sphinx-theme *
sphinx >=4.0,<5.0
sphinx-autodoc-typehints >=1.0
sphinx-copybutton >=0.3
sphinx-paramlinks >=0.5.1
sphinx-togglebutton >=0.2
sphinxcontrib-fulltoc >=1.0
sphinxcontrib-mockautodoc *

requirements/doctest.txt pypi

pytest * test
pytest-doctestplus * test
pytest-rerunfailures * test

requirements/image.txt pypi

lpips *
scipy *
torch-fidelity *
torchvision *

requirements/image_test.txt pypi

kornia >=0.6.7 test
pytorch-msssim ==0.2.1 test
scikit-image >0.17.1 test

requirements/integrate.txt pypi

pytorch-lightning >=1.5

requirements/multimodal.txt pypi

transformers >=4.10.0

requirements/nominal_test.txt pypi

dython * test
pandas * test
scipy * test

requirements/test.txt pypi

check-manifest * test
cloudpickle >=1.3 test
coverage >5.2 test
fire * test
mypy ==0.982 test
phmdoctest >=1.1.1 test
pre-commit >=1.0 test
psutil * test
pytest ==6. test
pytest-cov >2.10 test
pytest-doctestplus >=0.9.0 test
pytest-rerunfailures >=10.0 test
pytest-timeout * test
requests * test
scikit-learn >1.0,<1.1.1 test
types-PyYAML * test
types-emoji * test
types-protobuf * test
types-requests * test
types-setuptools * test
types-six * test
types-tabulate * test

requirements/text.txt pypi

nltk >=3.6
regex >=2021.9.24
tqdm >=4.41.0

requirements/text_test.txt pypi

bert_score ==0.3.10 test
huggingface-hub <0.7 test
jiwer >=2.3.0 test
rouge-score >=0.0.4 test
sacrebleu >=2.0.0 test
transformers >4.4.0 test

requirements/visual.txt pypi

matplotlib >=3.2.0

requirements.txt pypi

lightning-utilities >=0.4.1
numpy >=1.17.2
packaging *
torch >=1.8.1
typing-extensions *

.github/actions/pull-caches/action.yml actions

actions/cache/restore v3 composite

.github/actions/push-caches/action.yml actions

actions/cache/restore v3 composite
actions/cache/save v3 composite

.github/workflows/ci-checks.yml actions

.github/workflows/clear-cache.yml actions

.github/workflows/docs-build.yml actions

./.github/actions/pull-caches * composite
./.github/actions/push-caches * composite
actions/cache v3 composite
actions/checkout v3 composite
actions/download-artifact v3 composite
actions/setup-python v4 composite
actions/upload-artifact v3 composite
google-github-actions/auth v1 composite
google-github-actions/setup-gcloud v1 composite

pyproject.toml pypi

requirements/devel.txt pypi

requirements/typing.txt pypi

mypy ==1.4.1
types-PyYAML *
types-emoji *
types-protobuf *
types-requests *
types-setuptools *
types-six *
types-tabulate *

setup.py pypi

.github/workflows/ci-rtfd.yml actions

readthedocs/actions/preview v1 composite

.github/workflows/ci-tests.yml actions

./.github/actions/pull-caches * composite
./.github/actions/push-caches * composite
./.github/actions/unittesting * composite
actions/checkout v3 composite
actions/setup-python v4 composite

.github/workflows/docker-build.yml actions

actions/checkout v3 composite
docker/build-push-action v4 composite
docker/login-action v2 composite

.github/workflows/labeler.yml actions

actions/checkout v3 composite
actions/labeler v4 composite

.github/workflows/publish-pkg.yml actions

AButler/upload-release-assets v2.0 composite
actions/checkout v3 composite
actions/download-artifact v3 composite
actions/setup-python v4 composite
actions/upload-artifact v3 composite
juliangruber/sleep-action v1 composite
pypa/gh-action-pypi-publish v1.8.10 composite