Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/brainlesion/panoptica

panoptica -- instance-wise evaluation of 3D semantic and instance segmentation maps

https://github.com/brainlesion/panoptica

Science Score: 36.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ✓
    Academic publication links
    Links to: arxiv.org
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (15.2%) to scientific vocabulary

Keywords

3d evaluation instance instance-wise metrics panoptic segmentation semantic
Last synced: 5 months ago · JSON representation

Repository

panoptica -- instance-wise evaluation of 3D semantic and instance segmentation maps

Basic Info
Statistics
  • Stars: 26
  • Watchers: 1
  • Forks: 7
  • Open Issues: 27
  • Releases: 59
Topics
3d evaluation instance instance-wise metrics panoptic segmentation semantic
Created over 2 years ago · Last pushed 6 months ago
Metadata Files
Readme License

README.md

panoptica

Python Versions Stable Version Documentation Status tests codecov License

Computing instance-wise segmentation quality metrics for 2D and 3D semantic- and instance segmentation maps.

Use Cases & Tutorials | Documentation

Features

The package provides three core modules:

  1. Instance Approximator: instance approximation algorithms to extract instances from semantic segmentation maps/model outputs.
  2. Instance Matcher: matches predicted instances with reference instances.
  3. Instance Evaluator: computes segmentation and detection quality metrics for pairs of predicted - and reference segmentation maps.

workflow_figure

Installation

With a Python 3.10+ environment, you can install panoptica from pypi.org

sh pip install panoptica

Or, alternatively, from conda:

sh conda install -c conda-forge panoptica

Available Metrics

[!NOTE] Panoptica supports a large range of metrics.
An overview of the supported metrics and their formulas can be found here: panoptica/metrics.md

Use Cases & Tutorials

Minimal example

A minimal example of using panoptica could look e.g. like this (here with Matched Instances as Input): ```python from panoptica import InputType, Panoptica_Evaluator from panoptica.metrics import Metric

from auxiliary.io import read_image # feel free to use any other way to read nifti files

refmasks = readimage("reference.nii.gz") predmasks = readimage("prediction.nii.gz")

evaluator = PanopticaEvaluator( expectedinput=InputType.MATCHEDINSTANCE, decisionmetric=Metric.IOU, decision_threshold=0.5, )

result = evaluator.evaluate(predmasks, refmasks)["ungrouped"] ```

[!TIP] We provide Jupyter Notebook tutorials showcasing various use cases.
You can explore them here: BrainLesion/tutorials/panoptica

Semantic Segmentation Input

semantic_figure

Although an instance-wise evaluation is highly relevant and desirable for many biomedical segmentation problems, they are still addressed as semantic segmentation problems due to the lack of appropriate instance labels.

This tutorial leverages all three modules of panoptica: instance approximation, -matching and -evaluation.

nbviewer Open In Colab

Unmatched Instances Input

unmatched_instance_figure

It is a common issue that instance segmentation outputs feature good outlines but mismatched instance labels. For this case, the matcher module can be utilized to match instances and the evaluator to report metrics.

nbviewer Open In Colab

Matched Instances Input

matched_instance_figure

nbviewer Open In Colab

If your predicted instances already match the reference instances, you can directly compute metrics using the evaluator module.

Matching Algorithm Example

nbviewer Open In Colab

Using Configs (saving and loading)

You can construct Panoptica_Evaluator (among many others) objects and save their arguments, so you can save project-specific configurations and use them later. It uses ruamel.yaml in a readable way.

nbviewer Open In Colab

Using Panoptica Aggregator

Using our Panoptica Aggregator, you can evaluate your data while the Aggregator takes care of writing the results into a .tsv file (threading-safe). This allows you to focus only on the important part, that is loading and iterating over your data. From the aggregator, you can immediately get a Panoptica Statistics object that yield you the statistics over your metrics as well as create nice figures for you.

Tutorial: TBD

Documentation

We provide a readthedocs documentation of our codebase here

Citation

[!IMPORTANT] If you use panoptica in your research, please cite it to support the development!

Kofler, F., Möller, H., Buchner, J. A., de la Rosa, E., Ezhov, I., Rosier, M., Mekki, I., Shit, S., Negwer, M., Al-Maskari, R., Ertürk, A., Vinayahalingam, S., Isensee, F., Pati, S., Rueckert, D., Kirschke, J. S., Ehrlich, S. K., Reinke, A., Menze, B., Wiestler, B., & Piraud, M. (2023). Panoptica -- instance-wise evaluation of 3D semantic and instance segmentation maps. arXiv preprint arXiv:2312.02608.

@misc{kofler2023panoptica, title={Panoptica -- instance-wise evaluation of 3D semantic and instance segmentation maps}, author={Florian Kofler and Hendrik Möller and Josef A. Buchner and Ezequiel de la Rosa and Ivan Ezhov and Marcel Rosier and Isra Mekki and Suprosanna Shit and Moritz Negwer and Rami Al-Maskari and Ali Ertürk and Shankeeth Vinayahalingam and Fabian Isensee and Sarthak Pati and Daniel Rueckert and Jan S. Kirschke and Stefan K. Ehrlich and Annika Reinke and Bjoern Menze and Benedikt Wiestler and Marie Piraud}, year={2023}, eprint={2312.02608}, archivePrefix={arXiv}, primaryClass={cs.CV} }

Contributing

We welcome all kinds of contributions from the community!

Some guides on extending panoptica can be found in panoptica/extending_panoptica.md

Reporting Bugs, Feature Requests and Questions

Please open a new issue here.

Owner

  • Name: BrainLesion
  • Login: BrainLesion
  • Kind: organization

GitHub Events

Total
  • Create event: 32
  • Release event: 9
  • Issues event: 62
  • Watch event: 4
  • Delete event: 18
  • Member event: 1
  • Issue comment event: 193
  • Push event: 80
  • Pull request review comment event: 139
  • Pull request review event: 145
  • Pull request event: 50
  • Fork event: 2
Last Year
  • Create event: 32
  • Release event: 9
  • Issues event: 62
  • Watch event: 4
  • Delete event: 18
  • Member event: 1
  • Issue comment event: 193
  • Push event: 80
  • Pull request review comment event: 139
  • Pull request review event: 145
  • Pull request event: 50
  • Fork event: 2

Issues and Pull Requests

Last synced: 6 months ago

All Time
  • Total issues: 30
  • Total pull requests: 20
  • Average time to close issues: 20 days
  • Average time to close pull requests: 2 days
  • Total issue authors: 9
  • Total pull request authors: 3
  • Average comments per issue: 0.87
  • Average comments per pull request: 4.0
  • Merged pull requests: 15
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 30
  • Pull requests: 20
  • Average time to close issues: 20 days
  • Average time to close pull requests: 2 days
  • Issue authors: 9
  • Pull request authors: 3
  • Average comments per issue: 0.87
  • Average comments per pull request: 4.0
  • Merged pull requests: 15
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • Hendrik-code (21)
  • neuronflow (18)
  • aymuos15 (12)
  • MarcelRosier (5)
  • sarthakpati (4)
  • ogencoglu (3)
  • JosefBuchner (3)
  • evamariie (1)
  • ai-idt (1)
  • scap3yvt (1)
Pull Request Authors
  • Hendrik-code (37)
  • MarcelRosier (9)
  • neuronflow (6)
  • sarthakpati (5)
  • aymuos15 (3)
  • robert-graf (1)
Top Labels
Issue Labels
enhancement (50) metric (5) bug (4) question (4) documentation (4) good first issue (2) wontfix (1) help wanted (1)
Pull Request Labels
enhancement (21) bug (9) documentation (4) metric (3) speedimprove (1)

Packages

  • Total packages: 1
  • Total downloads:
    • pypi 7,575 last-month
  • Total dependent packages: 0
  • Total dependent repositories: 1
  • Total versions: 76
  • Total maintainers: 2
pypi.org: panoptica

Panoptic Quality (PQ) computation for binary masks.

  • Versions: 76
  • Dependent Packages: 0
  • Dependent Repositories: 1
  • Downloads: 7,575 Last month
Rankings
Downloads: 6.7%
Dependent packages count: 10.1%
Average: 15.4%
Forks count: 16.9%
Dependent repos count: 21.5%
Stargazers count: 21.6%
Maintainers (2)
neuronflow marcelrosier
Last synced: 6 months ago

Dependencies

pyproject.toml pypi
  • pytest ^6.2.5 develop
  • pytest-cov ^3.0.0 develop
  • pytest-mock ^3.6.0 develop
  • connected-components-3d ^3.12.3
  • numpy ^1.20.0
  • python ^3.8
  • scipy ^1.7.0
.github/workflows/release.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v4 composite
.github/workflows/tests.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v3 composite
.github/workflows/autoformat.yml actions
.github/workflows/pr-lint.yml actions