km3pipe

Extemporary analysis framework for KM3NeT

https://github.com/tamasgal/km3pipe

Science Score: 59.0%

This score indicates how likely this project is to be science-related based on various indicators:

○
CITATION.cff file
✓
codemeta.json file
Found codemeta.json file
✓
.zenodo.json file
Found .zenodo.json file
✓
DOI references
Found 2 DOI reference(s) in README
✓
Academic publication links
Links to: zenodo.org
✓
Committers with academic emails
1 of 32 committers (3.1%) from academic institutions
○
Institutional organization owner
○
JOSS paper metadata
○
Scientific vocabulary similarity
Low similarity (16.7%) to scientific vocabulary

Last synced: 7 months ago · JSON representation

Repository

Extemporary analysis framework for KM3NeT

Basic Info

Host: GitHub
Owner: tamasgal
License: mit
Language: Python
Default Branch: master
Size: 120 MB

Statistics

Stars: 14
Watchers: 4
Forks: 2
Open Issues: 0
Releases: 2

Created over 11 years ago · Last pushed about 1 year ago

Metadata Files

Readme Changelog Contributing License Codemeta Zenodo

README.rst

KM3Pipe
=======

.. image:: https://git.km3net.de/km3py/km3pipe/badges/master/pipeline.svg
    :target: https://git.km3net.de/km3py/km3pipe/pipelines

.. image:: https://git.km3net.de/km3py/km3pipe/badges/master/coverage.svg
    :target: https://km3py.pages.km3net.de/km3pipe/coverage

.. image:: https://api.codacy.com/project/badge/Grade/9df4849cb9f840289bf883de0dc8e28f
   :alt: Codacy Badge
   :target: https://app.codacy.com/app/tamasgal/km3pipe?utm_source=github.com&utm_medium=referral&utm_content=tamasgal/km3pipe&utm_campaign=Badge_Grade_Settings

.. image:: https://git.km3net.de/examples/km3badges/-/raw/master/docs-latest-brightgreen.svg
    :target: https://km3py.pages.km3net.de/km3pipe

.. image:: https://zenodo.org/badge/24634697.svg
   :target: https://doi.org/10.5281/zenodo.808829


KM3Pipe is a framework for KM3NeT related stuff including MC, data files, live
access to detectors and databases, parsers for different file formats and an
easy to use framework for batch processing.

The main Git repository, where issues and merge requests are managed can be
found at https://git.km3net.de/km3py/km3pipe.git

The framework tries to standardise the way the data is processed by providing
a Pipeline-class, which can be used to put together different built-in or user
made Pumps, Sinks and Modules. Pumps act as data readers/parsers (from files,
memory or even socket connections), Sinks are responsible for writing data to
disk and Modules take care of data processing, output and user interaction.
Such a Pipeline setup can then be used to iteratively process data in a file or
from a stream. In our case for example, we store several thousands of neutrino
interaction events in a bunch of files and KM3Pipe is used to stitch together
an analysis chain which processes each event one-by-one by passing them through
a pipeline of modules.

Although it is mainly designed for the KM3NeT neutrino detectors, it can easily
be extended to support any kind of data formats. The core functionality is
written in a general way and is applicable to all kinds of data processing
workflows.

To start off, run::

    pip install km3pipe

If you have Docker (https://www.docker.com) installed, you can start using
KM3Pipe immediately by typing::

    docker run -it docker.km3net.de/km3pipe

Feel free to get in touch if you’re looking for a small, versatile framework
which provides a quite straightforward module system to make code exchange
between your project members as easily as possible. KM3Pipe already comes with
several types of Pumps, so it should be easy to find an example to implement
your owns. As of version 8.0.0 you find Pumps and Sinks based on popular
formats like HDF5 (https://www.hdfgroup.org), ROOT (https://root.cern.ch) but
also some very specialised project internal binary data formats, which on the
other hand can act as templates for your own ones. Just have a look at the io
subpackage and of course the documentation if you’re interested!

Read the latest docs at https://km3py.pages.km3net.de/km3pipe.

KM3NeT public project homepage http://www.km3net.org

Acknowledgements
----------------

Thanks especially to the gracious help of all contributors:

Tamas Gal, Moritz Lotze, Johannes Schumann, Piotr Kalaczynski, Jonas Reubelt, Michael Moser, Thomas Heid, Alba Domi, Agustin Sanchez Losa, Zineb Aly, Jordan Seneca, Nicole Geisselbrecht, Javier Barrios, Valentin Pestel, Jannik Hofestaedt, Matthias Bissinger, Vladimir Kulikovskiy, Lukas Hennig, Godefroy Vannoye

Owner

Name: Tamas Gal
Login: tamasgal
Kind: user
Location: Nuremberg (Germany)
Company: Erlangen Centre for Astroparticle Physics, KM3NeT, IceCube

Website: http://www.tamasgal.com
Repositories: 170
Profile: https://github.com/tamasgal

Astroparticle physicist, software developer and DevOps engineer at Erlangen Centre for Astroparticle Physics (ECAP)

CodeMeta (codemeta.json)

{
  "@context": "https://doi.org/10.5063/schema/codemeta-2.0",
  "@type": "SoftwareSourceCode",
  "license": "https://spdx.org/licenses/MIT",
  "codeRepository": "git+https://git.km3net.de/km3py/km3pipe.git",
  "contIntegration": "https://git.km3net.de/km3py/km3pipe/-/pipelines",
  "dateCreated": "2014-09-30",
  "datePublished": "2014-10-08",
  "dateModified": "2021-09-27",
  "downloadUrl": "https://git.km3net.de/km3py/km3pipe",
  "issueTracker": "https://git.km3net.de/km3py/km3pipe/-/issues",
  "name": "km3pipe",
  "version": "9.5.0",
  "description": "Analysis framework for KM3NeT related data, heavily based on NumPy.",
  "applicationCategory": "Astroparticle physics",
  "developmentStatus": "active",
  "referencePublication": "https://doi.org/10.5281/zenodo.808829",
  "keywords": [
    "pipeline",
    "analysis",
    "astroparticle physics",
    "km3net"
  ],
  "programmingLanguage": [
    "Python 3"
  ],
  "operatingSystem": [
    "Linux",
    "macOS",
    "Windows"
  ],
  "relatedLink": [
    "https://km3py.pages.km3net.de/km3pipe/"
  ],
  "author": [
    {
      "@type": "Person",
      "@id": "https://orcid.org/0000-0001-7821-8673",
      "givenName": "Tamas",
      "familyName": "Gal",
      "email": "tamas.gal@fau.de",
      "affiliation": {
        "@type": "Organization",
        "name": "Friedrich-Alexander-University Erlangen-Nuremberg, Erlangen Centre for Astroparticle Physics"
      }
    },
    {
      "@type": "Person",
      "givenName": "Moritz",
      "familyName": "Lotze",
      "email": "mlotze@km3net.de"
    }
  ],
  "contributor": [
    {
      "@type": "Person",
      "@id": "https://orcid.org/0000-0003-3722-086X",
      "givenName": "Johannes",
      "familyName": "Schumann",
      "affiliation": {
        "@type": "Organization",
        "name": "Friedrich-Alexander-University Erlangen-Nuremberg, Erlangen Centre for Astroparticle Physics"
      }
    },
    {
      "@type": "Person",
      "@id": "https://orcid.org/0000-0001-9278-5906",
      "givenName": "Piotr",
      "familyName": "Kalaczynski"
    },
    {
      "@type": "Person",
      "givenName": "Jonas",
      "familyName": "Reubelt",
      "email": "jreubelt@km3net.de",
      "affiliation": {
        "@type": "Organization",
        "name": "Friedrich-Alexander-University Erlangen-Nuremberg, Erlangen Centre for Astroparticle Physics"
      }
    },
    {
      "@type": "Person",
      "@id": "https://orcid.org/0000-0003-3613-488X",
      "givenName": "Michael",
      "familyName": "Moser",
      "affiliation": {
        "@type": "Organization",
        "name": "Friedrich-Alexander-University Erlangen-Nuremberg, Erlangen Centre for Astroparticle Physics"
      }
    },
    {
      "@type": "Person",
      "givenName": "Thomas",
      "familyName": "Heid",
      "email": "theid@km3net.de",
      "affiliation": {
        "@type": "Organization",
        "name": "Friedrich-Alexander-University Erlangen-Nuremberg, Erlangen Centre for Astroparticle Physics"
      }
    },
    {
      "@type": "Person",
      "givenName": "Alba",
      "familyName": "Domi",
      "email": "adomi@km3net.de"
    },
    {
      "@type": "Person",
      "@id": "https://orcid.org/0000-0001-9596-7078",
      "givenName": "Agustin",
      "familyName": "Sanchez Losa"
    },
    {
      "@type": "Person",
      "@id": "https://orcid.org/0000-0002-5593-2580",
      "givenName": "Zineb",
      "familyName": "Aly"
    },
    {
      "@type": "Person",
      "givenName": "Jordan",
      "familyName": "Seneca"
    },
    {
      "@type": "Person",
      "givenName": "Nicole",
      "familyName": "Geisselbrecht",
      "affiliation": {
        "@type": "Organization",
        "name": "Friedrich-Alexander-University Erlangen-Nuremberg, Erlangen Centre for Astroparticle Physics"
      }
    },
    {
      "@type": "Person",
      "@id": "https://orcid.org/0000-0001-6721-4300",
      "givenName": "Javier",
      "familyName": "Barrios Marti"
    },
    {
      "@type": "Person",
      "givenName": "Valentin",
      "familyName": "Pestel"
    },
    {
      "@type": "Person",
      "givenName": "Daniel",
      "familyName": "Guderian"
    },
    {
      "@type": "Person",
      "givenName": "Jannik ",
      "familyName": "Hofestaedt",
      "affiliation": {
        "@type": "Organization",
        "name": "Friedrich-Alexander-University Erlangen-Nuremberg, Erlangen Centre for Astroparticle Physics"
      }
    },
    {
      "@type": "Person",
      "givenName": "Matthias",
      "familyName": "Bissinger",
      "affiliation": {
        "@type": "Organization",
        "name": "Friedrich-Alexander-University Erlangen-Nuremberg, Erlangen Centre for Astroparticle Physics"
      }
    },
    {
      "@type": "Person",
      "@id": "https://orcid.org/0000-0003-4096-5934",
      "givenName": "Vladimir",
      "familyName": "Kulikovskiy"
    },
    {
      "@type": "Person",
      "givenName": "Stefan",
      "familyName": "Reck",
      "affiliation": {
        "@type": "Organization",
        "name": "Friedrich-Alexander-University Erlangen-Nuremberg, Erlangen Centre for Astroparticle Physics"
      }
    },
    {
      "@type": "Person",
      "@id": "https://orcid.org/0000-0002-2816-2242",
      "givenName": "Lukas",
      "familyName": "Hennig",
      "affiliation": {
        "@type": "Organization",
        "name": "Friedrich-Alexander-University Erlangen-Nuremberg, Erlangen Centre for Astroparticle Physics"
      }
    }
  ]
}

GitHub Events

Total

Push event: 9
Create event: 6

Last Year

Push event: 9
Create event: 6

Committers

Last synced: 10 months ago

All Time

Total Commits: 5,836
Total Committers: 32
Avg Commits per committer: 182.375
Development Distribution Score (DDS): 0.296

Past Year

Commits: 16
Committers: 2
Avg Commits per committer: 8.0
Development Distribution Score (DDS): 0.063

Top Committers

Name	Email	Commits
Tamas Gal	t**l@k**e	4,109
Moritz Lotze	l**z@g**m	1,335
Moritz Lotze	m**e@k**e	181
Johannes Schumann	j**n@f**e	39
Piotr Kalaczy?ski	p**i@k**e	34
Jonas Reubelt	j**t@k**e	26
Michael Moser	m**r@k**e	24
zineb aly	a**z@g**m	17
heidtgg	h**g@g**m	16
theid	t**d@f**e	11
Alba Domi	a**i@k**e	7
Thomas Heid	t**d@c**r	6
Stefan Reck	s**k@f**e	4
Agustin Sanchez Losa	a**n@b**t	4
Daniel Guderian	d**n@k**e	3
Stefan Reck	s**k@k**e	3
Zineb Aly	z**y@k**e	2
Godefroy Vannoye	g**e@k**e	1
Jannik Hofestaedt	j**t@k**e	1
Javier Barrios	j**r@i**s	1
Jordan Seneca	j**a@k**e	1
Lukas Maderer	l**r@k**e	1
Matthias Bissinger	m**r@f**e	1
Nicole Geisselbrecht	n**t@k**e	1
Steffen Hallmann	s**n@k**e	1
Tamas Gal	t**l@c**r	1
Thomas Heid	m**6@w**e	1
Valentin Pestel	v**l@k**e	1
ViaFerrata	m**r@o**e	1
theid	t**d@k**e	1
and 2 more...

Committer Domains (Top 20 + Academic)

km3net.de: 17 fau.de: 5 onlinehome.de: 1 woodycap1.rrze.uni-erlangen.de: 1 ccage010.in2p3.fr: 1 ific.uv.es: 1 ba.infn.it: 1 cca009.in2p3.fr: 1

Issues and Pull Requests

Last synced: 10 months ago

All Time

Total issues: 4
Total pull requests: 1
Average time to close issues: 6 months
Average time to close pull requests: 5 minutes
Total issue authors: 2
Total pull request authors: 1
Average comments per issue: 0.0
Average comments per pull request: 0.0
Merged pull requests: 0
Bot issues: 0
Bot pull requests: 0

Past Year

Issues: 0
Pull requests: 0
Average time to close issues: N/A
Average time to close pull requests: N/A
Issue authors: 0
Pull request authors: 0
Average comments per issue: 0
Average comments per pull request: 0
Merged pull requests: 0
Bot issues: 0
Bot pull requests: 0

View more stats

Top Authors

Issue Authors

tamasgal (3)
mtzl (1)

Pull Request Authors

codacy-badger (1)

Top Labels

Issue Labels

enhancement (2) bug (1)

Pull Request Labels

Dependencies

requirements/dev.txt pypi

black ==20.8b1 development
km3net-testdata >=0.2.22 development
mock * development
numpydoc ==0.9.2 development
pillow * development
pydocstyle * development
pytest * development
pytest-cov * development
pytest-flake8 * development
pytest-pylint * development
pytest-watch * development
sphinx >=1.6.3 development
sphinx-autoapi * development
sphinx-gallery >=0.1.12 development
sphinx-rtd-theme >=0.3 development
sphinxcontrib-napoleon >=0.6.1 development
sphinxcontrib-programoutput >=0.11 development
sphinxcontrib-websupport >=1.0.1 development
twine * development
wheel * development

requirements/extras.txt pypi

ipykernel *
km3astro *
km3flux *
pandas *
scipy >=1.3.1
seaborn *

requirements/install.txt pypi

awkward >=1
docopt *
h5py *
km3db >=0.6.0
km3io >=0.20.0
matplotlib >=3.0.0
numba >=0.49.0
numpy >=1.17.0
particle >=0.8.0
passlib *
requests *
setuptools >=40.6.2
setuptools_scm *
tables >=3.4.2
thepipe >=1.3.5
toml *
tqdm *
urwid *
vg *

ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Open Source Science

km3pipe

Science Score: 59.0%

Repository

Basic Info

Statistics

Metadata Files

README.rst

Owner

CodeMeta (codemeta.json)

GitHub Events

Total

Last Year

Committers

All Time

Past Year

Top Committers

Committer Domains (Top 20 + Academic)

Issues and Pull Requests

All Time

Past Year

Top Authors

Issue Authors

Pull Request Authors

Top Labels

Issue Labels

Pull Request Labels

Dependencies