lstmcpipe

Scripts to analyse MC files on LST cluster at La Palma

https://github.com/cta-observatory/lstmcpipe

Keywords

Keywords from Contributors

Repository

Scripts to analyse MC files on LST cluster at La Palma

README.rst

lstMCpipe
=========

|code| |documentation| |slack| |CI| |coverage| |conda| |pypi| |zenodo| |fair| |fairsoftwarechecklist|

.. |code| image:: https://img.shields.io/badge/lstmcpipe-code-green
  :target: https://github.com/cta-observatory/lstmcpipe/
.. |CI| image:: https://github.com/cta-observatory/lstmcpipe/workflows/CI/badge.svg?branch=master
  :target: https://github.com/cta-observatory/lstmcpipe/actions?query=workflow%3ACI
.. |coverage| image:: https://codecov.io/gh/cta-observatory/lstmcpipe/branch/master/graph/badge.svg
  :target: https://codecov.io/gh/cta-observatory/lstmcpipe
.. |documentation| image:: https://img.shields.io/badge/lstmcpipe-documentation-orange
  :target: https://cta-observatory.github.io/lstmcpipe/
.. |conda| image:: https://img.shields.io/conda/v/conda-forge/lstmcpipe
   :alt: Conda
.. |pypi| image:: https://badge.fury.io/py/lstmcpipe.svg
  :target: https://badge.fury.io/py/lstmcpipe
.. |zenodo| image:: https://zenodo.org/badge/DOI/10.5281/zenodo.6460727.svg
  :target: https://doi.org/10.5281/zenodo.6460727
.. |fair| image:: https://img.shields.io/badge/fair--software.eu-%E2%97%8F%20%20%E2%97%8F%20%20%E2%97%8F%20%20%E2%97%8F%20%20%E2%97%8B-yellow
   :target: https://fair-software.eu
.. |slack| image:: https://img.shields.io/badge/CTA_North_slack-lstmcpipe_prods_channel-darkgreen?logo=slack&link=https%3A%2F%2Fcta-north.slack.com%2Farchives%2FC035H3C2HAS
   :alt: Static Badge
.. |fairsoftwarechecklist| image:: https://fairsoftwarechecklist.net/badge.svg
   :target: https://fairsoftwarechecklist.net/v0.2?f=21&a=32113&i=32300&r=133
   :alt: FAIR checklist badge



Scripts to ease the reduction of MC data on the LST cluster at La Palma.
With this package, the analysis/creation of R1/DL0/DL1/DL2/IRFs can be orchestrated.

Contact:
Thomas Vuillaume, thomas.vuillaume [at] lapp.in2p3.fr
Enrique Garcia, garcia [at] lapp.in2p3.fr
Lukas Nickel, lukas.nickel [at] tu-dortmund.de


Cite us 📝
----------

If lstMCpipe was used for your analysis, please cite:

https://doi.org/10.48550/arXiv.2212.00120

.. code-block::

  @misc{garcia2022lstmcpipe,
        title={The lstMCpipe library},
        author={Enrique Garcia and Thomas Vuillaume and Lukas Nickel},
        year={2022},
        eprint={2212.00120},
        archivePrefix={arXiv},
        primaryClass={astro-ph.IM}
  }

in addition to the exact lstMCpipe version used from https://doi.org/10.5281/zenodo.6460727


You may also want to include the config file with your published code for reproducibility.


Install 💻
----------

**As an user:**

For lstmcpipe >= 0.10.3, the preferred installation should be conda:

.. code-block::

    conda install lstmcpipe


Former versions:

.. code-block::

    VERSION=0.10.1  # change as desired
    wget https://raw.githubusercontent.com/cta-observatory/lstmcpipe/$VERSION/environment.yml
    conda env create -f environment.yml
    conda activate lstmcpipe
    pip install lstmcpipe==$VERSION


**As a developer:**

.. code-block::

    git clone https://github.com/cta-observatory/lstmcpipe.git
    cd lstmcpipe
    conda env create -n lstmcpipe_dev -f environment.yml
    conda activate lstmcpipe_dev
    pip install -e .
    pre-commit install

This will setup a pre-commit hook: Given that you are in the right enviroment, it will run and format files you are about
to commit with `black`. (You need to stage the changes again after that). This ensures the formatting of the
code follows our guidelines and there is less work dealing with the code checker in the CI.


Requesting a MC analysis 📊
---------------------------

You may find a longer, more detailed, version of these steps in our documentation.

You may find the list of already run productions in the documentation.
Please check in this list that the request you are about to make does not exist already!

To request a MC analysis:

#. Make sure to be part of the `github cta-observatory/lst-dev team `__. If not, ask one of the admins.
#. Clone the repository in the cluster at La Palma.
#. Create a new branch named with you ``prodID``
#. Make a new directory named ``date_ProdID`` in the `production_configs` dir (have a look at the ``production_configs/template_prod`` as an example)
#. Generate your config (see below)
#. Open a pull request into lstMCpipe with a clear description (probably the same as in the readme of your config dir)
#. The requested config must contain:

* a lstchain config file (please provide an exhaustive config that will help others and provide a more explicit provenance information)
* a lstmcpipe config file (to generate it, please refer to the documentation)
* a readme with a short description of why you require this analysis to be run. **Do not add information that should not appear publicly** (such as source names) here. If you are requesting a production for a specific new source, please edit `this table on LST wiki `_. Also add the command line to generate the lstmcpipe config, that will help debugging.

The proposed configuration will be tested for validity by the continuous integration tests and we will interact with you to run the analysis on the cluster at La Palma.

Depending on the number of requests, we may give priorities.

**Need help?**
Join the CTA North slack and ask for help in the |slack|

Launch jobs 🚀
--------------

To generate your lstmcpipe configuration file, use `lstmcpipe_generate_config` command.
If the type of production you want is not listed in the existing ones, you may create your own `PathConfig` class
from an existing one, or generate a config from an existing prod type and edit the file manually.

Once you have your configuration file, you way launch the pipeline with the described stages in the config using:

.. code-block:: python

    lstmcpipe -c config_MC_prod.yml -conf_lst lstchain_*.json [-conf_cta CONFIG_FILE_CTA] [-conf_rta CONFIG_FILE_RTA] [--debug] [--log-file LOG_FILE]

`lstmcpipe` is the **orchestrator** of the pipeline, it schedules the stages specified in the
``config_MC_prod.yml`` file. All the configuration related with the MC pipe must be declared in this file (stages,
particles to be analysed, zenith, pointing, type of MC production...).

Pipeline-specific configuration options (such as cleaning or model parameters) are declared in a different configuration file,
which is passed via the options ``-conf_lst/-conf_cta/-conf_rta``.

Note: You can always launch this command without fear; there is an intermediate step that verifies and
shows the configuration that you are passing to the pipeline.

Note that a complete pipeline still requires quite a lot of resources. Think about other LP-IT cluster users.

Owner

Citation (CITATION.cff)

cff-version: 1.2.0
title: lstmcpipe
type: software
authors:
- family-names: Vuillaume
  given-names: Thomas
  orcid: https://orcid.org/0000-0002-5686-2078
  email: thomas.vuillaume@lapp.in2p3.fr
- family-names: Garcia
  given-names: Enrique
  orcid: https://orcid.org/0000-0003-2224-4594
  email: enrique.garcia@lapp.in2p3.fr
- family-names: Nickel
  given-names: Lukas
  orcid: https://orcid.org/0000-0001-7110-0533
  email: lukas.nickel@tu-dortmund.de
preferred-citation:
  type: proceedings
  authors:
  - given-names: Enrique
    family-names: Garcia
  - given-names: Thomas
    family-names: Vuillaume
  - given-names: Lukas
    family-names: Nickel
  doi: "10.48550/arXiv.2212.00120"
  journal: "ADASS XXXII conference series"
  title: "The lstMCpipe library"
  year: 2022
message: If you use this software, please cite the version used from Zenodo (https://doi.org/10.5281/zenodo.6460727) as well as the paper describing the software (https://arxiv.org/abs/2212.00120).
license: MIT

CodeMeta (codemeta.json)

{
  "@context": "https://doi.org/10.5063/schema/codemeta-2.0",
  "@type": "SoftwareSourceCode",
  "license": "https://spdx.org/licenses/MIT",
  "codeRepository": "git+https://github.com/cta-observatory/lstmcpipe.git",
  "dateCreated": "2019-09-18",
  "datePublished": "2024-10-18",
  "downloadUrl": "https://github.com/cta-observatory/lstmcpipe/archive/refs/tags/v0.11.1.tar.gz",
  "issueTracker": "https://github.com/cta-observatory/lstmcpipe/issues",
  "name": "lstmcpipe",
  "version": "0.11.3",
  "softwareVersion": "0.11.3",
  "readme": "https://github.com/cta-observatory/lstmcpipe/blob/master/README.rst",
  "description": "Scripts to ease the reduction of MC data on the LST cluster at La Palma. With this package, the analysis/creation of R1/DL0/DL1/DL2/IRFs can be orchestrated.",
  "applicationCategory": "CTA, LST",
  "releaseNotes": "lstmcpipe v0.11.3 makes a hotfix about failed jobs directory. For bug fixes, see the **full changelog**: https://github.com/cta-observatory/lstmcpipe/compare/v0.11.2...v0.11.3",
  "funding": "824064",
  "developmentStatus": "active",
  "isPartOf": "https://www.cta-observatory.org/",
  "funder": {
    "@type": "Organization",
    "name": "European Unions Horizon 2020 research and innovation programme"
  },
  "keywords": [
    "workflow",
    "pipeline",
    "CTA",
    "ground-based telescopes"
  ],
  "programmingLanguage": [
    "Python 3"
  ],
  "author": [
    {
      "@type": "Person",
      "@id": "https://orcid.org/0000-0002-5686-2078",
      "givenName": "Thomas",
      "familyName": "Vuillaume",
      "email": "thomas.vuillaume@lapp.in2p3.fr",
      "affiliation": {
        "@type": "Organization",
        "name": "Univ. Savoie Mont-Blanc, CNRS, LAPP"
      }
    },
    {
      "@type": "Person",
      "@id": "https://orcid.org/0000-0003-2224-4594",
      "givenName": "Enrique",
      "familyName": "Garcia",
      "email": "enrique.garcia@lapp.in2p3.fr",
      "affiliation": {
        "@type": "Organization",
        "name": "Univ. Savoie Mont Blanc, CNRS, LAPP"
      }
    },
    {
      "@type": "Person",
      "@id": "https://orcid.org/0000-0001-7110-0533",
      "givenName": "Lukas",
      "familyName": "Nickel",
      "email": "lukas.nickel@tu-dortmund.de",
      "affiliation": {
        "@type": "Organization",
        "name": "TU Dortmund University, Department of Physics"
      }
    },
    {
      "@type": "Person",
      "@id": "https://orcid.org/0000-0001-8816-4920",
      "givenName": "Arnau",
      "familyName": "Aguasca-Cabot",
      "email": "arnau.aguasca@fqa.ub.edu",
      "affiliation": {
        "@type": "Organization",
        "name": "Departament de Física Quàntica i Astrofísica, Institut de Ciències del Cosmos (ICCUB), Universitat de Barcelona (IEEC-UB)"
      }
    }
  ],
  "maintainer": {
    "@type": "Person",
    "@id": "https://orcid.org/0000-0003-2224-4594",
    "givenName": "Enrique",
    "familyName": "Garcia",
    "email": "enrique.garcia@lapp.in2p3.fr",
    "affiliation": {
      "@type": "Organization",
      "name": "Univ. Savoie Mont Blanc, CNRS, LAPP"
    }
  }
}

GitHub Events

Committers

Last synced: almost 3 years ago

Issues and Pull Requests

Last synced: 6 months ago

Packages