SigCorr

SigCorr: A Python package for studies of trials factors - Published in JOSS (2023)

https://gitlab.com/sigcorr/sigcorr

Scientific Fields

Repository

https://gitlab.com/sigcorr/sigcorr/blob/master/

[![Documentation for the latest stable version](https://img.shields.io/badge/docs-dev-blue)](https://sigcorr.gitlab.io/sigcorr/latest/)
[![DOI](https://joss.theoj.org/papers/10.21105/joss.04989/status.svg)](https://doi.org/10.21105/joss.04989)
[![License Apache 2](https://img.shields.io/badge/license-apache--2-green)](https://www.apache.org/licenses/LICENSE-2.0)

# SigCorr

A framework to study the trials factor



    
    

    Alternative ways to estimate trials factor
    



In high-energy physics it is a recurring challenge to efficiently and precisely (enough) calculate
the global significance of, e.g., a potential new resonance.
The Gross and Vitells trials factor approximation
\[[1](https://doi.org/10.1140/epjc/s10052-010-1470-8), [2](https://doi.org/10.1016/j.astropartphys.2011.08.005)\]
is based on the average up-crossings of the significance in the search region,
or generally on the average Euler number of the set of significance measurements that exceed the threshold
of the local significance.
It has revolutionized the trials factor estimation for significances above 3 standard deviations,
but the challenges of actually calculating the average up-crossings and the validity of
the approximation for smaller significances remain.

In Ananiev&Read \[[3](https://arxiv.org/abs/2206.12328)\] a new method was proposed.
It models the significance in the search region as a Gaussian Process (GP).
The method was developed to overcome the limitations of the Gross and Vitells approach
via replacing expensive MC fits with lightweight GP toys.

Up-crossings, Euler numbers and Gaussian processes are commonly relied on in this field of research.
When studied together, they have many useful properties for the estimation of trials factors.
*SigCorr* is the first project that assembles all of them into one Python package, that also includes
the tools for parallel analysis of the MC toys and GP toys in a unified fashion.
Together with the fitting framework it allows the path
from the statistical model definition to a TF estimate to be travelled.

While *SigCorr* was developed with flexibility in mind,
in order to incorportate various approaches from the figure above,
it was architectured in a way that allows it to be used right away.

## Installation

```
pip install git+https://gitlab.com/sigcorr/sigcorr.git#egg=sigcorr
```

Find more detailed instructions in [the documentation](https://sigcorr.gitlab.io/sigcorr/dev/).

## Running

Source env vars to force JAX use float64 precision and disable its warnings:
```
source env.sh
```

```
sigcorr-run -o output_file.h5 -n 1000 GrossVitellsAsimov
```

Real world run (Hyy):

```
SIGCORR_FITTER_sbfit_batchsize=5 SIGCORR_FITTER_bfit_batchsize=300 SIGCORR_FITTER_sbfit_pool_size=250 SIGCORR_FITTER_bfit_pool_size=70 sigcorr-run -g sigcorr/grids/hyy-common.dat -o output/hyy-1m.h5 -f -n1000000 Hyy
```

Real world run (Gross and Vitells):

```
SIGCORR_FITTER_sbfit_batchsize=5 SIGCORR_FITTER_bfit_batchsize=300 SIGCORR_FITTER_sbfit_pool_size=250 SIGCORR_FITTER_bfit_pool_size=70 sigcorr-run -g sigcorr/grids/gross_vitells-sampling.dat -s sigcorr/grids/gross_vitells-scan.dat -o output/gross_vitells200k-1m.h5 -f -n1000000 GrossVitells
```

## Notebooks

Install dependencies

```
pip install -r notebooks/requirements.txt
```

Run jupyter lab:

```
jupyter lab notebooks/
```

## Build docs

For building docs, SigCorr should be installed (see the Installation above). Then run sphinx:

```
pip install sphinx matplotlib numpydoc sphinx-rtd-theme
./scripts/build-docs.sh
```


## Tests

Documentation includes a variety of code snippets,
both explicit and the ones used implicitly to produce figures for the docs.

They cover a major part of the functionality, and run everytime the docs are built (see previous section).

We treat documentation build similarly to unit tests, and run it in CI for each merge request.

The subdir [tests/](/tests) covers evaluation of some mathematical concepts and numerical methods.
We wrote them to convince ourselves that some of our analytical expressions match with numerical calculations.


## Communication

* If you want to contribute, feel free to fork and open a Merge request
* If you seek support or want to report bug, please open a new [Issue](https://gitlab.com/sigcorr/sigcorr/-/issues)


## Cite this package

```
@article{Ananiev2023,
    doi = {10.21105/joss.04989},
    url = {https://doi.org/10.21105/joss.04989},
    year = {2023},
    publisher = {The Open Journal},
    volume = {8},
    number = {87},
    pages = {4989},
    author = {V. Ananiev and A. L. Read},
    title = {SigCorr: A Python package for studies of trials factors},
    journal = {Journal of Open Source Software}
}
```

Owner

JOSS Publication

SigCorr: A Python package for studies of trials factors

Published

July 07, 2023

DOI

10.21105/joss.04989

Volume 8, Issue 87, Page 4989

Authors

V. Ananiev
Department of Physics, University of Oslo, Boks 1072 Blindern, Oslo, NO-0316, Norway

A. L. Read
Department of Physics, University of Oslo, Boks 1072 Blindern, Oslo, NO-0316, Norway

Editor

Øystein Sørensen

Tags

python statistics look-elsewhere effect lee trials factor statistical significance gaussian process resonance search

View PDF Review Thread Software Archive

Committers

Last synced: 6 months ago

Issues and Pull Requests

Last synced: 6 months ago