phrosty

phrosty (PHotometry for ROman with SFFT for tYpe Ia supernovae)

https://github.com/roman-supernova-pit/phrosty

Repository

phrosty (PHotometry for ROman with SFFT for tYpe Ia supernovae)

README.rst

Difference imaging forced photometry pipeline
=============================================

phrosty: **PHotometry for ROman with SFFT for tYpe Ia supernovae**

Aldoroty, L.,  *et al.*, 2025, in prep

This is lightcurve software being developed for the Roman Supernova Project Infrastructure Team (SNPIT).  It is designed to take a sequence of Roman images, divided into "template" and "science" images.  It subtracts each template image from each science image.  It then measures the flux in a point source at a specified RA/Dec on each difference image.  A combination of all of the fluxes measured for a given science image provide an estimate of the flux at the MJD of that science image.  All of these fluxes together provide the lightcurve.

Currently, *phrosty* only handles the OpenUniverse sims [ref needed].  It reads images from those sims in FITS format, using the standard FITS WCSes found in the headers of those files for image alignment.  The galsim package and a particular version of the roman imsim package (https://github.com/matroxel/roman_imsim.git) provide the PSF used both for PSF photometry and for the PSF matching in the SFFT image subtraction software (Hu *et al.*, 2022, ApJ, 936, 157).

Statement of Need
-----------------
Difference imaging analysis (DIA) is a technique in transient analysis that allows both transient detection and photometric measurements. Broadly, this method involves subtracting two images, one with a transient and a "reference" image without a transient, to isolate the object of interest. 

The Nancy Grace Roman Space Telescope (_Roman_) is NASA’s first survey-oriented flagship mission. Its design is optimized for cosmological studies, which includes extraction and precision analysis of Type Ia Supernova (SN Ia) light curves. Analysis of these data pose several challenges: (1) the enormous quantity of data that will be returned on a regular basis, and (2) the spatially-dependent nature of the image properties as a function of location on the detector, which directly affects ease of (3) obtaining the required <1\% flux precision on SN Ia measurements.

_Roman_'s High Latitude Time Domain Survey (HLTDS) is specifically designed for cosmological studies with SNe Ia. The core component of this survey will last two years, and have a cumulative 158 days of exposure time. The survey will yield an average of 241 Level 2 (rate files) from imaging mode observations per day, which totals to approximately 0.4 TB of data. Other DIA and forced photometry survey pipelines are insufficient to process this quantity of data. For example, the Dark Energy Survey (DES) pipeline, which uses the HOTPANTS algorithm, takes 10 minutes to create a difference image from an image from a single detector, with 8.3 million pixels per detector. Using this pipeline, it would take 80 hours to make difference images from all 241 _Roman_ HLTDS images obtained within one 24 hour period, which have nearly 17 million pixels per detector. In reality, the amount of time spent processing these data will be much more than 80 hours because (1) there are many steps associated with SN Ia light curve analysis in addition to creating the difference images, (2) data from other concurrent surveys will also be processed for transient studies, and (3) this amount of time also excludes re-processing data when pipelines or calibration information are updated.

`phrosty` is a DIA and forced photometry pipeline that addresses these challenges. We integrate the saccadic fast fourier transform (SFFT) method for difference imaging analysis (DIA), which uses a flexible delta basis function that can accommodate Roman’s spatially-varying point spread function (PSF). `phrosty` uses both GPU and CPU computation; this architecture has been carefully chosen to address the challenge of processing the large quantity of survey data within a reasonable amount of time. 

Because Roman software is currently under-development, there is no similar publicly available all-in-one pipeline that takes _Roman_ images as input and outputs light curves. The architecture choices made in development have substantially accelerated processing time such that `phrosty` is among the fastest astronomical DIA pipelines available. 

Prerequisites and Environment
-----------------------------

Although SFFT works in both CPU and GPU environments, currently some of the code in *phrosty* requires a CUDA-based GPU.  It requires a machine with an NVidia GPU that has at least 20MB (or more) of GPU RAM.  The Perlmutter GPU nodes at NERSC (with 40GB of GPU RAM) meet this requirement, but consumer graphics cards with only 12GB of RAM aren't sufficient.

A number of standard python libraries are required, all of which may be installed from pip, including ``numpy``, ``scipy``, ``astropy``, ``scikit-image``, ``cupy-cuda12x``, ``pandas``, ``requests``, ``fitsio``, ``pyarrow``, ``fastparquet``, and (maybe?) ``nvmath-python[cu12]``.  *Phrosty* also requires ``galsim``, which may be installed from pip.

*phrosty* requires commit the ``roman_imsim`` package, which may be git cloned from ``https://github.com/matroxel/roman_imsim.git``.  As of this writing, we use the ``main`` branch of ``roman_imsim``.  Hopefully it will not evolve in a way that breaks what we do....

*phrosty* should run in the `roman snpit environment `_.  In particular, it should run in the docker image created from that environment.  That environment is curretly heavily under development (just like *phrosty*), so exactly which version of the environment works will vary with time.  As of this writing, you can pull the latest version of the docker image from either of:

* ``rknop/roman-snpit-env:cuda-dev``
* ``registry.nersc.gov/m4385/rknop/roman-snpit-env:cuda-dev``

(For an example of actually running phrosty in this dockerized environment, see ``examples/perlmutter/README.md``.)

Manual installs
---------------

Not included in the docker image described above is SFFT— because, as of this writing, the version used with phrosty was under development.  SFFT must be separately cloned from git@github.com:Roman-Supernova-PIT/sfft.git, the directory to which you clone that archive must by in your ``PYTHONPATH``.  As of this writing, *phrosty* requires the ``fixes_20241022`` branch, though hopefully we will get that branch merged back to the ``master`` branch.  See "Examples" below for samples of this in action.

Necessary directories data files
--------------------------------

Currently, *phrosty* depends on the following environment variables to find data:

* ``SIMS_DIR`` : OpenUniverse roman data (on nersc: ``/dvs_ro/cfs/cdirs/lsst/shared/external/roman-desc-sims/Roman_data``)
* ``SN_INFO_DIR`` : A directory that holds information about the OpenUniverse sims, and transients from the OpenUniverse sims.  It must have a file ``tds.yaml`` that is copied from ``$SIMS_DIR/RomanTDS/tds.yaml`` and properly modified for whatever environment you're running (mostly by fixing paths).
* ``SNANA_PQ_DIR`` : A directory with the transient parquet files from the OpenUniverse sims (on nersc: /dvs_ro/cfs/cdirs/lsst/www/DESC_TD_PUBLIC/Roman+DESC/PQ+HDF5_ROMAN+LSST_LARGE)
* ``DIA_OUT_DIR`` : A directory where difference images (and psfs and other associated images) will be written.


Running the code
----------------

Please see the [documentation](https://roman-supernova-pit.github.io/phrosty/).

Examples
--------

Currently, these examples are intended for members of the Roman SN PIT.  Anybody may try them, but since we're early in development, we can't support non-PIT members yet.  Members of the PIT, if you have any issues running phrosty, ping Rob Knop and Lauren Aldoroty in the ``#photometry`` channel on the SN PIT Slack.

NERSC/Perlmutter
****************

An example running *phrosty* on an interactive node, and on a slurm-allocated node, on Perlmutter at NERSC may be found in the examples/perlmutter directory.

Running tests
-------------

The tests need to be run within an environment where *phrosty* will run properly.  At the moment, the only environment we've successfully used this with is the Docker environment described in the NERSC/Perlmutter environment above.  Follow that example through running ``bash interactive_podman.sh``.  Then, instead of doing what the example says, run::

  cd /home/phrosty/tests
  export DIA_OUT_DIR=../../dia_out_dir
  export SN_INFO_DIR=../../sn_info_dir
  pytest -v

TODO: we need to clean this up and make the test environment more self-contained.

Plotting lightcurves
--------------------

TODO.

For now, you can use your favorite plotting program and just read the `.csv` files produced by the pipeline.  However, we need to document how you read in the OpenUniverse truth files for plotting lightcurves against the truth files.

License
=======

This project is Copyright (c) Lauren Aldoroty and licensed under
the terms of the BSD-3Clause license. This package is based upon
the `Roman Supernova PIT packaging guide `_
which is licensed under the BSD 3-clause licence. See the licenses folder for
more information.

Owner

Citation (CITATION.cff)

cff-version: 1.2.0
message: "If you use this software in your work, please cite it using the following metadata."
authors:
- name: "Lauren Aldoroty et al."
  orcid: "https://orcid.org/0000-0003-0183-451X"
title: phrosty
version: 0.0
doi: TBD
date-released: 
url: "https://github.com/Roman-Supernova-PIT/phrosty"

GitHub Events

Issues and Pull Requests

Last synced: 6 months ago