Atmospheric Tomography with 3D Radiative Transfer (AT3D)

AT3D performs 3D reconstruction of cloud/aerosol microphysical properties from multi-angle, multi-spectral solar reflected radiation using a non-linear optimization procedure [1,2,3]. The core radiative transfer routines are sourced from the Fortran SHDOM (Spherical Harmonic Discrete Ordinate Method for 3D Atmospheric Radiative Transfer) code by Frank Evans [4,5].

The python package was created by Aviad Levis, Amit Aides (Technion - Israel Institute of Technology) and Jesse Loveridge (University of Illinois). Code contributions have been made so far by Linda Forster and Vadim Holodovsky.

Usage

The AT3D software is built around SHDOM, which is freely distributed online. Please contact Frank Evans if you have concerns about the licensing of his code as it appears in this package. This package (AT3D) is distributed under the GNU General Public License (see the LICENSE file). If you want to acknowledge the use of this repository in a publication (e.g. scientific journal article), then please cite the appropriate release, or the most recent release, which is available at the following DOI. See the CITATION.cff file for how to reference this repository.

If you want to acknowledge the scientific origin of a particular feature of this software in a publication, then please cite the appropriate journal or conference articles in which the feature originates [1,2,3]. If you use a feature of this repository that is not published elsewhere but is scientifically important to your study then please reach out to discuss the possibility of co-authorship. See the Contact section below.

This work relies on the generosity of Frank Evans in making his code publicly available, for which we are very grateful. Please acknowledge his work appropriately. In particular, use of the SHDOM solver as a part of the AT3D software in a scientific work should cite [5].

Any publications using the synthetic les clouds in the ./data/syntheticcloudfields/jpl_les directory which is included in the distribution must cite the following work [7].

Contact

If you find this package useful and/or would like to contribute code please let us know: aviad.levis@gmail.com; jesserl2@illinois.edu.

Features

Forward (RTE solver):

AT3D is a python wrapper for polarized SHDOM and can be used to compute radiative quantities for a variety of atmospheric configurations. The key features of polarized SHDOM are included 1. Solar/Thermal/Combined sources 2. A variety of (spatially variable) surface BRDFs 3. Vector or scalar radiative transfer. 4. Open or periodic boundary conditions.

Note that each RTE solution is serial (unlike SHDOM) but independent wavelengths and pixel radiance calculations are parallelized using either MPI or a multi-threading shared memory framework. Other key features that are implemented are: * Several sensor configurations (e.g. Perspective, Orthographic) and arbitrary observation geometries. * Mie & Rayleigh scattering optical property calculations including OPAC aerosols. Optical properties of other species (e.g. non-spherical ice or aerosol or absorbing gases) can be included but must be calculated externally. * Microphysical/optical properties can be generated or be read from netCDF or the SHDOM/I3RC file format.

Inverse (remote-sensing):

AT3D recovers microphysical/optical properties of atmospheric constituents that fit measured radiances. In contrast to most availble codes, AT3D can recover 3D variable atmospheric properties. This is achieved by local optimization procedures which employ an approximation to the Frechet derivatives of the RTE developed by Levis et al. [3].

Future Improvements

Future improvement include: * Add additional sensor geometries (cross-track scan, push-broom). * Parallelize RTE solution with MPI. * Include retrieval of surface BRDF.

To contribute to the development effort, contact us! see Contact section above.

Updates in AT3D 4.0

• Data is represented using xarray objects.
• Wide field of fiew radiances are now modeled.

Installation

Compilation of this package requires Fortran & C compilers (e.g. GCC 9.3.0_1) to be available and correctly linked. Installation has been tested on Mac and Linux using using anaconda package management.

The treatment of legacy Fortran code has changed from GCC 9.X to 10.X+ so currently there is a flag in the setup.py script which needs to be commented if trying to install using GCC 9.X or earlier versions. The flag is extra_f77_compile_args=["-fallow-argument-mismatch"]. There is additional discussion of this point written as comments in setup.py but please feel raise an issue or discussion if you run into issues with the compiler version. The default version of the package should compile with GCC v11.3.

Clone the repository into your local machine git clone https://github.com/CloudTomography/AT3D.git cd AT3D

Start a clean virtual environment and setup environment variables conda create -n at3d python=3.10.4 conda activate at3d

Install required packages pip install -r requirements.txt

Install AT3D distribution. This should be run from within the folder containing setup.py. For development mode add the flag -e. pip install .

Running Tests

After successful installation, run the tests using Python's nosetests package and make sure they all succeed: cd tests nose2 -v This command will execute all files starting with test_*.py. To execute only one specific test file, test.py use nose2 -v test

Don't panic if the tests involving the SHDOM solver has an error (VerifySolver.testsolver). This test uses a static reference generated on a particular computer with a particular compiler. Due to the use of the adaptive grid, small numerical differences due to differences in computer systems may be amplified to a level that tests can fail. Run the test again with the verbose option (nose2 test_shdom -vv) and check to see whether the number of grid points are the same between the reference and the test.

If they are the same but the test still fails then there is problem and it might be worth regenerating the static reference on your own machine using the original SHDOM code. The input data are supplied in the tests/data/ folder as shdom_verification_polarized. The test uses custom output from SHDOM which requires modifying the original distribution. If you have trouble reproducing this test and have some questions then please raise an issue on GitHub or email me (jesserl2@illinois.edu).

Basic usage

For basic usage follow the following jupyter notebook tutorials under the notebooks directory:

• MakeMieTablesExample
• MakePolydisperseMie
• MakeOpticalProperties
• MakeSensors
• SolveRTE
• SimulatingRadiances
• SimpleInverseProblem

Main scripts

For generating rendering and optimization scripts see the list below. The scripts folder contains another readme file with examples of how to run each script. TODO