PyAutoGalaxy - May 2025

• Results workflow API, which generates .csv, .png and .fits files of large libraries of results for quick and efficient inspection:

https://github.com/Jammy2211/autolens_workspace/tree/main/notebooks/results/workflow

• Visualization now outputs .fits files corresponding to each subplot, which more concisely contain all information of a fit and are used by the above workflow API.

• Visualization Simplified, removing customization of individual image outputs.

• Remove Analysis summing API, replacing all dataset combinations with AnalysisFactor and FactorGraphModel API used for graphical modeling:

https://github.com/Jammy2211/autolensworkspace/blob/main/notebooks/advanced/multi/modeling/starthere.ipynb

• Pixelized source reconstruction output as a .csv file which can be loaded and interpolated for better source science analysis.

• Latent variable API bug fixes and now used in some test example scripts.

Scientific Software - Peer-reviewed - Python
Published by Jammy2211 about 1 year ago

PyAutoGalaxy - January 2025

The main updates are visualization of Delaunay mesh's using Delaunah triangles and a significant refactoring of over sampling, with the primary motivation to make the code much less complex for the ongoing JAX implementation.

What's Changed

• Feature/disable noise by @Jammy2211 in https://github.com/Jammy2211/PyAutoGalaxy/pull/211
• feature/delaunay_visual by @Jammy2211 in https://github.com/Jammy2211/PyAutoGalaxy/pull/210
• feature/inversion noise map by @Jammy2211 in https://github.com/Jammy2211/PyAutoGalaxy/pull/212
• feature/operate deflections api by @rhayes777 in https://github.com/Jammy2211/PyAutoGalaxy/pull/195
• Revert "feature/operate deflections api" by @rhayes777 in https://github.com/Jammy2211/PyAutoGalaxy/pull/213
• Feature/over sampling refactor by @Jammy2211 in https://github.com/Jammy2211/PyAutoGalaxy/pull/214

Full Changelog: https://github.com/Jammy2211/PyAutoGalaxy/compare/2024.11.13.2...2025.1.18.7

Scientific Software - Peer-reviewed - Python
Published by Jammy2211 over 1 year ago

PyAutoGalaxy - November 2024 update

Small bug fixes and optimizations for Euclid lens modeling pipeline.

Scientific Software - Peer-reviewed - Python
Published by Jammy2211 over 1 year ago

PyAutoGalaxy - November 2024

Minor release with stability updates and one main feature.

• Extra Galaxies API for modeling multiple galaxies at once: https://github.com/Jammy2211/autogalaxyworkspace/blob/release/notebooks/features/extragalaxies.ipynb

Scientific Software - Peer-reviewed - Python
Published by Jammy2211 over 1 year ago

PyAutoGalaxy - September 2024

This release updates all projects to support Python 3.12, with support tested for Python 3.9 - 3.12 and 3.11 regarded as most stable.

This includes many project dependency updates:

https://github.com/rhayes777/PyAutoFit/blob/main/requirements.txt https://github.com/rhayes777/PyAutoFit/blob/main/optionalrequirements.txt https://github.com/Jammy2211/PyAutoGalaxy/blob/main/requirements.txt https://github.com/Jammy2211/PyAutoGalaxy/blob/main/optionalrequirements.txt

Workspace Restructure:

This release has a workspace restructure, which is now grouped at a high level by tasks (e.g. modeling, simulators) rather than datasets:

https://github.com/Jammy2211/autogalaxy_workspace

The readthedocs have been greatly simplified and include a new user guide to help navitgate the new workspace:

https://pyautogalaxy.readthedocs.io/en/latest/overview/overview2newuserguide.html

PyAutoGalaxy:

• Improved Cosmology wrapper to support new astropy and easier to use in models: https://github.com/Jammy2211/PyAutoGalaxy/pull/193
• Ellipse Fitting: https://github.com/Jammy2211/autogalaxy_workspace/tree/release/notebooks/advanced/misc/ellipse

PyAutoFit:

https://github.com/rhayes777/PyAutoFit/pulls?q=is%3Apr+is%3Aclosed

• Improvements to HowToFit lectures: https://github.com/rhayes777/PyAutoFit/pull/1022
• Support for NumPy arrays in model composition and prior creation, for example creating an ndarray of input shape where each value is a free parameter in the seach: https://github.com/rhayes777/PyAutoFit/pull/1021
• Name of optimize searches renamed to mle, for maximum likelihood estimator, with improvements to visualization: https://github.com/rhayes777/PyAutoFit/pull/1029
• Improvement to sensitivity mapping functionality and results: https://github.com/rhayes777/PyAutoFit/pulls?q=is%3Apr+is%3Aclosed
• More improvements to JAX Pytree interface, documentation still to come.

Scientific Software - Peer-reviewed - Python
Published by Jammy2211 over 1 year ago

PyAutoGalaxy - May 2024

PyAutoFit:

• Nautilus now outputs results on the fly: https://github.com/rhayes777/PyAutoFit/pull/961
• Output latent samples of a model-fit, which are parameters derived from a model which may be marginalized over:

PR: https://github.com/rhayes777/PyAutoFit/pull/994 Example: https://github.com/Jammy2211/autofit_workspace/blob/release/notebooks/cookbooks/analysis.ipynb

• model.info file displays complex models in a more concise and readable way: https://github.com/rhayes777/PyAutoFit/pull/1012
• All samples with a weight below an input value are now removed from samples.csv to save hard disk space: https://github.com/rhayes777/PyAutoFit/pull/979
• Documentation describing autofit scientific workflow: https://github.com/rhayes777/PyAutoFit/pull/1011
• Refactor visualization into stand alone module: https://github.com/rhayes777/PyAutoFit/pull/995
• Refactor how results are returned after a search: https://github.com/rhayes777/PyAutoFit/pull/989
• Improved parallelism logging: https://github.com/rhayes777/PyAutoFit/pull/1009
• Likelihood consistency check now performed internally: https://github.com/rhayes777/PyAutoFit/pull/987
• Generation of initial search samples is now performed in parallel: https://github.com/rhayes777/PyAutoFit/pull/997
• No longer store search_internal on hard-disk. simplifying source code internals: https://github.com/rhayes777/PyAutoFit/pull/938
• Multiple small bug fixes and improvements to interface.

PyAutoGalaxy:

• Remove Plane object and replace with Galaxies object
• Shapelets improvements: https://github.com/Jammy2211/PyAutoGalaxy/pull/173
• Adaptive over sampling of grids for a pixelization: https://github.com/Jammy2211/PyAutoGalaxy/pull/168
• BasisPlotter which plots each basis (e.g. each Gaussian of an MGE): https://github.com/Jammy2211/PyAutoGalaxy/pull/173
• Plot mappings between source and image plane of a pixelization as lines: https://github.com/Jammy2211/PyAutoGalaxy/pull/172
• For multi-wavelength datasets model offsets between each dataset: https://github.com/Jammy2211/PyAutoGalaxy/pull/171
• Modeling of background sky: https://github.com/Jammy2211/PyAutoGalaxy/pull/170
• Improvements to use of adapt images for adaptive pixelizations: https://github.com/Jammy2211/PyAutoGalaxy/pull/160
• Improved angle conversions for computing errors on mass profile and shear angles from ell_comps: https://github.com/Jammy2211/PyAutoGalaxy/pull/169
• Remove sub_size from all classes (e.g. Array2D, Mask2D) to simplify API.
• MaternKernel added: https://github.com/Jammy2211/PyAutoGalaxy/pull/148

Scientific Software - Peer-reviewed - Python
Published by Jammy2211 about 2 years ago

PyAutoGalaxy - January 2024 (2024.1.27.4)

• Log10 plots implemented in 1D and 2D, which show certain quantities (e.g. galaxy convergence) more clear and include contours showing log10 values:

• Improved subplots including addition of log10 panels:

• Pixelization API now has separate entry for an image_mesh, defining how the source pixel centres are computed (E.g. using a KMeans clustering) and the mesh is now just the method (e.g. Delaunay):

pixelization = al.Pixelization( image_mesh=al.image_mesh.Overlay(shape=(25, 25)), mesh=al.mesh.Delaunay(), regularization=al.reg.Constant(coefficient=1.0), )

• Implemented Hilbert image-mesh which is a significant improvement on KMeans clustering for creating the distribution of source pixels for a pixelization and inversion.

• Simplified adapt_dataset API to now only pass via an AdaptImage class, which is not passed as Galaxy attributes anymore but instead through the FitImaging object.

• Removed SetupAdapt object and pass image_mesh_pixels as an integer through pipelines.

• Added Exponential / Gaussian smoothing kernels for regularization documented in Vernardos 2022 (https://arxiv.org/abs/2202.09378)

Scientific Software - Peer-reviewed - Python
Published by Jammy2211 over 2 years ago

PyAutoGalaxy - October 2023 (2023.10.23.3)

• Support for Python 3.11 by updating requirement on core libraries (e.g. numpy, scipy, scikit-learn).
• Fix issues with sqlite database following switch from .pickle outputs to .json / .fits / .csv.
• Database use of Samples object much more efficient.
• Methods to output classes to hard-disk (e.g. output_to_json, from_json, to_dict) are now all handled and called from autoconf.
• Fix bug where nautilus parallel fits sometimes crashed.
• Fix bug where nautilus single CPU fits did not work.

Scientific Software - Peer-reviewed - Python
Published by Jammy2211 over 2 years ago

PyAutoGalaxy - September (v2023.9.18.4)

This release implements two major changes to PyAutoGalaxy:

Nautilus:

For the past ~3 years, model fitting has used the nested sampling algorithm Dynesty.

Recently, a new nested sampler, Nautilus (https://nautilus-sampler.readthedocs.io/en/stable/), was released, which uses machine-learning based techniques to improve sampling.

Extensive testing of modeling with Nautilus has revealed that it:

• Speeds up the fitting of simple models by ~x2 - x3.
• Speeds up the fitting of complex models by ~x3 - x5+.
• Is more robust and reliable (e.g less likely to infer a local maxima, can fit more complex lens models).
• Controlled predominantly by just one parameter n_live, so is simpler to use than dynesty.
• Parallelization using Python multiprocessing is more efficient than dynesty and now supports proper error handling.

Nautilus is therefore now the default modeler, with all workspace examples updated accordingly.

NOTE: Nautilus does not currently support on-the-fly output and to get the results of a lens model mid-fit a user can instead cancel the run (e.g. via Ctrl + C) and restart it, where the maximum likelihood model will be output.

Results Output

Result metadata was previously output as .pickle files, which were not human readable and depended on project imports, hurting backwards compatibility.

All metadata is now output as human readable .json files and dataset as .fits files, making it a lot more straight forward for a user to interpret how data is stored internally within PyAutoGalaxy:

Here is an example of the search.json file:

All internal functionality (e.g. the sqlite database) has been updated to use these files.

All workspace documentation has been updated accordingly.

Other:

• imaging/modeling/features split to make linear light profiles and multi gaussian expansion more visible.
• Improved HowToGalaxy tutorial 5 on linear light profiles.
• Power law with multipole parameterization updated, now supports multipoles of any order (https://github.com/Jammy2211/PyAutoGalaxy/pull/115).
• Update certain requirements (e.g. PyYAML) to mitigate installation issues (https://github.com/rhayes777/PyAutoConf/pull/41).
• Lots of quality-of-life improvements thoughout the code bases.

Scientific Software - Peer-reviewed - Python
Published by Jammy2211 over 2 years ago

PyAutoGalaxy - July (2023.5.7.2)

Bug fixes for new MacOS parallelization.

No new features.

Scientific Software - Peer-reviewed - Python
Published by Jammy2211 almost 3 years ago

PyAutoGalaxy - June 2023 (2023.6.18.3)

• Fixes bug so that the all_at_end_png and all_at_end_fits visualization configuration options now actually do output all images at the end of a model-fit as .png and .fits files.

• Fixes bug so that pixelized source reconstructions are output as .fits files at the end.

• Fixes bug so that visuals at end display correctly.

Scientific Software - Peer-reviewed - Python
Published by Jammy2211 almost 3 years ago

PyAutoGalaxy - June 2023 (2023.6.12.5)

• Visualization now outputs publication quality plots by default (e.g. less whitespace, bigger tick labels, units):

https://github.com/Jammy2211/PyAutoGalaxy/pull/96

• Improved visualization of FitImaging and FitInterferometer subpots:

https://github.com/Jammy2211/PyAutoGalaxy/pull/96

• Profiling tools implemented, will soon be added to workspace scripts:

https://github.com/Jammy2211/PyAutoGalaxy/pull/110

• PowerLawMultipole method generalized to all multipoles:

https://github.com/Jammy2211/PyAutoGalaxy/pull/103

• Critical Curves / Caustic plotter separating if there are more than one, and options to customize tangential and radial separately:

https://github.com/Jammy2211/PyAutoGalaxy/pull/92

• SMBH and SMBHBinary super massive black hole mass profiles implemented:

https://github.com/Jammy2211/PyAutoGalaxy/pull/98 https://github.com/Jammy2211/PyAutoGalaxy/pull/99

• Fix issues associated with visualization of linear light profiles and Basis objects:

https://github.com/Jammy2211/PyAutoGalaxy/pull/102

• PowerLaw potential_2d_from method faster:

https://github.com/Jammy2211/PyAutoGalaxy/pull/108

• ExternalShear now has potential_2d_from method implemented:

https://github.com/Jammy2211/PyAutoGalaxy/pull/109

• Removal of a number of unused legacy features (e.g. hyper galaxy noise scaling).

Scientific Software - Peer-reviewed - Python
Published by Jammy2211 almost 3 years ago

PyAutoGalaxy - March 2023 (2023.3.27.1)

• Shapelets basis functions for representing the emission of a galaxy implemented (see also https://github.com/Jammy2211/autogalaxy_workspace/blob/release/notebooks/imaging/modeling/features/shapelets.ipynb)

• Critical curves and caustics of mass profile projects are now computed such that if there are multiple curves, all are computed and returned as a list

• Updated critical curve calculations also mean Einstein Radii, Einstein Masses are other related quantities can be returned as a list of values (one for each curve) or an average

• Added matplotlib wrapper to the annotate function.

Scientific Software - Peer-reviewed - Python
Published by Jammy2211 about 3 years ago

PyAutoGalaxy - March 2023 (2023.3.21.5)

This is the latest version, which primarily brings in stability upgrades and fixes bugs.

Scientific Software - Peer-reviewed - Python
Published by Jammy2211 about 3 years ago

PyAutoGalaxy - January 2023 (JOSS)

This is a major release, which is tied to the publication of PyAutoGalaxy in the Journal of Open Source software (JOSS).

This release updates many aspects of the API, switches configuration files to YAML, updates library requirements and adds new functionality.

API Changes:

• All elliptical light profiles and mass profiles no longer prefix with the Ell tag, for conciseness / readability. For example, EllSersic is now just Sersic, and EllIsothermal is now Isothermal.
• The Sph prefix is now a suffix, for example SphSersic is now SersicSph and SphIsothermal is now Isothermal.
• The `elliptical_components parameter has been shorted to ell_comps.
• The ExternalShear input has been changed from elliptical_components to gamma_1 and gamma_2 (the shear is still defined the same, where in the olversion version elliptical_components[0] = gamma_2 and elliptical_components[1] = gamma_1.
• The manual_ API for data structures (e.g. Array2D, Grid2D) has been removed.

Yaml Configs

• Configuration files now support .yaml, which is provided with the autolensworkspace (https://github.com/Jammy2211/autogalaxyworkspace/tree/release/config).
• The workspace configuration files are now fully documented,.

Linear Light Profiles / Basis / Multi Gaussian Expansion

Linear light profiles are now supported, which are identical to ordinary light profiles but the intensity parameter is solved for via linear algebra. This means lower dimensionality models can be fitted, making dynesty converge more reliably:

https://github.com/Jammy2211/autogalaxyworkspace/blob/release/scripts/imaging/modeling/lightparametriclinearmasstotal_sourceparametric_linear.py

Fits use a Basis object composed of many linear light profiles are supports, for example using a Multi Gaussian Expansion of 20+ Gaussians to fit the lens's light:

https://github.com/Jammy2211/autogalaxyworkspace/blob/release/scripts/imaging/modeling/lightparametriclinearmasstotal_sourceparametric_linear.py

These features are described fully in the following HowToGalaxy tutorial:

https://github.com/Jammy2211/autogalaxyworkspace/blob/release/scripts/howtogalaxy/chapter2modeling/tutorial5linearprofiles.py

API Documentation

API documentation on readthedocs is now being written, which is still a work in progress but more useable than it was previously (https://pyautogalaxy.readthedocs.io/en/latest/api/data.html).

Requirements

The requirements of many projects have been updated to their latest versions, most notably dynesty v2.0.2.

Scientific Software - Peer-reviewed - Python
Published by Jammy2211 over 3 years ago

PyAutoGalaxy - July 07 2022 Release

• autogalaxy_workspace now has advanced packages which make navigation simpler for new users to find beginner scritps.

• LightProfileOperated objects implemented, which are already convolved with the imaging dataset's PSF for modeling point source components in a galaxy (see https://github.com/Jammy2211/autogalaxyworkspace/blob/release/scripts/imaging/modeling/advanced/lightparametric_operated.py).

• Numba is now an optional installation, see this doc page for a full description (https://pyautogalaxy.readthedocs.io/en/latest/installation/numba.html).

• Starting point API for starting an MCMC fit with walkers in certain positions or maximum likelihood estimator fit with a start point implemented (https://github.com/rhayes777/PyAutoFit/pull/562). The example tutorial script for this feature is not written yet.

Scientific Software - Peer-reviewed - Python
Published by Jammy2211 almost 4 years ago

PyAutoGalaxy -

This is the first major release of PyAutoGalaxy, with full documentation of the software package.

Checkout the readthedocs and workspace for a complete overview of the package.

Scientific Software - Peer-reviewed - Python
Published by Jammy2211 about 4 years ago