deepsensor - v0.4.2

• Minor fix to avoid pandas 3.0 FutureWarning

Full Changelog: https://github.com/alan-turing-institute/deepsensor/compare/v0.4.1...v0.4.2

- Python
Published by tom-andersson over 1 year ago

deepsensor - v0.4.1

• Rearrange forecast dimensions from (leadtime, inittime) to (inittime, leadtime)
• Raise error about deepsensor.plot.prediction not supporting forecasts

Full Changelog: https://github.com/alan-turing-institute/deepsensor/compare/v0.4.0...v0.4.1

- Python
Published by tom-andersson over 1 year ago

deepsensor - v0.4.0

What's Changed

• Fix forecasting lead times and improve forecasting functionality by @tom-andersson in https://github.com/alan-turing-institute/deepsensor/pull/132
• Fix astype in coordinates matching when sampling data frames by @acocac in https://github.com/alan-turing-institute/deepsensor/pull/126

Breaking changes

• model.predict for models trained with target_delta_t > 0 now return Prediction objects with init_time and lead_time dimensions with a time (valid time) coordinate, rather than a single time dimension (which used to correspond to forecast initialisation time).

Full Changelog: https://github.com/alan-turing-institute/deepsensor/compare/v0.3.8...v0.4.0

- Python
Published by tom-andersson over 1 year ago

deepsensor - v0.3.8

What's changed

• Add new DataProcessor normalisation method (`"positive_semidefinite") compatible with a Bernoulli-Gamma ConvCNP likelihood
• Expose k and scale parameters of "bernoulli-gamma" ConvNP likelihood in low-level and high-level prediction interface

Full Changelog: https://github.com/alan-turing-institute/deepsensor/compare/v0.3.7...v0.3.8

- Python
Published by tom-andersson over 1 year ago

deepsensor - v0.3.7

What's Changed

• Return active learning proposed placements with raw coordinate names by @RohitRathore1 in https://github.com/alan-turing-institute/deepsensor/pull/99
• Make use of multiprocessing.Pool in get_era5_reanalysis_data optional in https://github.com/alan-turing-institute/deepsensor/commit/b9594a24114058ad443639432a3f9ca4b99c76a5
• Fix python version in tests (plus additional minor fixes) by @davidwilby in https://github.com/alan-turing-institute/deepsensor/pull/116
• use unittest's setUpClass instead of overriding __init__ by @davidwilby in https://github.com/alan-turing-institute/deepsensor/pull/117

Full Changelog: https://github.com/alan-turing-institute/deepsensor/compare/v0.3.6...v0.3.7

- Python
Published by tom-andersson over 1 year ago

deepsensor - v0.3.6

What's Changed

• Support non-Gaussian ConvNP likelihoods in low-level and high-level prediction interfaces by @tom-andersson in https://github.com/alan-turing-institute/deepsensor/pull/97
• Fix multiple N-D target sets case in ConvNP; Update ConvNP API
• Minor suggestions to installation docs. by @davidwilby in https://github.com/alan-turing-institute/deepsensor/pull/98

New Contributors

• @davidwilby made their first contribution in https://github.com/alan-turing-institute/deepsensor/pull/98

Full Changelog: https://github.com/alan-turing-institute/deepsensor/compare/v0.3.5...v0.3.6

- Python
Published by tom-andersson about 2 years ago

deepsensor - v0.3.5

What's changed

• Fix bug in batchwise Trainer training with multi-dimensional context sets (https://github.com/tom-andersson/deepsensor/issues/93; thanks @nilsleh)

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.3.4...v0.3.5

- Python
Published by tom-andersson over 2 years ago

deepsensor - v0.3.4

• Fix PyPI upload by removing get-station-data dependency; instead raise error that it must be installed manually if user calls the function that uses it

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.3.3...v0.3.4

- Python
Published by tom-andersson over 2 years ago

deepsensor - v0.3.3

What's Changed

• New documentation by @tom-andersson and @kallewesterling, containing steps for getting started, a user guide, learning resources, a list of research ideas, community information, and more. Uses the Jupyter Book system with fully reproducible notebooks demoing features of the package.
• New deepsensor.data.sources module for downloading ERA5, GHCNd, topography, and land mask data directly into DeepSensor xarray/pandas format. Results can be cached to disk to avoid re-downloading. Leverages get-station-data tool by @scotthosking, @magnusross, and @tom-andersson.
• New plotting tools: deepsensor.plot.prediction and deepsensor.plot.task.
• Various bug fixes.

Breaking changes

• The ConvNP hyperparameter points_per_unit has been renamed to internal_density to be more intuitive. Instances of points_per_unit in code and model JSON configuration files will need to be renamed to internal_density.

New Contributors

• @scotthosking made their first contribution in https://github.com/tom-andersson/deepsensor/pull/86

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.3.2...v0.3.3

- Python
Published by tom-andersson over 2 years ago

deepsensor - v0.3.2

• Fix bug in ContextDist acquisition function (thanks @acocac)

- Python
Published by tom-andersson over 2 years ago

deepsensor - v0.3.1

What's Changed

• Fix bug in ContextDist acquisition function (thanks @acocac)

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.3.0...v0.3.1

- Python
Published by tom-andersson over 2 years ago

deepsensor - v0.3.0

What's Changed

• New Prediction object output by model.predict, containing xarray or pandas data (https://github.com/tom-andersson/deepsensor/issues/53)
• New TaskLoader functionality for generating satellite gap-filling training tasks (https://github.com/tom-andersson/deepsensor/issues/23, example)
• DataProcessor scales each spatial dimension equally to fix stripe artefacts, and raises a warning if user-provided coord mappings don't do this (https://github.com/tom-andersson/deepsensor/issues/77)
• Task operations now work like task.operation_method() and are tracked in the task["ops"] list
• TaskLoader does not sample target data for the Task if target_sampling isn't provided (supports context-only Tasks for inference with model.predict)
• Support spatiotemporal aux_at_targets xarray data (with a time dim), not spatial-only
• Fix model.predict not permitting numerical noise in X_t coordinates (https://github.com/tom-andersson/deepsensor/issues/78)
• Key classes are now exposed from higher import levels (https://github.com/tom-andersson/deepsensor/commit/de876d46cf0b59602b125f4c47471b27949022bb)
• Add CONTRIBUTING.md and CODE_OF_CONDUCT.md by @kallewesterling
• Issue templates by @kallewesterling
• Google docstrings added to the majority of the package by @kallewesterling
• DeepSensor Slack channel request form
• Preliminary sphinx readthedocs page by @kallewesterling

Breaking changes

• Replace any mean_ds, pred_ds, samples_ds = model.predict(...) with pred = model.predict(...). Prediction pandas/xarray objects can now be accessed with pred["<var_ID>"].
• Replace any remove_nans_from_task_Y_t_if_present(task) with task.remove_target_nans()
• DataProcessor now auto-normalises coordinates differently to preserve aspect ratio - see https://github.com/tom-andersson/deepsensor/issues/77. If you aren't specifying spatial coordinate normalisation mapping explicitly (through x1_map/x2_map or through config), your trained models may start receiving different data.

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.2.5...v0.3.0

- Python
Published by tom-andersson over 2 years ago

deepsensor - v0.2.5

What's Changed

• Fix incompatibility with Python < 3.10 (thanks @patel-zeel)

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.2.4...v0.2.5

- Python
Published by tom-andersson over 2 years ago

deepsensor - v0.2.4

What's Changed

• Modularise and track Task operations (eg adding batch dim, removing nans, converting to tensor).
• Fix bug in concat_tasks when there are NaNs in target data (thanks @nilsleh!)

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.2.3...v0.2.4

- Python
Published by tom-andersson over 2 years ago

deepsensor - v0.2.3

What's Changed

• Bug fixes

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.2.2...v0.2.3

- Python
Published by tom-andersson over 2 years ago

deepsensor - v0.2.2

What's Changed

• Add X_t_mask kwarg to DeepSensorModel.predict when predicting on-grid (model will not predict at False/0s and will be returned as NaNs in prediction xarray objects)
• Minor bug fixes

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.2.1...v0.2.2

- Python
Published by tom-andersson over 2 years ago

deepsensor - v0.2.1

What's changed

• Saving/loading bugfixes

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.2.0...v0.2.1

- Python
Published by tom-andersson over 2 years ago

deepsensor - v0.2.0

What's Changed

• DataProcessor, TaskLoader, and ConvNP now each have a .save(folder) method and can be re-initialised with .__init__(folder)
• DataProcessor init kwargs x1_map and x2_map now optional (if not provided, will be auto-inferred from first dataset passed to .__call__)
• Convert normalization parameters to a dictionary for data processor by @patel-zeel in https://github.com/tom-andersson/deepsensor/pull/48

New Contributors

• @patel-zeel made their first contribution in https://github.com/tom-andersson/deepsensor/pull/48

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.1.11...v0.2.0

- Python
Published by tom-andersson over 2 years ago

deepsensor - v0.1.11

What's changed

• Add Trainer class that tracks optimiser object to speed up training
• New diff boolean kwarg for GreedyAlgorithm.__call__ to compute difference after adding query context point

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.1.10...v0.1.11

- Python
Published by tom-andersson over 2 years ago

deepsensor - v0.1.10

What's changed

• Minor bugfix in TaskLoader aux_at_contexts functionality

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.1.9...v0.1.10

- Python
Published by tom-andersson over 2 years ago

deepsensor - v0.1.9

What's new

• Fix bug in TaskLoader.load_dask (thanks @magnusross)
• Handle dates with no pairs of stations in compute_pandas_data_resolution

Breaking changes

• The TaskLoader.__init__ aux_at_contexts arg is no longer a tuple of (int, xarray.DataArray), and is now just an xarray object that will be automatically sampled at the off-grid context locations from all context sets.

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.1.8...v0.1.9

- Python
Published by tom-andersson over 2 years ago

deepsensor - v0.1.8

What's new

• Compute analogue of resolution for scattered pandas data for inferring default ConvNP internal grid resolution
• Allow sampling auxiliary data at locations of another context set in TaskLoader with aux_at_contexts kwarg in TaskLoader.__init__
• deepsensor.plot.offgrid_context_observations utility to plot context observation numerical values
• Fix bug in concatenating/batching Task objects with gridded data (thanks @patel-zeel)
• Support active learning with aux_at_targets downscaling functionality by @RohitRathore1 in https://github.com/tom-andersson/deepsensor/pull/35
• Unified (single) progress bar during active learning with GreedyAlgorithm

Breaking changes * Update DataProcessor config to store user-provided normalisation method for each variable

Contributors

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.1.7...v0.1.8

- Python
Published by tom-andersson over 2 years ago

deepsensor - v0.1.7

• Minor bugfixes in deepsensor.active_learning.acquisition_fn

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.1.6...v0.1.7

- Python
Published by tom-andersson over 2 years ago

deepsensor - v0.1.6

• Downscaling using auxiliary output MLP
• Assign maximisation/minimisation attr to each AcquisitionFunction so the user doesn't need to decide (thanks @jonas-scholz123 @polpel)

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.1.5...v0.1.6

- Python
Published by tom-andersson over 2 years ago

deepsensor - v0.1.5

What's Changed

• Fix rounding errors in DeepSensorModel.predict coordinates from normalise-unnormalise operations by @tom-andersson and @polpel in https://github.com/tom-andersson/deepsensor/pull/25
• Support autoregressive (AR) sampling in DeepSensorModel.predict
• Support training with multiple non-overapping targets
• Support NaNs in context and target data for ConvNP
• deepsensor.active_learning enhancements
• Support numpy coordinates in TaskLoader context/target and DeepSensorModel.predict targets
• Switch default random sampling behaviour for xarray data to use linear interpolation rather than grid-cell wise to avoid prediction artifacts away from grid cells (thanks @jonas-scholz123)

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.1.4...v0.1.5

- Python
Published by tom-andersson over 2 years ago

deepsensor - v0.1.4

• Fix decoder scale not being inferred from model discretisation density (ppu). Closes https://github.com/tom-andersson/deepsensor/issues/18
• Add deepsensor.plot.feature_maps method
• Provisional deepsensor.active_learning functionality with acquisition functions and greedy algorithm

Breaking changes * ConvNP class moved from deepsensor.model.models to deepsensor.model.convnp

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.1.3...v0.1.4

- Python
Published by tom-andersson over 2 years ago

deepsensor - v0.1.3

Try to trigger PyPI upload with publisher set up on PyPI

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.1.2...v0.1.3

- Python
Published by tom-andersson over 2 years ago

deepsensor - v0.1.2

What's New

• Breaking change: Plotting module moved and methods renamed for nicer-looking imports. E.g. now deepsensor.plot.plot.plot_context_encoding is deepsensor.plot.context_encoding.
• Added deepsensor.plot.receptive_field method for plotting model's RF in unnormalised space with cartopy coastlines.

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.1.1...v0.1.2

- Python
Published by tom-andersson over 2 years ago

deepsensor - v0.1.1

What's New

• Published on PyPI
• ConvNP class defaults to CNP model
• More intuitive names for ConvCNP or ConvCNP: ConvNP(..., likelihood="cnp") and ConvNP(..., likelihood="gnp")

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.1.0...v0.1.1

- Python
Published by tom-andersson over 2 years ago

deepsensor - v0.1.0

Pre-release with basic DeepSensor functionality for data processing, task loading, and neural process training/inference.

What's New

• DataProcessor for normalising xarray and pandas data + standardising coordinates
• TaskLoader for loading neural process meta-learning tasks from xarray and/or pandas data, outputting Task objects
• TaskLoader.__call__ provides sampling schemes for generating context and target sets. Options:
  • random sampling (xarray/pandas),
  • passing all observations (xarray/pandas),
  • randomly splitting into context & target (pandas only).
• ProbabilisticModel class providing blueprint for generic model interface
• DeepSensorModel(ProbabilisticModel) class for outputting unnormalised model predictions in xarray (grid) or pandas (off-grid)
• ConvNP(DeepSensorModel) model class wrapping around neuralprocesses (https://github.com/wesselb/neuralprocesses) for convolutional neural process modelling
• train_epoch method implementing simple training scheme on a list of Tasks

Contributors

• Thanks to @wesselb for support with backend-agnosticism!
• DataProcessor dimension validation + unit tests by @jonas-scholz123 in https://github.com/tom-andersson/deepsensor/pull/2
• fix str of tensorflow backend by @acocac in https://github.com/tom-andersson/deepsensor/pull/3
• Fix else level in setgpudefault_device() by @polpel in https://github.com/tom-andersson/deepsensor/pull/4
• Fix DataProcessor's validaiton of dimension ordering in xr.Dataset by @polpel in https://github.com/tom-andersson/deepsensor/pull/5

Full Changelog: https://github.com/tom-andersson/deepsensor/commits/v0.1.0

- Python
Published by tom-andersson over 2 years ago