mici - v0.3.0

What's Changed

🚨 Breaking changes

• System classes now define a backend argument which defaults to None specifying automatic differentiation backend to use if derivative functions not directly provided (with None default raising an error if derivative functions are not specified). Previous behaviour which was to fallback to using Autograd can be restored by passing backend="autograd" to system class initialiser. ### ✨ New features
• Add support for alternative automatic differentiation backends by @matt-graham in https://github.com/matt-graham/mici/pull/11
• Make progress bars display in pyodide based interactive shells by @matt-graham in https://github.com/matt-graham/mici/pull/18 ### 📦 Dependency updates
• Python 3.9 support dropped
• Bump actions/setup-python from 4 to 5 by @dependabot in https://github.com/matt-graham/mici/pull/14
• Bump actions/checkout from 3 to 4 by @dependabot in https://github.com/matt-graham/mici/pull/17
• Bump peaceiris/actions-gh-pages from 3 to 4 by @dependabot in https://github.com/matt-graham/mici/pull/15
• Bump actions/cache from 3 to 4 by @dependabot in https://github.com/matt-graham/mici/pull/16

Full Changelog: https://github.com/matt-graham/mici/compare/v0.2.1...v0.3.0

- Python
Published by matt-graham over 1 year ago

mici - v0.2.1

Minor release.

Bug fixes: * Fixed bug in handling of NaN Hamiltonian values when computing Metropolis accept probability due to min(0, float("nan")) evaluating to 0. * Fixed CSS styling of progress bar HTML representation to avoid overflow in container div element causing horizontal scrollbars to appear. * Fixed bug causing sampling stage progress bar to still be displayed when calling sample_chains method with argument display_progress=False.

Packaging changes * Excluded docs and images folders from sdist builds to minimize download size.

- Python
Published by matt-graham over 2 years ago

mici - v0.2.0

Major release.

Dependency changes

• Dropped support for Python 3.6, 3.7 and 3.8.
• Minimum NumPy version changed to 1.22 per NEP 29.

API changes

• Interface to sampler classes simplified to remove previous sample_chain and sample_chains_with_adaptive_warm_up methods in favour of combining all functions into a single sample_chains method, which now requires arguments to be passed specifying both number of warm-up iterations (n_warm_up_iter, can be zero) and number of main iterations (n_main_iter). All arguments to sample_chain other than n_warm_up_iter, n_main_iter and init_states are now keyword only.
• metric and optional arguments specifying derivatives (for example grad_neg_log_dens and jacob_constr) to system classes are now all keyword only.
• sample_chains methods now return named tuples with entries final_states, traces and statistics.
• memmap_enabled keyword argument to sample_chains removed and replaced with force_memmap argument, with adjusted semantics that memory mapping is now always enabled when sampling chain in parallel on multiple processes, with force_memmap allowing memory mapping to also be used when sampling a chain or chains on a single process.

Bug fixes

• Bug due to mici.transitions.Transition.statistic_types being shared across subclasses as a mutable set fixed.
• Accept probabilities now computed using exp(min(.)) rather than min(exp(.)) to improve numerical stability.
• reverse_norm argument to ConstrainedLeapfrogIntegrator initializer was previously ignored - this is now fixed.
• Usages of deprecated numpy.bool type removed.

New, changed and removed features

• Added implicit midpoint integrator (mici.integrators.ImplicitMidpointIntegrator) for non-separable Hamiltonian systems, with dh2_dpos method added to EuclideanMetricSystem to allow also using with implicit midpoint integrator for testing purposes.
• Added family of symmetric composition integrators described in Blanes, Casas, Sanz-Serna (2014).
• Added option to record traces during warm-up (trace_warm_up argument to sample_chains method).
• Added new mici.interop module with convenience functions for converting Mici sample_chains output to an arviz.InferenceData object, sampling a PyMC model with Mici and sampling a PyStan model with Mici.
• Added new projection solver for use with ConstrainedLeapfrogIntegrator which performs a Newton iteration with backtracking line search (mici.solvers.solve_projection_onto_manifold_newton_with_line_search).
• Default for projection_solver argument to ConstrainedLeapfrogIntegrator initializer changed to mici.solvers.solve_projection_onto_manifold_newton.
• Step size adapter now allows using custom reducer for combining per-chain adapted step sizes.
• Current step size now recorded in integration transition statistics (with key step_size) allowing live monitoring using monitor_stats argument to sample_chains.
• Handling of matrices initialised with non finite values improved with new mici.errors.LinAlgError exception being raised.
• Automatic chain truncation on keyboard interrupt removed.
• Norm functions now do not explicitly use NumPy API, instead using array methods / operator overloads, to allow using to array-like inputs such as jax.Array instances.

Documentation and packaging

• Documentation now built using Sphinx.
• Additional details and references added to docstrings and formatting improved.
• Project metadata now stored in pyproject.toml file with setup.py script removed.

- Python
Published by matt-graham over 2 years ago

mici - v0.1.10

Minor release.

Bug fixes:

• Fixed bug with metric adaptation due to momentum not being resampled. Previously after the metric was updated at the end of an adaptive sampling stage with a metric adapter active the chain state momentum component was left unchanged. As under the new metric this momentum may have a very high kinetic energy (low density) and the cached Hamiltonian and gradients depending on this momentum are no longer valid this could cause the initialisation of the step size in an immediately subsequent adaptive sampling stage to fail due to the Hamiltonian at the initial state being incorrect and thus a large chain in Hamiltonian always being computed irrespective of the step size.

- Python
Published by matt-graham over 5 years ago

mici - v0.1.9

Minor release.

New features

• Added option to limit threads used per process when sampling chain in parallel with max_threads_per_process option to sample_chains* methods if threadpoolctl is installed.

Bug fixes

• Fixing non pass through of call counter on state copy. Bug introduced in fdea526 when switching state._call_counts from dictionary to collections.Counter instance which meant call counter not shared by all copies of state which is intended behaviour to ensure that all calls are recorded.
• Changed to always reinitializing integrator step size at beginning of sampling stages with step size adaptation engaged. Changes from previous behaviour of not reinitializing step size of integrator if previously set on beginning of adaptive sampling stage, which occured if for example step size was set manually when constructing integrator or had been set in a previous adaptive chain stage. Instead when using step size adaptation the integrator step size is now always reinitialized to a reasonable value when initializing the adapter, which ensures the initial step size is appropriate if e.g. the metric has been changed or a previous chain (when running sequential chains) had adapted to a step size inappropriate for the position in state space of the current chain.
• Adjusted step size initialization logic to be robust to cases where Hamiltonian function may evaluate to NaN at some states
• Fixed bug with exception variable not being initialized in edge case of sample chains call with zero length collection of initial states.
• Removed faulty custom subclass / instance check behaviour for matrix classes.

- Python
Published by matt-graham over 5 years ago

mici - v0.1.8

Bug fixes

• Fixed bug when sampling chains in parallel without multiprocess installed introduced in v0.1.6 release due to use of non-pickleable local function in default value for trace_funcs argument.
• Corrected handling of zero and infinity edge cases by LogRepFloat class.
• Fixed bug in constructing diagonal of *BlockDiagonalMatrix classes.
• Fixed bug when using sample_chain method with display_progress=False.

Dependency changes and deprecations

• Minimum numpy version changed to 1.17 to ensure new-style numpy.random.Generator RNGs are available with use of numpy.random.RandomState instances deprecated.

API changes

• tqdm based progress bar implementation mici.progressbars.TqdmProgressBar removed as alternative to inbuilt progress bar class.
• mici.utils.convert_to_arviz_inference_data helper function removed as traces dictionary can be directly used in most ArviZ API functions and/or explicitly converted to an ArviZ InferenceData container using the inbuilt convert functions.
• Logic controlling staging of chain iterations in sampler sample_chains_with_adaptive_warm_up methods factored out to 'stager' classes in mici.stagers module. The sample_chains_with_adaptive_warm_up methods now accept a new keyword argument stager which replaces the previous windowed-adaptation specific n_init_fast_stage_iter, n_init_slow_window_iter, n_final_fast_stage_iter and slow_window_multiplier keywords, with these parameters now able to be instead set via the initializer for the mici.stagers.WindowedWarmUpStager class.
• Memoizing decorator multi_cache_in_state, used to cache outputs of system functions with additionally auxiliary outputs, renamed to cache_in_state_with_aux.

- Python
Published by matt-graham over 5 years ago

mici - v0.1.7

Minor release.

Bug fixes

• [sampler].sample_chains_with_adaptive_warmup methods added in previous release for sampling chains with adaptive warm up stage(s) previously were reinitialising each stage at the initial state rather than the final state from the previous stage. This is now fixed.

New features

• HTML representation added for LabelledSequenceProgressBar used for indicating progress through stages of adaptive chains to improve visualisation in interfaces supporting HTML output such as Jupyter notebooks.

- Python
Published by matt-graham over 5 years ago

mici - v0.1.6

New features and efficiency improvements

• Added framework for adaptation of transition parameters via classes in new mici.adapters module and associated sample_chains_with_adaptive_warm_up methods added to sampler classes. Initial adapters implemented are a dual averaging step size adapter, a diagonal metric adapter based on estimated variances and dense metric adapter based on estimated covariance matrices.
• New labelled sequence progress bar implementation to use with adaptive sampling chains and generalisation of progress bars to allow wrapping arbitrary sequences.
• Improved matrix operation efficiency. Repeated calls to matrix constructors avoided by caching matrices constructed by transpose / sqrt / inv methods and explicit matrix-vector multiplies used for inverted factored definite matrices to avoid bottleneck of scipy.linalg.solve_triangular which is much slower than a direct matrix-vector multiply.
• Remove redundant recaluclation of product of momentum and inverse metric when calculating value and gradient of h2 Hamiltonian component.

Bug fixes

• Non-display of progress bars when running on Google Colab fixed.

Backwards incompatible changes

• Previously ExplicitLeapfrogIntegrator was aliased to LeapfrogIntegrator. ExplicitLeapfrogIntegrator has now been renamed to LeapfrogIntegrator with no aliases.
• The dictionaries of chain statistics returned by sample_chain and sample_chains call no longer contain an hamiltonian key. Instead by default the traces dictionary returned by these methods includes a hamiltonian entry (as generated by the default trace function used when a custom list of trace functions are not specified by the trace_funcs keyword argument). The rationale for this change is to standardize the chain statistics dictionary as only including values which don't correspond directly to a function of the chain states at the end of each iteration but instead generally depend on the whole transition, with the trace_funcs keyword argument already allowing recording of quantities outside this category.
• The dictionaries of chain statistics returned by sample_chain and sample_chains call no longer contain an accept_prob key due to the ambiguity of its meaning and differing usage in the static Metropolis and dynamic HMC integration transitions. Both classes of transitions now return a accept_stat statistic in the statistics dictionary, this corresponding to an overall measure of the probability of accepting a move both due to non-zero changes in the Hamiltonian over a trajectory and any early termination of trajectories due to convergence errors or non-reversible steps when using an implicit integrator. This single summary statistic is intended to both provide a quick summary of sampling performance e.g. in the monitored statistics in chain progress bars, and as the default target for adaptation of the integrator step size. Individual transitions also provide additional acceptance probability related statistics with differing meanings. The dynamic integration transitions (subclasses of DynamicIntegrationTransition and corresponding Dynamic*HMC sampler classes) additionally return an av_metrop_accept_prob statistic which is the average Metropolis acceptance probability for a move from the initial state to each of the states on the whole trajectory tree built (including subtrees which are invalid under the no-U-turn criteria); this does not include any adjustment for early termination of trajectories due to convergence errors / non-reversible steps in implicit integrators so should be used with caution in such cases. The Metropolis integration transitions (subclasses of MetropolisIntegrationTransition and corresponding *MetropolisHMC sampler classes) additionally return a metrop_accept_prob statistic which is the Metropolis acceptance probability of the move proposed by the transition without any adjustment for rejection due to convergence errors / non-reversible steps in implicit integrators.
• Documentation, internal naming and API has been standardized to use -ize rather than -ise endings. The only public facing change outside of the documentation is the renaming of mici.systems.SoftAbsRegularisedPositiveDefiniteMatrix to mici.systems.SoftAbsRegularizedPositiveDefiniteMatrix.

- Python
Published by matt-graham over 5 years ago

mici - v0.1.5

Minor release.

New features

• Implementation of dynamic integration time (NUTS-like) samplers has been refactored, with new implementation including both existing multinomial, and a new slice sampling, variants, with the latter corresponding to the original NUTS algorithm. An option to enable extra subtree termination checks to create more robust no-U-turn checking (see https://github.com/stan-dev/stan/pull/2800) has also been added, and enabled by default for the DynamicMultinomialHMC sampler.
• When used with Numpy v1.17 or above, Mici will now by default use the new-style random number generators for per-chain RNGs, with the new objects supporting more robust production of independent parallel random streams.
• Documentation of classes in mici.systems module is now much fuller.
• An option to disable display of progress bars when sampling chains was added.
• Display refreshes of progress bars rate limited to ~ 4 times per second to prevent the display updates becoming a bottleneck for very fast chains.
• Steffensen's method fixed point solver made more robust.
• Further matrix classes added to support low-rank update matrices and specialised matrix product classes, and matrix classes extended to be hashable to allow equality testing.
• Improved robustness to linear-algebra errors during sampling, with NumPy / SciPy LinAlgError exceptions when attempting Cholesky decomposition of non (numerically) positive definite matrix now handled rather than halting sampling.

Bug fixes

• Fixed bug in log determinant gradient implementation for triangular factored definite matrix classes.
• Fixed bug when integrating Gaussian constrained system with large timestep due to prior incorrect assumption of all eigenvalues of matrix being positive.

- Python
Published by matt-graham almost 6 years ago

mici - v0.1.4

Minor release with several bug fixes. Thanks to @kx-au for spotting and suggesting fixes to the bugs in the ImplicitLeapfrogIntegrator and solve_fixed_point_direct implementations.

• A previous attempt at adding a more helpful message when an error is encountered due to using parallel sampling with Autograd model functions without having multiprocess installed incorrectly assumed the exception raised would always be a PicklingError while in reality an AttributeError was being raised. The code now handles this case and correctly outputs an error message suggesting for the user to try installing multiprocess when trying to sample chains in parallel with Autograd model functions when multiprocess is not available.
• Several bugs / name errors in the implementation of the mici.integrators.ImplicitLeapfrogIntegrator have been fixed which previously prevented instantiation of this class.
• A bug in the naming of the tolerance parameter in mici.solvers.solve_fixed_point_direct introduced in 5e012034 has been fixed.
• The classes in the mici.matrices module have been refactored to use abc abstract base classes to make the expected interfaces for the various matrix types more explicit and docstrings added to most classes and methods.
• The mici.matrices unit tests have been updated to improve the coverage of the classes and reduce code duplication in writing the tests.
• The mici.integrator unit tests coverage has also been improved slightly by adding a basic test for a ImplicitLeapfrogIntegrator instance.

- Python
Published by matt-graham about 6 years ago

mici - v0.1.3

Minor release with improvements to non-linear equation solvers, additional block diagonal matrix classes and fix to bug when monitoring boolean chain statistics.

• Non-linear equation solvers, including solvers for projection on to manifold for constrained systems, all now implement a test for the change in the position vector between iterations being less than a tolerance as well as the tolerance on the non-linear equation being satisfied. For the projection solvers, controlling the position error in addition to the constraint error helps reduce the probability of spurious detections of non-reversible steps due to positions not converging to within the reverse check tolerance.
• Matrix classes corresponding to square block diagonal matrices added, allowing for example efficient use of metric matrix representations with such structure.
• Bug in update of running mean of monitored chain statistics due to assumption of floating point values fixed by casting all values to floats prior to computing update.

- Python
Published by matt-graham about 6 years ago

mici - v0.1.2

Minor release with sampling efficiency improvements, new facility for tracking the number of model function calls during sampling, improved robustness to use of non-NumPy arrays, and miscellaneous bug fixes.

• In constrained leapfrog integrator some derivatives of quantities are now pre-evaluated before the associated quantity is evaluated as often the quantity itself will be calculated alongside the derivative and cached. The resulting reduction in the number of model function evaluations provides a small gain in sampling efficiency in models where it applies.
• Chain state objects now optionally may be initialised with a _count_calls dictionary object which will be used to record the number of calls to system methods decorated with the cache_in_state and multi_cache_in_state decorators. This can be used to give a hardware independent measure of the operation cost of a sampling run.
• Code has been refactored to remove explicit dependence on the in-place semantics of the augmented assignment operators such as += when acting on NumPy arrays. NumPy array like objects such as JAX DeviceArray instances do not perform updates in place, instead creating a new array object, meaning relying on such operations happening in place was silently failing when using such arrays.
• An non-guarded dependence on a decorator imported from autograd in the autograd_wrapper module has now been fixed meaning the package can be imported without autograd being installed as intended.
• Progress bar styling in Jupyter notebooks has been updated to remove the dependence on CSS variables defined in the ipywidgets stylesheets, instead now using explicit values.

- Python
Published by matt-graham over 6 years ago

mici - v0.1.1

Minor release improving robustness of parallel sampling and progress bars.

• Now possible to get partial chains sampled so far on manual interrupt when sampling multiple chain in parallel.
• Added in-built progress bar implementation as alternative to tqdm and improving handling of progress bars when sampling chains in parallel by moving progress bar updates to parent process by communicating via a shared queue.
• Added more informative error message when a PicklingError is raised due try to pickle an Autograd generated function using inbuilt multiprocessing module.

- Python
Published by matt-graham over 6 years ago

mici - v0.1.0-r1

Post-release update to add HTML API documentation

- Python
Published by matt-graham over 6 years ago

mici - v0.1.0

Initial PyPI release

- Python
Published by matt-graham over 6 years ago