careless - v0.5.1

This version adds clearer error messages when careless can't figure out what columns to use for metadata, intensities, uncertainties, and image identifiers.

What's Changed

• Add error messages for data keys by @kmdalton in https://github.com/rs-station/careless/pull/185

Full Changelog: https://github.com/rs-station/careless/compare/v0.5.0...v0.5.1

- Python
Published by kmdalton 9 months ago

careless - v0.5.0

What's Changed

• Removed reset_index() call causing DataFrame indices to be plotted by @PrinceWalnut in https://github.com/rs-station/careless/pull/178
• make cchalf use unweighted pearson by @kmdalton in https://github.com/rs-station/careless/pull/181

New Contributors

• @PrinceWalnut made their first contribution in https://github.com/rs-station/careless/pull/178

Full Changelog: https://github.com/rs-station/careless/compare/v0.4.9...v0.5.0

- Python
Published by kmdalton about 1 year ago

careless - v0.4.9 Update TFP to 0.25

This release simplifies install using TensorFlow probability 0.25 and TensorFlow version 2.18.0. The cuda install script should no longer be needed, but can still be used.

What's Changed

• set tf max version to 2.17 by @kmdalton in https://github.com/rs-station/careless/pull/176
• update pre-install script to tfp 0.25 by @kmdalton in https://github.com/rs-station/careless/pull/177

Full Changelog: https://github.com/rs-station/careless/compare/v0.4.8...v0.4.9

- Python
Published by kmdalton over 1 year ago

careless - v0.4.8 Address Various Issues

This PR addresses a number of longstanding issues. The user facing changes are: - Improved numerical stability for the --scale-bijector=exp setting - Allow setting the y-axis limits for many of the careless stats scripts - Enforce a consistent dmin between ASUs (#71 ) - Add a script for calculating I / SigI (#148) - Make it clear that careless poly does not support .stream files

On the developer side: - Adds a missing test for the Double Wilson prior

- Python
Published by kmdalton over 1 year ago

careless - v0.4.7 Improved Numerical Stability

This release changes the default bijector for the standard deviation of the scale distribution from Exp to Softplus to increase numerical stability.

- Python
Published by kmdalton over 1 year ago

careless - v0.4.6 Add Gradient Clipping Options

This release adds 3 new flags controlling gradient clipping during training. - --clipvalue - --clipnorm - --global-clipnorm These each take floating point values and are disabled by default. The meaning of each is documented in the Keras Adam docs.

- Python
Published by kmdalton over 1 year ago

careless - v0.4.5 Update Build Tooling

This version updates packaging and distribution without touching any user-facing functionality.

- Python
Published by kmdalton over 1 year ago

careless - v0.4.4 careless devices

This release does little more than rename careless test to the more semantically meaningful careless devices. This release is needed to ensure compatibility with the cuda installation script.

- Python
Published by kmdalton almost 2 years ago

careless - v0.4.3 DW Reindexing Ops, Update TF, Install Script

This release has 3 major changes: - Add and document an install script for CUDA support - Update TensorFlow to 2.16 - Add reindexing operations for the double Wilson prior.

- Python
Published by kmdalton almost 2 years ago

careless - Fix Double-Wilson on CPU

This release addresses a bug that prevented careless from running with the double-Wilson prior on CPU. (see #150)

- Python
Published by kmdalton about 2 years ago

careless - v0.4.1 Harmonic Deconvolution Bug Fix

This release fixes a minor bug that could cause careless to crash in certain situations with Laue data at high resolution. This should only affect cases with the --separate-files flag. See #145 and #146 for more details.

- Python
Published by kmdalton over 2 years ago

careless - Update CLI Parameters

This release introduces two small changes to improve usability. 1) Add a description of .stream file metadata to the CLI. These were previously undocumented and not obvious. 2) Change the default of --mlp-width to 10. Previously this parameter was dynamically inferred from the metadata. This behavior made it dangerous to compare CCpred between different scaling scenarios owing to differences in model capacity. Furthermore, it led to unreasonably high memory usage when positional encoding was used.

- Python
Published by kmdalton over 2 years ago

careless - v0.3.9 Fix Stats Module

This release fixes some dependency issues which caused various scripts from the stats submodule to fail. Impacted functions: - careless.ccpred - careless.cchalf - careless.rsplit - careless.completeness - careless.filterimage - careless.imageccpred - careless.ccanom

- Python
Published by kmdalton over 2 years ago

careless - v0.3.8 Standardize Metadata

This is a very minor release that changes how metadata columns with zero variance are reported.

- Python
Published by kmdalton over 2 years ago

careless - v0.3.7 Image CCpred

This release adds new functions to calculate and filter by CCpred for each image. - careless.image_ccpred is a new function with similar syntax to the other stats scripts. It calculates the correlation coefficient between the predictions and data for each image. - careless.filter_image is a tool that can be used to reject images in unmerged data on the basis of CCpred from a previous careless run.

- Python
Published by kmdalton over 2 years ago

careless - v0.3.6 Stability and Stats Improvements

In this release there are two major changes - All correlation coefficient based stats functions now use a weighted Pearson correlation by default - If training encounters numerical issues, optimization will now terminate early rather than crashing

- Python
Published by kmdalton over 2 years ago

careless - v0.3.5 Fix Numerical Instability

This release fixes a bug which could lead to numerical instability of the truncated normal surrogate posterior.

- Python
Published by kmdalton over 2 years ago

careless - v0.3.4 kl weight

This version adds a new feature to the CLI, the --kl-weight parameter. When --kl-weight is specified, the log likelihood and kl divergence terms in the loss function are reduced by taking the mean rather than the sum. The ELBO is then computed as

ELBO = mean(log_likelihood) + kl_weight * mean(kl_div) This feature will be useful in making comparisons across isomorphous data sets of different sizes.

- Python
Published by kmdalton over 2 years ago

careless - v0.3.3 Improve Predictions Output

This release does two things - Fix a bug whereby the miller indices for Laue predictions files were incorrect - Add fileid, asuid, and image_id columns to the predictions files

- Python
Published by kmdalton over 2 years ago

careless - v0.3.2 Output Scales

In v0.3.1, saving the mean and uncertainty of scale factors for each reflection was implemented incorrectly. This release fixes the bug.

- Python
Published by kmdalton almost 3 years ago

careless - v0.3.1

This release - More automated detection of uncertainty estimates in unmerged mtz files - Better handling of systematic absences under changes in space group for the double Wilson prior - Add a file ID column to the data available at the merging API level - Add the first and second moments of $\Sigma$ (scale factor) to the predictions - Refactor the stats module and make the interface more uniform - Add a couple new scripts to help with b-factor issues

- Python
Published by kmdalton almost 3 years ago

careless - v0.3.0 XDS Double Wilson and Progress Bar

This release adds a few features requested by users. 1) There's a new command line script careless.xds2mtz to generate mtz files for use with careless from XDS .HKL files. 2) The Double Wilson prior now respects the --wilson-prior-b flag for the root node. 3) There's a new --disable-progress-bar flag which turns off the tqdm progress bar during optimization. This is typically sends a lot of text to stderr which is wasteful for non-interactive jobs where you want to log stderr to a file.

- Python
Published by kmdalton almost 3 years ago

careless - v0.2.9 Output Intensities

This release updates careless to output merged intensities in addition to structure factors. The I and SigI estimates are computed in a statistically rigorous manner which uses the fourth moments of the structure factor distributions. However, due to precision loss, the second moments at very high signal to noise may be truncated to reflect a maximum I/SigI value.

- Python
Published by kmdalton about 3 years ago

careless - v0.2.7 Update CCpred

This version - updates ccpred to add the --overall flag which pools results from multiple predictions files - updates behavior to skip normalization for metadata columns with zero variance

- Python
Published by kmdalton about 3 years ago

careless - v0.2.6 Add stats submodule

This version just adds a new stats submodule with several command line entry points defined in setup.py. Specifically, this version adds - careless.cchalf for computing CC1/2 from *_xval_#.mtz files - careless.ccanom for computing CCanom from *_xval_#.mtz files - careless.ccpred for computing CCpred from (potentially multiple) *_predictions_#.mtz files - careless.rsplit for computing Rsplit from *_xval_#.mtz files - careless.completeness for computing completeness from merged mtz files

- Python
Published by kmdalton over 3 years ago

careless - New Home

careless is now part of the rs-station org!

- Python
Published by kmdalton over 3 years ago

careless - v0.2.4 Fix CUDA on Some Systems

The previous version of careless would pull the latest release of TensorFlow which was often a release candidate. This sometimes breaks compatibility with CUDA on making it impossible to train on GPU. This release restricts TF to major versions which are more stable.

- Python
Published by kmdalton over 3 years ago

careless - v0.2.3 Make Graph Execution the Default Again

This release reverts an earlier change which made the slower eager execution the default during training and inadvertently flipped the semantics of the --run-eagerly flag. There are no new features or CLI changes to note. Users will notice much faster training however.

- Python
Published by kmdalton over 3 years ago

careless - v0.2.2 Fix numerical instability

This version fixes the numerical instability issue introduced in v0.2.1 (#61). This version adds a new command line parameter --epsilon which controls the minimum width of variational distributions. The default value of 1e-7 should be fine for most any data.

- Python
Published by kmdalton over 3 years ago

careless - v0.2.1

New Feature: Saving and loading weights in v0.2.1

• Output neural network weights to {outputprefix}scale
• Output structure factor distribution weights {outputprefix}structure_factor
• Load previous structure factors with --structure-factor-file=
• Load previous neural network weights with --scale-file=
• Freeze structure factors with --freeze-structure-factors
• Freeze neural network scales with --freeze-scales

Minor changes

• Switch all bijectors to Exp rather than Softplus
• Print a version number at run time
• Report the log likelihood of test sets
• Initialize the variational distribution to match the width of the prior
• Add CLI flag to control this initial width
• Add a tensorflow implementation of the fourth moment of truncated normals

Bug fixes

• Fixed a bug in metric reporting which lead to incorrect KL Divergences in the history
• Fix a bug whereby the second moment of the predicted intensities was inaccurate

- Python
Published by kmdalton over 3 years ago

careless - v0.2.0

This version is a complete rewrite relative to the previous release. There is very little that this release doesn't touch. It is not safe to assume the CLI arguments are the same. Many features from the previous version (like reference priors) have been removed. If you rely on these for your work, please - continue to use the previous version - file an issue

I will make every effort to reintroduce the feature.

- Python
Published by kmdalton about 4 years ago

careless - v0.1.7 PyPI release

This is the latest version of the original careless implementation. This release is being made for debugging purposes.

- Python
Published by kmdalton over 4 years ago

careless - v0.1.5 Original careless implementation

This is the stable version of the original careless implementation with the original command line interface.

All subsequent releases are going to be based on a completely new API and will initially support a drastically reduced feature set. This rewrite is being conducted to - support more advanced crossvalidation paradigms - make it easier to support add multiple GPU support

- Python
Published by kmdalton over 4 years ago