Releases | Open Source Science

ivis - ivis: dimensionality reduction in very large datasets using Siamese Networks

What's Changed

GitHub actions by @Szubie in https://github.com/beringresearch/ivis/pull/120
Update doc requirements by @Szubie in https://github.com/beringresearch/ivis/pull/121
GitHub pages by @Szubie in https://github.com/beringresearch/ivis/pull/122

Full Changelog: https://github.com/beringresearch/ivis/compare/2.09...2.10

Scientific Software - Peer-reviewed - Python
Published by idroz over 1 year ago

ivis - ivis: dimensionality reduction in very large datasets using Siamese Networks

Full Changelog: https://github.com/beringresearch/ivis/compare/2.08...2.09

Scientific Software - Peer-reviewed - Python
Published by idroz about 3 years ago

ivis - ivis: dimensionality reduction in very large datasets using Siamese Networks

Scientific Software - Peer-reviewed - Python
Published by idroz over 3 years ago

ivis - ivis: dimensionality reduction in very large datasets using Siamese Networks

Added ability to save/load ivis models that have not been trained. This also fixes an issue when using GridSearchCV in conjunction with ivis
Bugfix for triplet generator when used in conjunction with a dataset exposing the custom get_triplet_data method

Scientific Software - Peer-reviewed - Python
Published by idroz almost 4 years ago

ivis - ivis: dimensionality reduction in very large datasets using Siamese Networks

New features:

ivis models are now serializable via pickle/dill/joblib. Thanks to @imatheussm for his contributions toward this.
The savemodel method now accepts an optional "saveformat" argument. Setting it to "tfs" will export ivis models in the TensorFlow SavedModel format, which integrates well with other TensorFlow libraries.

Scientific Software - Peer-reviewed - Python
Published by idroz over 4 years ago

ivis - ivis: dimensionality reduction in very large datasets using Siamese Networks

Highlights: * Improved training speed for numpy arrray inputs thanks to a faster triplet generator. * Batched retrieval capabilities that makes ivis much faster when training on out-of-memory data that is retrieved in parallel. * Improved performance when using Ivis with precompute=False option by using multi-threading when retrieving batches of KNN on-demand. * Added deprecation notices for minor upcoming changes to API for consistency and adherence to sklearn API.

Scientific Software - Peer-reviewed - Python
Published by idroz over 4 years ago

ivis - ivis: dimensionality reduction in very large datasets using Siamese Networks

Knn retrieval made more efficient by switching from multi-processing to multi-threading. Memory savings depend on OS and core count.
Fixed issue where saved ivis models would attempt to load the index at the path they were saved with - this can't be relied on when the index is temporary and deleted after use.
Fixed issue where Annoy Index metric parameter was not passed to an index that was loaded from disk.
A few other things changed, including better error handling, cleaner code, and allowing for saving AnnoyKnnMatrix via pickle

Scientific Software - Peer-reviewed - Python
Published by idroz over 4 years ago

ivis - ivis: dimensionality reduction in very large datasets using Siamese Networks

improved memory utilization during KNN retrieval
AnnoyIndex is now removed from disk after running Ivis

Scientific Software - Peer-reviewed - Python
Published by idroz over 4 years ago

ivis - ivis: dimensionality reduction in very large datasets using Siamese Networks

Minor release * Fixes zero chunk error #90

Scientific Software - Peer-reviewed - Python
Published by idroz almost 5 years ago

ivis - ivis: dimensionality reduction in very large datasets using Siamese Networks

Minor release addressing:

Tensorflow 2.4 model save compatibility (#82)
Training/Inference batch size concordance

Scientific Software - Peer-reviewed - Python
Published by idroz about 5 years ago

ivis - ivis: dimensionality reduction in very large datasets using Siamese Networks

Major ivis release!

Version 2.0 features:

Unsupervised, semi-supervised, and fully supervised dimensionality reduction
Support for arbitrary datasets:
- N-dimensional arrays
- Image files on disk
- Custom data connectors
In- and out-of-memory data ingestion
Resumable training
Arbitrary neural network backbones
Customizable neighbour retrieval
Callbacks and Tensorboard integration

Scientific Software - Peer-reviewed - Python
Published by idroz about 5 years ago

ivis - ivis: dimensionality reduction in very large datasets using Siamese Networks

Added support for TensorFlow 2.3.0
Visualise embeddings using EmbeddingsExplorer class through datashader in Jupyter notebooks

Scientific Software - Peer-reviewed - Python
Published by idroz over 5 years ago

ivis - ivis: dimensionality reduction in very large datasets using Siamese Networks

Improved handling of n-dimensional arrays and HDF5 files

Scientific Software - Peer-reviewed - Python
Published by idroz over 5 years ago

ivis - ivis: dimensionality reduction in very large datasets using Siamese Networks

Support for dimensionality reduction of arbitrary n-dimensional arrays.

Scientific Software - Peer-reviewed - Python
Published by idroz over 5 years ago

ivis - ivis: dimensionality reduction in very large datasets using Siamese Networks

Compatibility fixes with tensorflow ≥1.13.1

Scientific Software - Peer-reviewed - Python
Published by idroz over 5 years ago

ivis - ivis: dimensionality reduction in very large datasets using Siamese Networks

Introducing neighbour_matrix parameter for provision of arbitrary KNNs.
Transition to tf.Datasets, improving memory efficiency and overall stability

Scientific Software - Peer-reviewed - Python
Published by idroz almost 6 years ago

ivis - ivis: dimensionality reduction in very large datasets using Siamese Networks

Scientific Software - Peer-reviewed - Python
Published by idroz almost 6 years ago

ivis - ivis: dimensionality reduction in very large datasets using Siamese Networks

This release addresses #50 by making it easy to alternate between CPU- and GPU-enabled tensor flow backend

Scientific Software - Peer-reviewed - Python
Published by idroz about 6 years ago

ivis - ivis: dimensionality reduction in very large datasets using Siamese Networks

Major features:

Support for semi-supervised dimensionality reduction
Switch from using fit_generator to fit for training the Keras model
Address eager execution issues with TF 2.0
User-configurable on-disk-building of Annoy index.
Tidy handling of interrupted multi-thread processes

Minor features:

Tests for semi-supervised DR
Improved input validation
Better hyper parameter validation
Slight changes to default hyperparameters
Bug fixes