Data

Tools

Indexes

Applications

Experiments

Open Source Science

ivis

ivis: dimensionality reduction in very large datasets using Siamese Networks - Published in JOSS (2019)

https://github.com/beringresearch/ivis

Science Score: 93.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ✓
    DOI references
    Found 4 DOI reference(s) in README and JOSS metadata
  • ✓
    Academic publication links
    Links to: nature.com, joss.theoj.org, zenodo.org
  • ○
    Committers with academic emails
  • ○
    Institutional organization owner
  • ✓
    JOSS paper metadata
    Published in Journal of Open Source Software

Keywords

data-visualization dimensionality-reduction machine-learning neural-network siamese-neural-network

Scientific Fields

Biochemistry, Genetics and Molecular Biology Life Sciences - 83% confidence
Last synced: 6 months ago · JSON representation

Repository

Dimensionality reduction in very large datasets using Siamese Networks

Basic Info
Statistics
  • Stars: 338
  • Watchers: 12
  • Forks: 45
  • Open Issues: 3
  • Releases: 36
Topics
data-visualization dimensionality-reduction machine-learning neural-network siamese-neural-network
Created over 7 years ago · Last pushed over 1 year ago
Metadata Files
Readme Contributing License

README.md

DOI DOI Documentation Status Downloads Build Status

ivis

Implementation of the ivis algorithm as described in the paper Structure-preserving visualisation of high dimensional single-cell datasets. Ivis is designed to reduce dimensionality of very large datasets using a siamese neural network trained on triplets. Both unsupervised and supervised modes are supported.

ivis 10M data points

Installation

Ivis runs on top of TensorFlow. To install the latest ivis release from PyPi running on the CPU TensorFlow package, run:

```

TensorFlow 2 packages require a pip version >19.0.

pip install --upgrade pip ```

pip install ivis[cpu]

If you have CUDA installed and want ivis to use the tensorflow-gpu package, run

pip install ivis[gpu]

Development version can be installed directly from from github:

git clone https://github.com/beringresearch/ivis cd ivis pip install -e '.[cpu]'

The following optional dependencies are needed if using the visualization callbacks while training the Ivis model: - matplotlib - seaborn

Upgrading

To upgrade, run:

pip install ivis --upgrade

Features

  • Scalable: ivis is fast and easily extends to millions of observations and thousands of features.
  • Versatile: numpy arrays, sparse matrices, and hdf5 files are supported out of the box. Additionally, both categorical and continuous features are handled well, making it easy to apply ivis to heterogeneous problems including clustering and anomaly detection.
  • Accurate: ivis excels at preserving both local and global features of a dataset. Often, ivis performs better at preserving global structure of the data than t-SNE, making it easy to visualise and interpret high-dimensional datasets.
  • Generalisable: ivis supports addition of new data points to original embeddings via a transform method, making it easy to incorporate ivis into standard sklearn Pipelines.

And many more! See ivis readme for latest additions and examples.

Examples

``` from ivis import Ivis from sklearn.preprocessing import MinMaxScaler from sklearn import datasets

iris = datasets.loadiris() X = iris.data Xscaled = MinMaxScaler().fit_transform(X)

model = Ivis(embedding_dims=2, k=15)

embeddings = model.fittransform(Xscaled) ```

Copyright 2024 Bering Limited

Owner

  • Name: beringresearch
  • Login: beringresearch
  • Kind: user
  • Company: Bering Limited

JOSS Publication

ivis: dimensionality reduction in very large datasets using Siamese Networks
Published
August 06, 2019
DOI
Volume 4, Issue 40, Page 1596
Authors
Benjamin Szubert
Bering Limited
Ignat Drozdov ORCID
Bering Limited
Editor
Lorena Pantano ORCID
Tags
dimensionality reduction unsupervised learning neural network
View PDF Review Thread Software Archive

Papers & Mentions

Total mentions: 2

Supervised and unsupervised language modelling in Chest X-Ray radiological reports
Last synced: 4 months ago
Structure-preserving visualisation of high dimensional single-cell datasets
Last synced: 4 months ago

GitHub Events

Total
  • Watch event: 6
  • Fork event: 2
Last Year
  • Watch event: 6
  • Fork event: 2

Committers

Last synced: 7 months ago

All Time
  • Total Commits: 561
  • Total Committers: 10
  • Avg Commits per committer: 56.1
  • Development Distribution Score (DDS): 0.478
Past Year
  • Commits: 453
  • Committers: 9
  • Avg Commits per committer: 50.333
  • Development Distribution Score (DDS): 0.468
Top Committers
Name Email Commits
Szubie b****t@g****m 293
idroz i****v@b****m 242
Igor Matheus Souza Moreira i****m@g****m 13
beringresearch i****v@g****m 4
Kevin Rue-Albrecht k****7@g****m 3
Ignat Drozdov i****t@I****l 2
candalfigomoro c****o@o****m 1
John Sheffield s****e 1
Eugene Scherba e****a@g****m 1
Ignat Drozdov i****v@c****d 1
Committer Domains (Top 20 + Academic)
c1.lxd: 1 beringresearch.com: 1

Issues and Pull Requests

Last synced: 6 months ago

All Time
  • Total issues: 53
  • Total pull requests: 52
  • Average time to close issues: about 1 month
  • Average time to close pull requests: about 11 hours
  • Total issue authors: 34
  • Total pull request authors: 11
  • Average comments per issue: 3.17
  • Average comments per pull request: 0.35
  • Merged pull requests: 48
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 0
  • Pull requests: 1
  • Average time to close issues: N/A
  • Average time to close pull requests: 1 day
  • Issue authors: 0
  • Pull request authors: 1
  • Average comments per issue: 0
  • Average comments per pull request: 0.0
  • Merged pull requests: 1
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • idroz (8)
  • candalfigomoro (4)
  • SaskiaFreytag (4)
  • kevinrue (4)
  • imatheussm (3)
  • sheffe (2)
  • hhaootian (1)
  • vjcitn (1)
  • bitcometz (1)
  • ZHUGUODONG1 (1)
  • paul-harambee (1)
  • r0f1 (1)
  • mihajenko (1)
  • ForrestCKoch (1)
  • ttgump (1)
Pull Request Authors
  • idroz (22)
  • Szubie (17)
  • imatheussm (4)
  • kevinrue (3)
  • sheffe (1)
  • wna26 (1)
  • spiderrobot (1)
  • guyong824 (1)
  • candalfigomoro (1)
  • escherba (1)
  • DaDaGuai-hwm (1)
Top Labels
Issue Labels
discussion (4) bug (3) enhancement (2) R (1) question (1) help wanted (1)
Pull Request Labels

Packages

  • Total packages: 1
  • Total downloads:
    • pypi 680 last-month
  • Total dependent packages: 1
  • Total dependent repositories: 7
  • Total versions: 32
  • Total maintainers: 1
pypi.org: ivis

Artificial neural network-driven visualization of high-dimensional data using triplets.

  • Versions: 32
  • Dependent Packages: 1
  • Dependent Repositories: 7
  • Downloads: 680 Last month
Rankings
Stargazers count: 3.6%
Dependent packages count: 4.8%
Dependent repos count: 5.5%
Forks count: 6.3%
Average: 6.9%
Downloads: 14.5%
Maintainers (1)
beringresearch
Last synced: 6 months ago

Dependencies

R-package/DESCRIPTION cran
  • R >= 3.2.3 depends
  • reticulate * imports
  • Seurat * suggests
  • TENxPBMCData * suggests
  • knitr * suggests
  • rmarkdown * suggests
  • testthat * suggests
docs_requirements.txt pypi
  • Cython *
  • sphinx *
requirements.txt pypi
  • annoy >=1.15.2
  • dill *
  • numpy *
  • scikit-learn >0.20.0
  • tensorflow >=1.13.1
  • tqdm *
setup.py pypi
  • annoy >=1.15.2
  • dill *
  • numpy *
  • scikit-learn >0.20.0
  • tqdm *
.github/workflows/documentation.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v3 composite
  • peaceiris/actions-gh-pages v3 composite
.github/workflows/test.yml actions
  • actions/checkout v4 composite
  • actions/setup-python v4 composite