genieclust
Genie: Fast and Robust Hierarchical Clustering with Noise Point Detection - in Python and R
Science Score: 39.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
✓DOI references
Found 21 DOI reference(s) in README -
○Academic publication links
-
○Committers with academic emails
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (12.3%) to scientific vocabulary
Keywords
Repository
Genie: Fast and Robust Hierarchical Clustering with Noise Point Detection - in Python and R
Basic Info
- Host: GitHub
- Owner: gagolews
- License: other
- Language: C++
- Default Branch: master
- Homepage: https://genieclust.gagolewski.com
- Size: 108 MB
Statistics
- Stars: 66
- Watchers: 7
- Forks: 12
- Open Issues: 27
- Releases: 17
Topics
Metadata Files
README.md
genieclust Package for R and Python
Genie: Fast and Robust Hierarchical Clustering with Noise Point Detection
Genie finds meaningful clusters. It does so quickly, even in large datasets.
A comprehensive tutorial, benchmarks, and a reference manual is available at https://genieclust.gagolewski.com/.
When using genieclust in research publications, please cite (Gagolewski, 2021) and (Gagolewski, Bartoszuk, Cena, 2016) as specified below. Thank you.
About
Genie is a robust and outlier-resistant hierarchical clustering algorithm (see Gagolewski, Bartoszuk, Cena, 2016). Its original implementation was included in the R package genie. This is its faster and more powerful version.
The idea behind Genie( is beautifully simple. First, make each individual point the only member of its own cluster. Then, keep merging pairs of the closest clusters, one after another. However, to *prevent the formation of clusters of highly imbalanced sizes*, a point group of the *smallest size is sometimes combined with its nearest counterpart.
Genie's appealing simplicity goes hand in hand with its usability. It often outperforms other clustering approaches such as K-means, BIRCH, or average, complete, and Ward's linkage on benchmark data. Of course, there is no, nor will there ever be, a single best universal clustering approach for every kind of problem, but Genie is definitely worth a try.
Genie is based on minimal spanning trees of pairwise distance graphs. Thus, it can also be pretty fast: thanks to quitefastmst, determining the entire cluster hierarchy for datasets containing millions of points can be completed in minutes. Therefore, it is well suited to solving extreme clustering tasks (involving large datasets with a high number of clusters to detect).
genieclust allows clustering with respect to mutual reachability distances,
enabling it to act as a noise point detector or a version
of HDBSCAN* (see Campello et al., 2013) that can identify a predefined
number of clusters or their entire hierarchy. The good news is that it
doesn't depend on DBSCAN's somewhat difficult-to-set eps parameter.
The package also features an implementation of:
- economic inequality indices (the Gini, Bonferroni, or de Vergottini index),
- external cluster validity measures (e.g., the normalised clustering accuracy and partition similarity indices such as the adjusted Rand, Fowlkes-Mallows, or mutual information scores),
- internal cluster validity indices (e.g., the Calinski-Harabasz, Davies-Bouldin, Ball-Hall, Silhouette, or generalised Dunn indices).
Author and Contributors
Author and Maintainer: Marek Gagolewski
Contributors:
Maciej Bartoszuk, Anna Cena (R packages
genie and
CVI), and
Peter M. Larsen
(rectangular_lsap).
Examples, Tutorials, and Documentation
The R interface is compatible with stats::hclust(),
but there is more:
```r X <- ... # some data h <- gclust(X) plot(h) # plot cluster dendrogram cutree(h, k=2)
or simply: genie(X, k=2)
```
To learn more about R, check out Marek's open-access textbook Deep R Programming.
The Python language version of genieclust has a scikit-learn-like API:
python
import genieclust
X = ... # some data
g = genieclust.Genie(n_clusters=2)
labels = g.fit_predict(X)
Tutorials and the package documentation are available here.
To learn more about Python, check out Marek's open-access textbook Minimalist Data Wrangling in Python.
How to Install
Python Version
To install from PyPI, call:
bash
pip3 install genieclust # python3 -m pip install genieclust
The package requires Python 3.9+ with cython, numpy, scikit-learn, matplotlib, and quitefastmst.
R Version
To install from CRAN, call:
r
install.packages("genieclust")
Other
The core functionality is implemented in the form of a header-only C++ library. It can thus be easily adapted for use in other projects.
New contributions are welcome, e.g., Julia, Matlab/GNU Octave wrappers.
License
Copyright (C) 20182025 Marek Gagolewski https://www.gagolewski.com/
This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License Version 3, 19 November 2007, published by the Free Software Foundation.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License Version 3 for more details. You should have received a copy of the License along with this program. If not, see (https://www.gnu.org/licenses/).
The file src/c_scipy_rectangular_lsap.h is adapted from the
scipy project (https://scipy.org/scipylib), source:
/scipy/optimize/rectangular_lsap/rectangular_lsap.cpp.
Author: Peter M. Larsen. Distributed under the BSD-3-Clause license.
The implementation of internal cluster validity measures
were adapted from our previous project (Gagolewski, Bartoszuk, Cena, 2021);
see optim_cvi.
Originally distributed under the GNU Affero General Public License Version 3.
References
Gagolewski, M., genieclust: Fast and robust hierarchical clustering, SoftwareX 15, 2021, 100722. DOI: 10.1016/j.softx.2021.100722. https://genieclust.gagolewski.com/.
Gagolewski, M., Bartoszuk, M., Cena, A., Genie: A new, fast, and outlier-resistant hierarchical clustering algorithm, Information Sciences 363, 2016, 823. DOI: 10.1016/j.ins.2016.05.003.
Gagolewski, M., Bartoszuk, M., Cena, A., Are cluster validity measures (in)valid?, Information Sciences 581, 2021, 620636. DOI: 10.1016/j.ins.2021.10.004.
Gagolewski, M., Cena, A., Bartoszuk, M., Brzozowski, L., Clustering with minimum spanning trees: How good can it be?, Journal of Classification 42, 2025, 90112. DOI: 10.1007/s00357-024-09483-1.
Gagolewski, M., Normalised clustering accuracy: An asymmetric external cluster validity measure, Journal of Classification 42, 2025, 230. DOI: 10.1007/s00357-024-09482-2.
Gagolewski, M., A framework for benchmarking clustering algorithms, SoftwareX 20, 2022, 101270. DOI: 10.1016/j.softx.2022.101270. https://clustering-benchmarks.gagolewski.com/.
Campello, R.J.G.B., Moulavi, D., Sander, J., Density-based clustering based on hierarchical density estimates, Lecture Notes in Computer Science 7819, 2013, 160172. DOI: 10.1007/978-3-642-37456-2_14.
Mueller, A., Nowozin, S., Lampert, C.H., Information theoretic clustering using minimum spanning trees, DAGM-OAGM, 2012.
See genieclust's homepage for more references.
Owner
- Name: Marek Gagolewski
- Login: gagolews
- Kind: user
- Location: Melbourne, VIC, Australia
- Company: Deakin University
- Website: https://www.gagolewski.com
- Repositories: 23
- Profile: https://github.com/gagolews
Free universities!
GitHub Events
Total
- Issues event: 4
- Watch event: 7
- Issue comment event: 5
- Push event: 119
- Fork event: 1
Last Year
- Issues event: 4
- Watch event: 7
- Issue comment event: 5
- Push event: 119
- Fork event: 1
Committers
Last synced: 9 months ago
Top Committers
| Name | Commits | |
|---|---|---|
| gagolews | m****i@g****m | 540 |
| Gökçen Eraslan | g****n@g****m | 2 |
| dicorynia | 3****a | 1 |
| Mark Rucker | r****k@g****m | 1 |
Issues and Pull Requests
Last synced: 6 months ago
All Time
- Total issues: 86
- Total pull requests: 3
- Average time to close issues: 4 months
- Average time to close pull requests: about 1 month
- Total issue authors: 14
- Total pull request authors: 3
- Average comments per issue: 1.08
- Average comments per pull request: 1.33
- Merged pull requests: 3
- Bot issues: 0
- Bot pull requests: 0
Past Year
- Issues: 2
- Pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Issue authors: 2
- Pull request authors: 0
- Average comments per issue: 0.5
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Top Authors
Issue Authors
- gagolews (72)
- leonardlin (2)
- mike-kratz (1)
- estevezdo (1)
- kc-howe (1)
- lm-jkominek (1)
- gokceneraslan (1)
- mrucker (1)
- sergeyf (1)
- tritolol (1)
- niedakh (1)
- nileshpatra (1)
- chrico-bu-uab (1)
- MichaelRFox (1)
Pull Request Authors
- mrucker (2)
- dicorynia (2)
- gokceneraslan (1)
Top Labels
Issue Labels
Pull Request Labels
Packages
- Total packages: 2
-
Total downloads:
- pypi 5,312 last-month
- cran 801 last-month
- Total docker downloads: 20,358
-
Total dependent packages: 5
(may contain duplicates) -
Total dependent repositories: 4
(may contain duplicates) - Total versions: 26
- Total maintainers: 2
pypi.org: genieclust
Genie: Fast and Robust Hierarchical Clustering with Noise Point Detection
- Homepage: https://genieclust.gagolewski.com/
- Documentation: https://genieclust.gagolewski.com/
- License: GNU Affero General Public License v3
-
Latest release: 1.2.0
published 7 months ago
Rankings
Maintainers (1)
cran.r-project.org: genieclust
Fast and Robust Hierarchical Clustering with Noise Point Detection
- Homepage: https://genieclust.gagolewski.com/
- Documentation: http://cran.r-project.org/web/packages/genieclust/genieclust.pdf
- License: AGPL-3
-
Latest release: 1.2.0
published 7 months ago
Rankings
Maintainers (1)
Dependencies
- Rcpp >= 1.0.4 imports
- stats * imports
- utils * imports
- datasets * suggests
- mlpack * suggests
- cython *
- matplotlib *
- nmslib *
- numpy *
- scikit-learn *
- scipy *
- cython *
- matplotlib *
- nmslib *
- numpy *
- scikit-learn *
- scipy *
- actions/checkout v3 composite
- actions/setup-python v3 composite
- actions/upload-artifact v3 composite
- actions/checkout v2 composite
- actions/setup-python v1 composite
- actions/checkout v2 composite