Data

Tools

Indexes

Applications

Experiments

Open Source Science

infotheory

infotheory: A C++/Python package for multivariate information theoretic analysis - Published in JOSS (2020)

https://github.com/madvn/infotheory

Science Score: 95.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ✓
    DOI references
    Found 4 DOI reference(s) in README and JOSS metadata
  • ✓
    Academic publication links
    Links to: arxiv.org, joss.theoj.org, zenodo.org
  • ✓
    Committers with academic emails
    2 of 3 committers (66.7%) from academic institutions
  • ○
    Institutional organization owner
  • ✓
    JOSS paper metadata
    Published in Journal of Open Source Software

Keywords

entropy information-theory mutual-information partial-information-decomposition
Last synced: 6 months ago · JSON representation

Repository

C++/Python Information theoretic analyses tools

Basic Info
Statistics
  • Stars: 40
  • Watchers: 4
  • Forks: 9
  • Open Issues: 3
  • Releases: 1
Topics
entropy information-theory mutual-information partial-information-decomposition
Created about 7 years ago · Last pushed almost 3 years ago
Metadata Files
Readme License

README.md

Infotheory

Build Status License status DOI

Website: http://mcandadai.com/infotheory/

Introduction

Infotheory, written in C++, and usable in Python as well, is a software package to perform information theoretic analysis on multivariate data. This package implements traditional as well as more recent measures that arise from multivariate extensions to information theory, specifically

  1. Entropy [1]
  2. Mutual Information [2]
  3. Partial Information Decomposition [3]
    • Unique Information
    • Redundant Information
    • Synergistic Information

The main highlights of this package include:

  • being written in C++ for efficiency
  • ease of use via python bindings and compatibility with numpy
  • an API that allows adding the data once to then be able to perform various analyses across different sub-spaces of the dataset very quickly
  • use of sparse data structures that work well with high-dimensional data
  • user-controllable estimation of data distribution using averaged shifted histograms [4]
  • flexibilty to specify binning allows proper estimation of information measures between continuous and discrete variables
  • perform PI-decomposition over 3 (two sources and 1 target) and 4 (three variables and 1 target) variables

The package can be used in Python or C++. While the C++ headers should function well on all platforms, the python package has currently been tested on MacOS and Linux.

Citing

If you use this package, please cite preprint available at: https://arxiv.org/abs/1907.02339

Candadai, M., & Izquierdo, E. J. (2019). infotheory: A C++/Python package for multivariate information theoretic analysis. arXiv preprint arXiv:1907.02339.

  @article{candadai2019infotheory,
    title={infotheory: A C++/Python package for multivariate information theoretic analysis},
    author={Candadai, Madhavun and Izquierdo, Eduardo J},
    journal={arXiv preprint arXiv:1907.02339},
    year={2019}
  }

Installation

pip install infotheory

On MacOS, upgrade to MacOS-Mojave (or later) and update Xcode. You might have to set these environment variables from your terminal.

export CXXFLAGS="-mmacosx-version-min=10.9"
export LDFLAGS="-mmacosx-version-min=10.9"

For C++, simply download InfoTools.h and VectorMatrix.h and include those header files in your code.

Dependencies

The package doesn't have any dependencies per se but the automated tests require numpy

Usage

Using this package in your own code involves the following steps.

See demos and website for sample programs on how to use this package.

Contact

Created by Madhavun Candadai and Eduardo J. Izquierdo. If you have questions or if you've found a bug, please file an issue or feel free to contact Madhavun at madvncv[at]gmail.com

Contribution

If you'd like a feature to be added to infotheory, please file an issue. Or, better yet, open a pull request. We'll work with you to ensure that all pull requests are in a mergable state.

References

  1. http://www.scholarpedia.org/article/Entropy#Shannon_entropy
  2. http://www.scholarpedia.org/article/Mutual_information
  3. Williams, P. L., & Beer, R. D. (2010). Nonnegative decomposition of multivariate information. arXiv preprint arXiv:1004.2515.
  4. Scott, D. W. (1985). Averaged shifted histograms: effective nonparametric density estimators in several dimensions. The Annals of Statistics, 1024-1040.
  5. Timme, N., Alford, W., Flecker, B., & Beggs, J. M. (2014). Synergy, redundancy, and multivariate information measures: an experimentalist’s perspective. Journal of computational neuroscience, 36(2), 119-140.

Owner

  • Name: Madhavun
  • Login: madvn
  • Kind: user
  • Location: Columbus, OH, U.S.A.

CV/AI + Robotics @ Path Robotics | Ph.D. Cognitive Science, Indiana University, Bloomington, 2020.

JOSS Publication

infotheory: A C++/Python package for multivariate information theoretic analysis
Published
March 09, 2020
DOI
Volume 5, Issue 47, Page 1609
Authors
Madhavun Candadai ORCID
Cognitive Science Program, Indiana University, Bloomington, IN, U.S.A., School of Informatics, Computing, and Engineering, Indiana University, Bloomington, IN, U.S.A.
Eduardo J. Izquierdo ORCID
Cognitive Science Program, Indiana University, Bloomington, IN, U.S.A., School of Informatics, Computing, and Engineering, Indiana University, Bloomington, IN, U.S.A.
Editor
Jack Poulson ORCID
Tags
information theory multivariate data analysis entropy mutual information partial information decomposition
View PDF Review Thread Software Archive

Papers & Mentions

Total mentions: 1

Sources of predictive information in dynamical neural networks
Last synced: 4 months ago

GitHub Events

Total
  • Watch event: 2
Last Year
  • Watch event: 2

Committers

Last synced: 7 months ago

All Time
  • Total Commits: 81
  • Total Committers: 3
  • Avg Commits per committer: 27.0
  • Development Distribution Score (DDS): 0.198
Past Year
  • Commits: 0
  • Committers: 0
  • Avg Commits per committer: 0.0
  • Development Distribution Score (DDS): 0.0
Top Committers
Name Email Commits
madvn m****v@g****m 65
Madhavun Candadai m****a@i****u 14
Daniel S. Katz d****z@i****g 2
Committer Domains (Top 20 + Academic)
ieee.org: 1 indiana.edu: 1

Issues and Pull Requests

Last synced: 6 months ago

All Time
  • Total issues: 6
  • Total pull requests: 3
  • Average time to close issues: 5 days
  • Average time to close pull requests: 12 minutes
  • Total issue authors: 6
  • Total pull request authors: 2
  • Average comments per issue: 1.33
  • Average comments per pull request: 0.0
  • Merged pull requests: 2
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 0
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • impredicative (1)
  • annecvdw (1)
  • perevo (1)
  • artemyk (1)
  • buhrmann (1)
  • manolomartinez (1)
Pull Request Authors
  • danielskatz (2)
  • jcussen (1)
Top Labels
Issue Labels
Pull Request Labels

Packages

  • Total packages: 1
  • Total downloads:
    • pypi 116 last-month
  • Total dependent packages: 0
  • Total dependent repositories: 1
  • Total versions: 1
  • Total maintainers: 1
pypi.org: infotheory

Information theoretic analysis tools

  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 1
  • Downloads: 116 Last month
Rankings
Dependent packages count: 10.0%
Average: 21.1%
Dependent repos count: 21.7%
Downloads: 31.5%
Maintainers (1)
madhavun
Last synced: 6 months ago