distfit

distfit is a python library for probability density fitting.

Science Score: 54.0%

This score indicates how likely this project is to be science-related based on various indicators:

✓
CITATION.cff file
Found CITATION.cff file
✓
codemeta.json file
Found codemeta.json file
✓
.zenodo.json file
Found .zenodo.json file
○
DOI references
✓
Academic publication links
Links to: zenodo.org
○
Committers with academic emails
○
Institutional organization owner
○
JOSS paper metadata
○
Scientific vocabulary similarity
Low similarity (12.1%) to scientific vocabulary

Keywords

cumulative-distribution-function density-functions fitting-curve hypothesis-testing kolmogorov-smirnov pdf plot probability-distribution probability-statistics pypi qqplot sse

Last synced: 4 months ago · JSON representation ·

Repository

distfit is a python library for probability density fitting.

Basic Info

Host: GitHub
Owner: erdogant
License: other
Language: Jupyter Notebook
Default Branch: master
Homepage: https://erdogant.github.io/distfit
Size: 75.7 MB

Statistics

Stars: 400
Watchers: 7
Forks: 30
Open Issues: 12
Releases: 56

Topics

cumulative-distribution-function density-functions fitting-curve hypothesis-testing kolmogorov-smirnov pdf plot probability-distribution probability-statistics pypi qqplot sse

Created almost 6 years ago · Last pushed 4 months ago

Metadata Files

Readme Funding License Citation

distfit is a Python package for probability density fitting of univariate distributions for random variables. The distfit library can determine the best fit for over 90 theoretical distributions. The goodness-of-fit test is used to score for the best fit and after finding the best-fitted theoretical distribution, the loc, scale, and arg parameters are returned. It can be used for parametric, non-parametric, and discrete distributions. ⭐️Star it if you like it⭐️

Key Features

| Feature | Description | Docs | Medium | Gumroad+Podcast| |---------|-------------|---------------|--------|---------| | Parametric Fitting | Fit distributions on empirical data X. | Link | Link | Link | | Non-Parametric Fitting | Fit distributions on empirical data X using non-parametric approaches (quantile, percentiles). | Link | - | - | | Discrete Fitting | Fit distributions on empirical data X using binomial distribution. | Link | - | - | | Predict | Compute probabilities for response variables y. | Link | - | - | | Outlier Detection | Detect anomalies using fitted distributions. | Link | Link | Link | | Synthetic Data | Generate synthetic data. | Link | Link | Link | | Plots | Various plotting functionalities. | Link | - | - |

Resources and Links

Example Notebooks: Examples
Medium Blogs Medium
Gumroad Blogs with podcast: GumRoad
Documentation: Website
Bug Reports and Feature Requests: GitHub Issues

Background

For the parametric approach, The distfit library can determine the best fit across 89 theoretical distributions. To score the fit, one of the scoring statistics for the good-of-fitness test can be used used, such as RSS/SSE, Wasserstein, Kolmogorov-Smirnov (KS), or Energy. After finding the best-fitted theoretical distribution, the loc, scale, and arg parameters are returned, such as mean and standard deviation for normal distribution.
For the non-parametric approach, the distfit library contains two methods, the quantile and percentile method. Both methods assume that the data does not follow a specific probability distribution. In the case of the quantile method, the quantiles of the data are modeled whereas for the percentile method, the percentiles are modeled.
In case the dataset contains discrete values, the distift library contains the option for discrete fitting. The best fit is then derived using the binomial distribution.

Installation

Install distfit from PyPI

bash pip install distfit

Install from Github source

bash pip install git+https://github.com/erdogant/distfit

Imort Library

```python import distfit print(distfit.version)

Import library

from distfit import distfit ```

Examples

Example: Quick start to find best fit for your input data

```python

[distfit] >INFO> fit

[distfit] >INFO> transform

[distfit] >INFO> [norm ] [0.00 sec] [RSS: 0.00108326] [loc=-0.048 scale=1.997]

[distfit] >INFO> [expon ] [0.00 sec] [RSS: 0.404237] [loc=-6.897 scale=6.849]

[distfit] >INFO> [pareto ] [0.00 sec] [RSS: 0.404237] [loc=-536870918.897 scale=536870912.000]

[distfit] >INFO> [dweibull ] [0.06 sec] [RSS: 0.0115552] [loc=-0.031 scale=1.722]

[distfit] >INFO> [t ] [0.59 sec] [RSS: 0.00108349] [loc=-0.048 scale=1.997]

[distfit] >INFO> [genextreme] [0.17 sec] [RSS: 0.00300806] [loc=-0.806 scale=1.979]

[distfit] >INFO> [gamma ] [0.05 sec] [RSS: 0.00108459] [loc=-1862.903 scale=0.002]

[distfit] >INFO> [lognorm ] [0.32 sec] [RSS: 0.00121597] [loc=-110.597 scale=110.530]

[distfit] >INFO> [beta ] [0.10 sec] [RSS: 0.00105629] [loc=-16.364 scale=32.869]

[distfit] >INFO> [uniform ] [0.00 sec] [RSS: 0.287339] [loc=-6.897 scale=14.437]

[distfit] >INFO> [loggamma ] [0.12 sec] [RSS: 0.00109042] [loc=-370.746 scale=55.722]

[distfit] >INFO> Compute confidence intervals [parametric]

[distfit] >INFO> Compute significance for 9 samples.

[distfit] >INFO> Multiple test correction method applied: [fdr_bh].

[distfit] >INFO> Create PDF plot for the parametric method.

[distfit] >INFO> Mark 5 significant regions

[distfit] >INFO> Estimated distribution: beta [loc:-16.364265, scale:32.868811]

```

Example: Plot summary of the tested distributions

After we have a fitted model, we can make some predictions using the theoretical distributions. After making some predictions, we can plot again but now the predictions are automatically included.

Example: Make predictions using the fitted distribution

Example: Test for one specific distributions

The full list of distributions is listed here: https://erdogant.github.io/distfit/pages/html/Parametric.html

Example: Test for multiple distributions

The full list of distributions is listed here: https://erdogant.github.io/distfit/pages/html/Parametric.html

Example: Fit discrete distribution

```python from scipy.stats import binom

Generate random numbers

Set parameters for the test-case

n = 8 p = 0.5

Generate 10000 samples of the distribution of (n, p)

X = binom(n, p).rvs(10000) print(X)

[5 1 4 5 5 6 2 4 6 5 4 4 4 7 3 4 4 2 3 3 4 4 5 1 3 2 7 4 5 2 3 4 3 3 2 3 5

4 6 7 6 2 4 3 3 5 3 5 3 4 4 4 7 5 4 5 3 4 3 3 4 3 3 6 3 3 5 4 4 2 3 2 5 7

5 4 8 3 4 3 5 4 3 5 5 2 5 6 7 4 5 5 5 4 4 3 4 5 6 2...]

Import distfit

from distfit import distfit

Initialize for discrete distribution fitting

dfit = distfit(method='discrete')

Run distfit to and determine whether we can find the parameters from the data.

dfit.fit_transform(X)

[distfit] >fit..

[distfit] >transform..

[distfit] >Fit using binomial distribution..

[distfit] >[binomial] [SSE: 7.79] [n: 8] [p: 0.499959] [chi^2: 1.11]

[distfit] >Compute confidence interval [discrete]

```

Example: Make predictions on unseen data for discrete distribution

Example: Generate samples based on the fitted distribution

Star history

Contributors

Thank the contributors!

Maintainer

Erdogan Taskesen, github: erdogant
Contributions are welcome.
Yes! This library is entirely free but it runs on coffee! :) Feel free to support with a Coffee.

Owner

Name: Erdogan
Login: erdogant
Kind: user
Location: Den Haag

Website: https://erdogant.github.io/
Repositories: 51
Profile: https://github.com/erdogant

Machine Learning | Statistics | Bayesian | D3js | Visualizations

Citation (CITATION.cff)

# YAML 1.2
---
authors: 
  -
    family-names: Taskesen
    given-names: Erdogan
    orcid: "https://orcid.org/0000-0002-3430-9618"
cff-version: "1.1.0"
date-released: 2020-01-02
keywords: 
  - "probability-distribution"
  - "hypothesis-testing"
  - "probability-statistics"
  - "density-functions"
  - "fitting-curve"
license: "MIT"
message: "If you use this software, please cite it using these metadata."
repository-code: "https://erdogant.github.io/distfit"
title: "distfit is a python library for probability density fitting."
version: "1.4.0"
...

GitHub Events

Total

Create event: 6
Release event: 7
Issues event: 2
Watch event: 35
Issue comment event: 12
Push event: 41
Fork event: 5

Last Year

Create event: 6
Release event: 7
Issues event: 2
Watch event: 35
Issue comment event: 12
Push event: 41
Fork event: 5

Committers

Last synced: 9 months ago

All Time

Total Commits: 516
Total Committers: 8
Avg Commits per committer: 64.5
Development Distribution Score (DDS): 0.017

Past Year

Commits: 13
Committers: 2
Avg Commits per committer: 6.5
Development Distribution Score (DDS): 0.154

Top Committers

Name	Email	Commits
erdogant	e**t@g**m	507
maxmekiska	m**a@g**m	2
Killaars	2****s	2
taskesene	e**n@r**l	1
ksachdeva	k**7@g**m	1
Matthew Scheffel	m**t@d**m	1
KrystofS	5****S	1
Grozby	g**i@b**m	1

Committer Domains (Top 20 + Academic)

beckman.com: 1 dataheck.com: 1 rws.nl: 1

Issues and Pull Requests

Last synced: 4 months ago

All Time

Total issues: 45
Total pull requests: 6
Average time to close issues: 3 months
Average time to close pull requests: 2 days
Total issue authors: 39
Total pull request authors: 6
Average comments per issue: 2.87
Average comments per pull request: 2.17
Merged pull requests: 6
Bot issues: 0
Bot pull requests: 0

Past Year

Issues: 1
Pull requests: 0
Average time to close issues: N/A
Average time to close pull requests: N/A
Issue authors: 1
Pull request authors: 0
Average comments per issue: 0.0
Average comments per pull request: 0
Merged pull requests: 0
Bot issues: 0
Bot pull requests: 0

View more stats

Top Authors

Issue Authors

tirthajyoti (5)
bbxudavid (2)
KrystofS (2)
valentynbez (1)
lesshaste (1)
hummm310 (1)
JulianFerry (1)
francoisdejgr (1)
Buedenbender (1)
danishTUE (1)
magicaltoast (1)
jamesee (1)
marcellobullo (1)
fkiraly (1)
wanglinxiao (1)

Pull Request Authors

maxmekiska (2)
KrystofS (1)
erdogant (1)
Grozby (1)
ksachdeva (1)
isosphere (1)

Top Labels

Issue Labels

enhancement (2) documentation (1)

Pull Request Labels

Packages

Total packages: 2
Total downloads:
- pypi 17,727 last-month
Total docker downloads: 33

Total dependent packages: 8
(may contain duplicates)
Total dependent repositories: 14
(may contain duplicates)
Total versions: 59
Total maintainers: 1

pypi.org: distfit

distfit is a Python library for probability density fitting.

Homepage: https://erdogant.github.io/distfit
Documentation: https://distfit.readthedocs.io/
License: MIT License Copyright (c) 2020 Erdogan Taskesen distfit - Python package for probability distribution fitting and hypothesis testing. Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Latest release: 1.8.8
published 6 months ago

Versions: 55
Dependent Packages: 8
Dependent Repositories: 14
Downloads: 17,727 Last month
Docker Downloads: 33

Rankings

Downloads: 1.3%

Dependent packages count: 1.6%

Docker downloads count: 3.0%

Stargazers count: 3.6%

Average: 3.6%

Dependent repos count: 3.9%

Forks count: 8.3%

Maintainers (1)

erdogant

Last synced: 4 months ago

conda-forge.org: distfit

Distfit is a python package for probability density fitting across 89 univariate distributions to non-censored data by residual sum of squares (RSS), and hypothesis testing. Probability density fitting is the fitting of a probability distribution to a series of data concerning the repeated measurement of a variable phenomenon. Distfit scores each of the 89 different distributions for the fit with the empirical distribution and return the best scoring distribution.

Homepage: https://github.com/erdogant/distfit
License: MIT
Latest release: 1.4.5
published over 3 years ago

Versions: 4
Dependent Packages: 0
Dependent Repositories: 0

Rankings

Stargazers count: 24.9%

Dependent repos count: 34.0%

Average: 37.7%

Forks count: 40.9%

Dependent packages count: 51.2%

Last synced: 4 months ago

Dependencies

docs/source/requirements.txt pypi

pipinstallsphinx_rtd_theme *

requirements-dev.txt pypi

irelease * development
pytest * development
rst2pdf * development
sphinx * development
sphinx_rtd_theme * development
sphinxcontrib-fulltoc * development
spyder-kernels ==2.2.1 development

requirements.txt pypi

matplotlib *
numpy *
pandas *
pypickle *
scipy *
statsmodels *
tqdm *

setup.py pypi

matplotlib *
numpy *
pandas *
pypickle *
scipy *
statsmodels *
tqdm *

.github/workflows/codeql-analysis.yml actions

actions/checkout v2 composite
github/codeql-action/analyze v1 composite
github/codeql-action/autobuild v1 composite
github/codeql-action/init v1 composite

.github/workflows/pytest.yml actions

actions/checkout v2 composite
conda-incubator/setup-miniconda v2 composite