Data

Tools

Indexes

Applications

Experiments

Open Source Science

ExpectationMaximization

A simple but generic implementation of Expectation Maximization algorithms to fit mixture models.

https://github.com/dmetivie/expectationmaximization.jl

Science Score: 44.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ✓
    CITATION.cff file
    Found CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Committers with academic emails
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (10.9%) to scientific vocabulary

Keywords

clustering expectation-maximization gaussian-mixture-models julia mixture-models

Keywords from Contributors

matrix-exponential ida ode numerical finite-volumes symbolic-computation hybrid-differential-equation
Last synced: 6 months ago · JSON representation ·

Repository

A simple but generic implementation of Expectation Maximization algorithms to fit mixture models.

Basic Info
Statistics
  • Stars: 33
  • Watchers: 2
  • Forks: 1
  • Open Issues: 5
  • Releases: 8
Topics
clustering expectation-maximization gaussian-mixture-models julia mixture-models
Created almost 4 years ago · Last pushed 8 months ago
Metadata Files
Readme License Citation Authors

README.md

ExpectationMaximization

ExpectationMaximization.jl

Docs

This package provides a simple implementation of the Expectation Maximization (EM) algorithm used to fit mixture models. Due to Julia's amazing dispatch system, generic and reusable code spirit, and the Distributions.jl package, the code while being very generic is both very expressive and fast! Take a look at the Benchmark section.

What type of mixtures?

In particular, it works on a lot of mixtures:

  • Mixture of Univariate continuous distributions
  • Mixture of Univariate discrete distributions
  • Mixture of Multivariate distributions (continuous or discrete)
  • Mixture of mixtures (univariate or multivariate and continuous or discrete)
  • User defined mixtures (e.g. custom distributions)
  • More?

What EM algorithm?

So far, the classic EM algorithm and the Stochastic EM are implemented. Look at the Bibliography section for references.

How?

Just define a mix::MixtureModel and do fit_mle(mix, y) where y is your observation array (vector or matrix). That's it! For Stochastic EM, just do fit_mle(mix, y, method = StochasticEM()). Take a look at the Examples section.

To work, the only requirements are that the components of the mixture dist ∈ dists = components(mix) considered (custom or coming from an existing package)

  1. Are a subtype of Distribution i.e. dist<:Distribution.
  2. The logpdf(dist, y) is defined (it is used in the E-step)
  3. The fit_mle(dist, y, weights) returns the distribution with the updated parameters maximizing the likelihood. This is used in the M-step of the ClassicalEM algorithm. For the StochasticEM version, only fit_mle(dist, y) is needed. Type or instance version of fit_mle for your dist are accepted thanks to this conversion line.

TODO (feel free to contribute)

  • Add more variants of the EM algorithm (so far there are the classic and stochastic version).

  • Better benchmark against other EM

  • Improve robustness against edge cases (e.g. empty data, empty mixture, etc.). See PR #12 (if you have suggestions, I am still undecided about the best way to handle this).

  • Add advice and better default for atol and rtol choice (it is not obvious how to select them).

  • Speed up code (always!). So far, I focused on readable code.

  • Connecting ExpectationMaximization.jl to MLJ.jl and MLJModels.jl in the clustering algorithm section.

  • Do a proper software paper.

Citation

If you use this package, please cite it with the following biblatex code:

@software{EM.jl-HAL, Author = {David Métivier}, Title = {ExpectationMaximization.jl: A simple but generic implementation of Expectation Maximization algorithms to fit mixture models}, Doi = {hal-04784091}, Url = {https://hal.inrae.fr/hal-04784091}, Copyright = {MIT License} }

For now, it is only on the HAL open archive (that my institute wants me to use) and is linked to a Software Heritage ID SWHID.

Basic usage

Also take a look at the examples section.

julia using Distributions using ExpectationMaximization

Model

```julia N = 50_000 θ₁ = 10 θ₂ = 5 α = 0.2 β = 0.3

Mixture Model here one can put any classical distributions

mix_true = MixtureModel([Exponential(θ₁), Gamma(α, θ₂)], [β, 1 - β])

Generate N samples from the mixture

y = rand(mix_true, N) ```

Inference

```julia

Initial guess

mix_guess = MixtureModel([Exponential(1), Gamma(0.5, 1)], [0.5, 1 - 0.5])

Fit the MLE with the EM algorithm

mixmle = fitmle(mix_guess, y; display = :iter, atol = 1e-3, robust = false, infos = false) ```

Verify results

julia rtol = 5e-2 p = params(mix_mle)[1] # (θ₁, (α, θ₂)) isapprox(β, probs(mix_mle)[1]; rtol = rtol) isapprox(θ₁, p[1]...; rtol = rtol) isapprox(α, p[2][1]; rtol = rtol) isapprox(θ₂, p[2][2]; rtol = rtol)

EM_mixture_example.svg

Owner

  • Name: David Métivier
  • Login: dmetivie
  • Kind: user
  • Location: Montpellier, France
  • Company: INRAe, MISTEA

I am a research scientist with a physics background. Now, I do statistics to tackle environmental, and climate change problems. Julia enthusiast!

Citation (CITATION.bib) 

@software{EM.jl-HAL,
  Author = {David Métivier},
  Title = {ExpectationMaximization.jl: A simple but generic implementation of Expectation Maximization algorithms to fit mixture models},
  Doi = {hal-04784091},
  Url = {https://hal.inrae.fr/hal-04784091},
  Copyright = {MIT License}
}

GitHub Events

Total
  • Create event: 3
  • Commit comment event: 4
  • Release event: 1
  • Watch event: 1
  • Delete event: 1
  • Issue comment event: 4
  • Push event: 26
  • Pull request event: 4
  • Fork event: 1
Last Year
  • Create event: 3
  • Commit comment event: 4
  • Release event: 1
  • Watch event: 1
  • Delete event: 1
  • Issue comment event: 4
  • Push event: 26
  • Pull request event: 4
  • Fork event: 1

Committers

Last synced: over 1 year ago

All Time
  • Total Commits: 94
  • Total Committers: 3
  • Avg Commits per committer: 31.333
  • Development Distribution Score (DDS): 0.074
Past Year
  • Commits: 24
  • Committers: 2
  • Avg Commits per committer: 12.0
  • Development Distribution Score (DDS): 0.083
Top Committers
Name Email Commits
David Métivier 4****e 87
CompatHelper Julia c****y@j****g 5
Tim Holy t****y@g****m 2
Committer Domains (Top 20 + Academic)
julialang.org: 1

Issues and Pull Requests

Last synced: 10 months ago

All Time
  • Total issues: 6
  • Total pull requests: 9
  • Average time to close issues: 6 months
  • Average time to close pull requests: 3 days
  • Total issue authors: 6
  • Total pull request authors: 4
  • Average comments per issue: 7.17
  • Average comments per pull request: 0.89
  • Merged pull requests: 8
  • Bot issues: 0
  • Bot pull requests: 5
Past Year
  • Issues: 0
  • Pull requests: 2
  • Average time to close issues: N/A
  • Average time to close pull requests: 6 days
  • Issue authors: 0
  • Pull request authors: 2
  • Average comments per issue: 0
  • Average comments per pull request: 1.0
  • Merged pull requests: 2
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • dmetivie (1)
  • lrnv (1)
  • jonasmac16 (1)
  • danielinteractive (1)
  • JuliaTagBot (1)
  • timholy (1)
Pull Request Authors
  • github-actions[bot] (5)
  • dmetivie (2)
  • timholy (2)
  • abhro (2)
  • dependabot[bot] (1)
Top Labels
Issue Labels
Pull Request Labels
dependencies (1) github_actions (1)

Packages

  • Total packages: 1
  • Total downloads:
    • julia 9 total
  • Total dependent packages: 1
  • Total dependent repositories: 0
  • Total versions: 12
juliahub.com: ExpectationMaximization

A simple but generic implementation of Expectation Maximization algorithms to fit mixture models.

  • Versions: 12
  • Dependent Packages: 1
  • Dependent Repositories: 0
  • Downloads: 9 Total
Rankings
Dependent repos count: 9.9%
Average: 37.7%
Dependent packages count: 38.9%
Stargazers count: 48.5%
Forks count: 53.5%
Last synced: 6 months ago

Dependencies

.github/workflows/CI.yml actions
  • actions/cache v1 composite
  • actions/checkout v2 composite
  • codecov/codecov-action v1 composite
  • julia-actions/julia-buildpkg v1 composite
  • julia-actions/julia-processcoverage v1 composite
  • julia-actions/julia-runtest v1 composite
  • julia-actions/setup-julia v1 composite
.github/workflows/TagBot.yml actions
  • JuliaRegistries/TagBot v1 composite
.github/workflows/CompatHelper.yml actions
.github/workflows/Documentation.yml actions
  • actions/checkout v2 composite
  • codecov/codecov-action v1 composite
  • julia-actions/julia-processcoverage v1 composite
  • julia-actions/setup-julia latest composite