EnsembleKalmanProcesses.jl

EnsembleKalmanProcesses.jl: Derivative-free ensemble-based model calibration - Published in JOSS (2022)

https://github.com/clima/ensemblekalmanprocesses.jl

Science Score: 98.0%

This score indicates how likely this project is to be science-related based on various indicators:

○
CITATION.cff file
✓
codemeta.json file
Found codemeta.json file
✓
.zenodo.json file
Found .zenodo.json file
✓
DOI references
Found 5 DOI reference(s) in README and JOSS metadata
✓
Academic publication links
Links to: joss.theoj.org, zenodo.org
✓
Committers with academic emails
5 of 31 committers (16.1%) from academic institutions
✓
Institutional organization owner
Organization clima has institutional domain (clima.caltech.edu)
✓
JOSS paper metadata
Published in Journal of Open Source Software

Keywords from Contributors

pde climate ocean graphics climate-change data-assimilation fluid-dynamics meshing ode dynamical-systems

Scientific Fields

Economics Social Sciences - 40% confidence

Engineering Computer Science - 40% confidence

Last synced: 4 months ago · JSON representation

Repository

Implements Optimization and approximate uncertainty quantification algorithms, Ensemble Kalman Inversion, and Ensemble Kalman Processes.

Basic Info

Host: GitHub
Owner: CliMA
License: apache-2.0
Language: Julia
Default Branch: main
Homepage: https://clima.github.io/EnsembleKalmanProcesses.jl/dev
Size: 39.1 MB

Statistics

Stars: 106
Watchers: 4
Forks: 21
Open Issues: 38
Releases: 47

Created about 5 years ago · Last pushed 4 months ago

Metadata Files

Readme License

EnsembleKalmanProcesses.jl

Implements optimization and approximate uncertainty quantification algorithms, Ensemble Kalman Inversion, and other Ensemble Kalman Processes.

| Documentation | | |-----------------------------------|--------------------------------------------------| | DOI | | | Docs Build | | | Unit tests | | | Code Coverage | | | JOSS | | | Downloads | |

Requirements

Julia LTS version or newer

What does the package do?

EnsembleKalmanProcesses (EKP) enables users to find an (locally-) optimal parameter set u for a computer code G to fit some (noisy) observational data y. It uses a suite of methods from the Ensemble Kalman filtering literature that have a long history of success in the weather forecasting community.

What makes EKP different? - EKP algorithms are efficient (complexity doesn't strongly scale with number of parameters), and can optimize with noisy and complex parameter-to-data landscapes. - We don't require differentiating the model G at all! you just need to be able to run it at different parameter configurations. - We don't even require G to be coded up in Julia! - Ensemble model evaluations are fully parallelizable - so we can exploit our HPC systems capabilities! - We provide some lego-like interfaces for creating complex priors and observations. - We provied easy interfaces to toggle between many different algorithms and configurable features.

What does it look like to use?

Below we will outline the current user experience for using EnsembleKalmanProcesses.jl. Copy-paste the snippets to reproduce the results (up to random number generation).

We solve the classic inverse problem where we learn y = G(u), noisy forward map G distributed as N(0,Γ). For example, julia using LinearAlgebra G(u) = [ 1/abs(u[1]), sum(u[2:5]), prod(u[3:4]), u[1]^2-u[2]-u[3], u[4], u[5]^3, ] .+ 0.1*randn(6) true_u = [3, 1, 2,-3,-4] y = G(true_u) Γ = (0.1)^2*I We assume some prior knowledge of the parameters u in the problem (such as approximate scales, and the first parameter being positive), then we are ready to go!

```julia using EnsembleKalmanProcesses using EnsembleKalmanProcesses.ParameterDistributions

prioru1 = constrainedgaussian("positivewithmean2", 2, 1, 0, Inf) prioru2 = constrainedgaussian("fourwithspread5", 0, 5, -Inf, Inf, repeats=4) prior = combinedistributions([prioru1, prior_u2])

Nensemble = 50 initialensemble = constructinitialensemble(prior, Nensemble) ensemblekalmanprocess = EnsembleKalmanProcess( initialensemble, y, Γ, Inversion(), verbose=true)

Niterations = 10 for i in 1:Niterations paramsi = getϕfinal(prior, ensemblekalman_process)

G_matrix = hcat(
    [G(params_i[:, i]) for i in 1:N_ensemble]... # Parallelize here!
)

update_ensemble!(ensemble_kalman_process, G_matrix)

end

finalsolution = getϕmeanfinal(prior, ensemblekalmanprocess)

Let's see what's going on!

using Plots p = plot(prior) for (i,sp) in enumerate(p.subplots) vline!(sp, [trueu[i]], lc="black", lw=4) vline!(sp, [finalsolution[i]], lc="magenta", lw=4) end display(p) ``` quick-readme-example

See a similar working example here!. Check out our many example scripts above in examples/

Quick links!

Citing us

If you use the examples or code, please cite our article at JOSS in your published materials.

Getting Started

eki-getting-started <!---

Link to Miro for editing photo (ask haakon for access): https://miro.com/app/board/uXjVNm1teY=/?sharelink_id=329380184889

-->

Owner

Name: Climate Modeling Alliance
Login: CliMA
Kind: organization
Email: clima@caltech.edu

Website: https://clima.caltech.edu
Repositories: 67
Profile: https://github.com/CliMA

An alliance of scientists, engineers and applied mathematicians, dedicated to pioneering a new, data-informed approach to climate modeling

JOSS Publication

EnsembleKalmanProcesses.jl: Derivative-free ensemble-based model calibration

Published

December 15, 2022

DOI

10.21105/joss.04869

Volume 7, Issue 80, Page 4869

Authors

Oliver R. a. Dunbar

Division of Geological and Planetary Sciences, California Institute of Technology

Ignacio Lopez-Gomez

Division of Geological and Planetary Sciences, California Institute of Technology

Alfredo Garbuno-Iñigo

Department of Statistics, Mexico Autonomous Institute of Technology

Daniel Zhengyu Huang

Division of Geological and Planetary Sciences, California Institute of Technology

Eviatar Bach

Division of Geological and Planetary Sciences, California Institute of Technology

Jin-long Wu

Department of Mechanical Engineering, University of Wisconsin-Madison

Editor

Mehmet Hakan Satman

GitHub Events

Total

Create event: 57
Commit comment event: 19
Release event: 9
Issues event: 63
Watch event: 23
Delete event: 6
Issue comment event: 66
Push event: 782
Pull request review comment event: 51
Pull request review event: 58
Pull request event: 108
Fork event: 3

Last Year

Create event: 57
Commit comment event: 19
Release event: 9
Issues event: 63
Watch event: 23
Delete event: 6
Issue comment event: 66
Push event: 782
Pull request review comment event: 51
Pull request review event: 58
Pull request event: 108
Fork event: 3

Committers

Last synced: 5 months ago

All Time

Total Commits: 532
Total Committers: 31
Avg Commits per committer: 17.161
Development Distribution Score (DDS): 0.592

Past Year

Commits: 55
Committers: 7
Avg Commits per committer: 7.857
Development Distribution Score (DDS): 0.145

Top Committers

Name	Email	Commits
odunbar	o**r@c**u	217
Navid C. Constantinou	n****y	90
ilopezgp	i**p@g**m	84
Melanie	m**e@c**l	20
Jinlong	j**g@J**t	19
agarbuno	a**o@i**x	14
Jake Bolewski	j**i@g**m	12
CompatHelper Julia	c**y@j**g	11
Eviatar Bach	e**h@p**m	9
mhowlan3	m**3@g**m	8
Thomas Jackson	t**4@g**m	6
costachris	c**a@g**m	5
Daniel-Huang	d**g@c**u	4
Nat Efrat-Henrici	6****i	4
Haakon Ludvig Langeland Ervik	4****e	3
Kevin Phan	9****v	3
github-actions[bot]	4****]	3
sydneyvernon	5****n	3
Gregory L. Wagner	g**r@g**m	2
Daniel Z. Huang	h**4@g**m	2
Mehmet Hakan Satman	m**n@g**m	2
bielim	b****m	2
Anna Jaruga	a**a@c**u	1
Pietro Monticone	3****e	1
Rebecca Gjini	8****i	1
Simon Byrne	s**e@g**m	1
Jinlong	j**g@J**l	1
Costa Christopoulos	c**s@C**l	1
Tomas Chor	t**r@g**m	1
Wilson Duan	w**n@c**u	1
and 1 more...

Committer Domains (Top 20 + Academic)

caltech.edu: 4 julialang.org: 1 itam.mx: 1 jinlongs-imac.attlocal.net: 1 charney.bieli.email: 1

Issues and Pull Requests

Last synced: 4 months ago

All Time

Total issues: 139
Total pull requests: 229
Average time to close issues: 3 months
Average time to close pull requests: 11 days
Total issue authors: 24
Total pull request authors: 19
Average comments per issue: 1.14
Average comments per pull request: 1.73
Merged pull requests: 195
Bot issues: 1
Bot pull requests: 32

Past Year

Issues: 49
Pull requests: 102
Average time to close issues: about 1 month
Average time to close pull requests: 8 days
Issue authors: 9
Pull request authors: 9
Average comments per issue: 0.2
Average comments per pull request: 0.69
Merged pull requests: 77
Bot issues: 1
Bot pull requests: 21

View more stats

Top Authors

Issue Authors

odunbar (91)
nefrathenrici (6)
ilopezgp (5)
haakon-e (5)
eviatarbach (4)
bielim (3)
costachris (2)
Julians42 (2)
sdwfrost (2)
ph-kev (2)
wilsonduan10 (2)
Sbozzolo (2)
glwagner (1)
odow (1)
bischtob (1)

Pull Request Authors

odunbar (158)
github-actions[bot] (27)
ilopezgp (14)
dependabot[bot] (12)
eviatarbach (9)
nefrathenrici (6)
costachris (6)
ph-kev (5)
sydneyvernon (4)
wilsonduan10 (2)
glwagner (2)
jbytecode (2)
ArneBouillon (2)
haakon-e (2)
rgjini (1)

Top Labels

Issue Labels

I'm a bug - please fix me (8) enhancement (6) documentation (5) SDI (3) help wanted (2) stale dependencies (2) bug (2) robustness (1)

Pull Request Labels

stale dependencies (11) do not merge (3) documentation (1) bug (1) dependencies (1) github_actions (1)

Packages

Total packages: 1
Total downloads:
- julia 66 total

Total dependent packages: 8
Total dependent repositories: 0
Total versions: 46

juliahub.com: EnsembleKalmanProcesses

Implements Optimization and approximate uncertainty quantification algorithms, Ensemble Kalman Inversion, and Ensemble Kalman Processes.

Homepage: https://clima.github.io/EnsembleKalmanProcesses.jl/dev
Documentation: https://docs.juliahub.com/General/EnsembleKalmanProcesses/stable/
License: Apache-2.0
Latest release: 2.5.0
published 6 months ago

Versions: 46
Dependent Packages: 8
Dependent Repositories: 0
Downloads: 66 Total

Rankings

Forks count: 9.8%

Dependent repos count: 9.9%

Average: 10.9%

Dependent packages count: 11.0%

Stargazers count: 12.9%

Last synced: 4 months ago

Dependencies

.github/workflows/DocPreviewCleanup.yml actions

actions/checkout v2 composite

.github/workflows/Docs.yml actions

actions/cache v1 composite
actions/checkout v2 composite
julia-actions/setup-julia v1 composite
styfle/cancel-workflow-action 0.9.1 composite

.github/workflows/JuliaFormatter.yml actions

actions/checkout v2 composite
dorny/paths-filter v2.9.1 composite
julia-actions/setup-julia latest composite
styfle/cancel-workflow-action 0.9.1 composite

.github/workflows/TagBot.yml actions

JuliaRegistries/TagBot v1 composite

.github/workflows/Tests.yml actions

actions/checkout v2 composite
codecov/codecov-action v1 composite
julia-actions/julia-buildpkg v1 composite
julia-actions/julia-runtest v1 composite
julia-actions/setup-julia v1 composite
styfle/cancel-workflow-action 0.9.1 composite

.github/workflows/CompatHelper.yml actions

EnsembleKalmanProcesses.jl

Science Score: 98.0%

Keywords from Contributors

Scientific Fields

Repository

Basic Info

Statistics

Metadata Files

README.md

EnsembleKalmanProcesses.jl

Requirements

What does the package do?

What does it look like to use?

Let's see what's going on!

Quick links!

Citing us

Getting Started

Link to Miro for editing photo (ask haakon for access): https://miro.com/app/board/uXjVNm1teY=/?sharelink_id=329380184889

Owner

JOSS Publication

EnsembleKalmanProcesses.jl: Derivative-free ensemble-based model calibration

Authors

Editor

Tags

GitHub Events

Total

Last Year

Committers

All Time

Past Year

Top Committers

Committer Domains (Top 20 + Academic)

Issues and Pull Requests

All Time

Past Year

Top Authors

Issue Authors

Pull Request Authors

Top Labels

Issue Labels

Pull Request Labels

Packages

juliahub.com: EnsembleKalmanProcesses

Rankings

Dependencies