pcmdi_metrics

Open-source Python package for Systematic Evaluation of Climate and Earth System Models

Science Score: 85.0%

This score indicates how likely this project is to be science-related based on various indicators:

✓
CITATION.cff file
Found CITATION.cff file
✓
codemeta.json file
Found codemeta.json file
✓
.zenodo.json file
Found .zenodo.json file
✓
DOI references
Found 9 DOI reference(s) in README
✓
Academic publication links
Links to: scholar.google, zenodo.org
✓
Committers with academic emails
27 of 40 committers (67.5%) from academic institutions
✓
Institutional organization owner
Organization pcmdi has institutional domain (pcmdi.llnl.gov)
○
JOSS paper metadata
○
Scientific vocabulary similarity
Low similarity (15.5%) to scientific vocabulary

Keywords

climate climate-analysis climate-data climate-model climate-model-evaluation climate-models climate-science climate-variability python

Keywords from Contributors

annotation degoogle energy-systems

Last synced: 6 months ago · JSON representation ·

Repository

Open-source Python package for Systematic Evaluation of Climate and Earth System Models

Basic Info

Host: GitHub
Owner: PCMDI
License: bsd-3-clause
Language: Python
Default Branch: main
Homepage: http://pcmdi.github.io/pcmdi_metrics/
Size: 241 MB

Statistics

Stars: 117
Watchers: 9
Forks: 44
Open Issues: 45
Releases: 47

Topics

climate climate-analysis climate-data climate-model climate-model-evaluation climate-models climate-science climate-variability python

Created over 12 years ago · Last pushed 6 months ago

Metadata Files

Readme Contributing License Code of conduct Citation Support

PCMDI Metrics Package (PMP)

Conda-forge (CURRENT, recommended):

PCMDI Conda Channel (old, deprecated):

The PCMDI Metrics Package (PMP) is used to provide "quick-look" objective comparisons of Earth System Models (ESMs) with one another and available observations. Results are produced in the context of all model simulations contributed to CMIP6 and earlier CMIP phases. Among other purposes, this enables modeling groups to evaluate changes during the development cycle in the context of the structural error distribution of the multi-model ensemble. Currently, the comparisons emphasize metrics of large- to global-scale annual cycle, tropical and extra-tropical modes of variability, ENSO, MJO, regional monsoons, high frequency characteristics of simulated precipitation, and cloud feedback.

PCMDI uses the PMP to produce quick-look simulation summaries across generations of CMIP.

The metrics package consists of the following parts: * Analysis software * Observation-based reference database of global (or near global, land or ocean) time series and climatologies * Package documentation and interactive jupyter notebook demos * Database of performance metrics computed for CMIP models

The package expects model data to be CF-compliant. To successfully use the package some input data "conditioning" may be required. We provide several demo scripts within the package.

Documentation

Getting Started

Installation requirements and instructions are available on the Install page
Users will need to contact the PMP developers (pcmdi-metrics@llnl.gov) to obtain supporting datasets and get started using the package.
An overview for using the package and template scripts are detailed on the Using-the-package page
View Demo

References

Latest:

Lee, J., Gleckler, P. J., Ahn, M.-S., Ordonez, A., Ullrich, P. A., Sperber, K. R., Taylor, K. E., Planton, Y. Y., Guilyardi, E., Durack, P., Bonfils, C., Zelinka, M. D., Chao, L.-W., Dong, B., Doutriaux, C., Zhang, C., Vo, T., Boutte, J., Wehner, M. F., Pendergrass, A. G., Kim, D., Xue, Z., Wittenberg, A. T., and Krasting, J.: Systematic and objective evaluation of Earth system models: PCMDI Metrics Package (PMP) version 3, Geosci. Model Dev., 17, 3919–3948, https://doi.org/10.5194/gmd-17-3919-2024, 2024.

Earlier versions:

Gleckler, P. J., Doutriaux, C., Durack, P. J., Taylor, K. E., Zhang, Y., Williams, D. N., Mason, E., and Servonnat, J.: A more powerful reality test for climate models, Eos T. Am. Geophys. Un., 97, https://doi.org/10.1029/2016eo051663, 2016.
Gleckler, P. J., Taylor, K. E., and Doutriaux, C.: Performance metrics for climate models, J. Geophys. Res., 113, D06104, https://doi.org/10.1029/2007jd008972, 2008.

Contact

Report Bug

Request Feature

Some installation support for CMIP participating modeling groups is available: pcmdi-metrics@llnl.gov

Acknowledgement

Content in this repository is developed by climate and computer scientists from the Program for Climate Model Diagnosis and Intercomparison (PCMDI) at Lawrence Livermore National Laboratory (LLNL). This work is sponsored by the Regional and Global Model Analysis (RGMA) program, of the Earth and Environmental Systems Sciences Division (EESSD) in the Office of Biological and Environmental Research (BER) within the Department of Energy's Office of Science. The work is performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344.

LLNL-CODE-2004137

DOE CODE ID: #153383

United States Department of Energy Lawrence Livermore National Laboratory

License

Distributed under the BSD 3-Clause License. See LICENSE for more information.

Release Notes and History

v3.9.1 | v3.9 | v3.8.2 | v3.8.1 | v3.8 | v3.7.2 | v3.7.1 | v3.7 | v3.6.1 | v3.6 | v3.5.2 | v3.5.1 | v3.5 | v3.4.1 | v3.4 | v3.3.4 | v3.3.3 | v3.3.2 | v3.3.1 | v3.3 | v3.2 | v3.1.2 | v3.1.1 | v3.1 | v3.0.2 | v3.0.1 | v3.0.0 s">Versions

| Update summary | | New capability (new modes for modes of variability metrics: EA, SCA) and technical update | New capability (Decision-Relevant metrics, Database access API) and new demo notebooks | Technical update | Technical update with new figure (modes of variability multi-panel plot) | New capability (figure generation for ENSO, xCDAT migration completed for Monsoon Wang with figure generation), major dependency update (numpy >= 2.0) | Technical update | Technical update with documentation improvements | New capability (figure generation for mean climate) and technical update | Technical update, additional QC repair functions | New capability (regional application of precip variability) and technical update | New capability (QC, new modes for modes of variability metrics: PSA1, PSA2) and technical update | Technical update | Technical update: MJO and Monsoon Sperber xCDAT conversion | Technical update | Technical update: Modes of variability xCDAT conversion | Technical update | Technical update | Technical update | Technical update | New metric added: Sea-Ice | New metric added: Extremes | Technical update | Technical and documentation update | New metric added: Precipitation Benchmarking -- distribution bimodality | Minor patch and more documentation added | Minor technical patch
| New metric added: Cloud feedback metric by @mzelinka. xCDAT implemented for mean climate metrics

Click here for older versions

[Versions]

| Update summary | | ------------- | ------------------------------------- | | [v2.5.1] | Technical update | [v2.5.0] | New metric added: **Precipitation Benchmarking -- distribution**. Graphics updated | [v2.4.0] | New metric added: **AMO** in variability modes | [v2.3.2] | CMEC interface updates | [v2.3.1] | Technical update | [v2.3] | New graphics using [archived PMP results](https://github.com/PCMDI/pcmdi_metrics_results_archive) | [v2.2.2] | Technical update | [v2.2.1] | Minor update | [v2.2] | New metric implemented: **precipitation variability across time scale** | [v2.1.2] | Minor update | [v2.1.1] | Simplified dependent libraries and CI process | [v2.1.0] | [**CMEC**](https://cmec.llnl.gov/) driver interfaced added. | [v2.0] | New capabilities: **ENSO** metrics, demos, and documentations. | [v1.2] | Tied to CDAT 8.0. Extensive regression testing added. New metrics: **Diurnal cycle and intermittency of precipitation**, sample **monsoon** metrics. | [v1.1.2] | Now managed through Anaconda, and tied to UV-CDAT 2.10. Weights on bias statistic added. Extensive provenance information incorporated into json files. | [v1.1] | First public release, emphasizing **climatological statistics**, with development branches for ENSO and regional monsoon precipitation indices | [v1.0] | Prototype version of the PMP

Current Core Development Team

All Contributors

Thanks goes to these wonderful people (emoji key):

_{Jiwoo Lee} 💻 📖 👀 ⚠️ ✅ 🔬 🤔 🚇	_{Peter Gleckler} 💻 📖 🔬 👀 ⚠️ 🔣 🤔	_{Kristin Chang} 💻	_{Ana Ordonez} 💻 📖 👀 ⚠️ ✅ 🚇	_{Min-Seop Ahn} 💻 📖 👀 ⚠️ ✅ 🔬	_{Paul Ullrich} 🤔 🔬	_{Charles Doutriaux} 💻
_{Bo Dong} 💻	_{Shixuan Zhang} 💻	_{Alex Jonko} 💻	_{Karl Taylor} 🔬 🤔	_{Paul J. Durack} 💻	_{Mark Zelinka} 💻	_{Li-Wei Chao} 💻
_{Celine Bonfils} 🔬	_{Curtis C. Covey} 💻 🔬	_{Zeshawn Shaheen} 💻	_{Lina Muryanto} 🚇	_{Tom Vo} 🚇	_{Jason Boutte} 🚇	_{Jeffrey Painter} 🔣 🚇 💻
_{Stephen Po-Chedley} 🔣 🚇	_{Xylar Asay-Davis} 🚇	_{John Krasting} 💻 ⚠️	_{Angeline G Pendergrass} 💻 🔬 🤔	_{Michael Wehner} 💻 🔬	_{Daehyun Kim} 💻 🔬

This project follows the all-contributors specification.

Owner

Name: Program for Climate Model Diagnosis and Intercomparison
Login: PCMDI
Kind: organization
Email: pcmdi-cmip@llnl.gov
Location: Lawrence Livermore National Lab

Website: http://pcmdi.llnl.gov
Repositories: 45
Profile: https://github.com/PCMDI

Established in 1989, PCMDI aims to develop advanced methods and tools for diagnosis and intercomparison to improve Earth System and Climate Models

Citation (CITATION.cff)

# This CITATION.cff file was generated with cffinit.
# Visit https://bit.ly/cffinit to generate yours today!

cff-version: 1.2.0
title: PCMDI Metrics Package
message: >-
  If you use this software, please cite it using the
  metadata from this file.
type: software
authors:
  - given-names: Jiwoo
    family-names: Lee
    email: lee1043@llnl.gov
    affiliation: Lawrence Livermore National Laboratory
    orcid: 'https://orcid.org/0000-0002-0016-7199'
  - family-names: Gleckler
    given-names: Peter
    affiliation: Lawrence Livermore National Laboratory
    orcid: 'https://orcid.org/0000-0003-2816-6224'
  - family-names: Ordonez
    given-names: Ana
    affiliation: Lawrence Livermore National Laboratory
  - given-names: Bo
    family-names: Dong
    affiliation: Lawrence Livermore National Laboratory
  - affiliation: Lawrence Livermore National Laboratory
    given-names: Kristin
    family-names: Chang
  - given-names: Paul
    family-names: Ullrich
    affiliation: Lawrence Livermore National Laboratory and UC Davis
    orcid: 'https://orcid.org/0000-0003-4118-4590'
identifiers:
  - type: doi
    value: 10.5281/zenodo.592790
    description: Software
repository-code: 'https://github.com/PCMDI/pcmdi_metrics'
url: 'http://pcmdi.github.io/pcmdi_metrics/'
abstract: ' Open-source Python package for Systematic Evaluation of Climate and Earth System Models '
keywords:
  - climate
  - climate modeling
  - climate model evaluation
  - Earth System Model evaluation
  - climatology
  - climate variability
license: BSD-3-Clause
version: '3.9.1'
date-released: '2025-04-10'

GitHub Events

Total

Create event: 110
Commit comment event: 2
Release event: 9
Delete event: 104
Member event: 2
Pull request event: 206
Fork event: 8
Issues event: 83
Watch event: 15
Issue comment event: 232
Push event: 664
Gollum event: 7
Pull request review comment event: 5
Pull request review event: 44

Last Year

Create event: 110
Commit comment event: 2
Release event: 9
Delete event: 104
Member event: 2
Pull request event: 206
Fork event: 8
Issues event: 83
Watch event: 15
Issue comment event: 232
Push event: 664
Gollum event: 7
Pull request review comment event: 5
Pull request review event: 44

Committers

Last synced: 6 months ago

All Time

Total Commits: 4,050
Total Committers: 40
Avg Commits per committer: 101.25
Development Distribution Score (DDS): 0.526

Past Year

Commits: 626
Committers: 12
Avg Commits per committer: 52.167
Development Distribution Score (DDS): 0.251

Top Committers

Name	Email	Commits
Jiwoo Lee	l**3@l**v	1,920
Charles Doutriaux	d**1@l**v	576
Ana Ordonez	o**4@l**v	381
Ana Ordonez	a**z@c**t	368
Peter Gleckler	g**1@l**v	177
Paul J. Durack	d**1@l**v	172
zshaheen	z**3@g**m	112
Min-Seop Ahn	a**6@l**v	63
Kristin Chang	c**1@l**v	33
Peter Gleckler	g**1@g**v	31
muryanto1	m**1@l**v	25
Bo Dong postdoc	d**n@h**m	24
Tom Vo	t**r@g**m	22
Min-Seop Ahn	4****n	20
Jason Boutte	b**n@g**m	20
Muryanto	m**1@m**v	14
Muryanto	m**1@M**v	12
allcontributors[bot]	4****]	10
ShixuanZhang	s**g@p**v	10
Peter Gleckler	g**1@c**v	10
omahs	7****s	7
mzelinka	z**1@l**v	7
Jeff Painter	p**1@l**v	6
Peter Gleckler	g**1@o**v	5
Jeff Painter	p**1@o**v	4
Paul Durack	p**d@p**)	3
jeffrey Painter	p**1@p**v	3
mfwehner	m**r@l**v	2
Lina Muryanto	3****1	2
John Krasting	J**g@n**v	1
and 10 more...

Committer Domains (Top 20 + Academic)

llnl.gov: 10 oceanonly.llnl.gov: 4 comcast.net: 1 gates.llnl.gov: 1 md-9776485.llnl.gov: 1 ml-9516678.llnl.gov: 1 pnnl.gov: 1 crunchy.llnl.gov: 1 public1.(none): 1 pcmdi11.llnl.gov: 1 lbl.gov: 1 noaa.gov: 1 login20.chn.perlmutter.nersc.gov: 1 durack1m1.llnl.gov: 1 durack1ml.llnl.gov: 1 quartz380.llnl.gov: 1 ciclad-ng.private.ipsl.fr: 1 loftair-108.lan: 1 loftair-85.llnl.gov: 1 lanl.gov: 1

Issues and Pull Requests

Last synced: 6 months ago

All Time

Total issues: 148
Total pull requests: 532
Average time to close issues: 10 months
Average time to close pull requests: 20 days
Total issue authors: 17
Total pull request authors: 17
Average comments per issue: 1.5
Average comments per pull request: 1.61
Merged pull requests: 454
Bot issues: 0
Bot pull requests: 8

Past Year

Issues: 59
Pull requests: 267
Average time to close issues: 9 days
Average time to close pull requests: 4 days
Issue authors: 12
Pull request authors: 10
Average comments per issue: 0.69
Average comments per pull request: 1.03
Merged pull requests: 222
Bot issues: 0
Bot pull requests: 6

View more stats

Top Authors

Issue Authors

lee1043 (87)
gleckler1 (14)
acordonez (11)
ShihengDuan (6)
msahn (6)
durack1 (5)
zhangshixuan1987 (4)
kristinchang3 (4)
doutriaux1 (2)
bosup (2)
xylar (1)
rbeucher (1)
tomvothecoder (1)
dnadeau4 (1)
ajonko (1)

Pull Request Authors

lee1043 (377)
acordonez (88)
durack1 (10)
zhangshixuan1987 (10)
kristinchang3 (9)
allcontributors[bot] (8)
msahn (7)
gleckler1 (6)
jasonb5 (4)
bosup (4)
mzelinka (2)
ajonko (2)
omahs (1)
tomvothecoder (1)
lewisjared (1)

Top Labels

Issue Labels

enhancement (24) New feature (15) bug (14) doc (10) graphic (5) feature request (3) demo (2) build (1) duplicate (1) QC (1) clean up (1)

Pull Request Labels

doc (96) enhancement (78) bug (42) demo (39) minor (34) New feature (27) graphic (23) clean up (15) build (13) feature request (10) workflow (6) CRITICAL (5) QC (3) TOPPRIORITY (2) DecisionNeededNow (2)

Packages

Total packages: 3
Total downloads: unknown

Total dependent packages: 0
(may contain duplicates)
Total dependent repositories: 0
(may contain duplicates)
Total versions: 71

proxy.golang.org: github.com/pcmdi/pcmdi_metrics

Documentation: https://pkg.go.dev/github.com/pcmdi/pcmdi_metrics#section-documentation
License: bsd-3-clause
Latest release: v3.9.1+incompatible
published 11 months ago

Versions: 32
Dependent Packages: 0
Dependent Repositories: 0

Rankings

Dependent packages count: 5.4%

Average: 5.6%

Dependent repos count: 5.8%

Last synced: 6 months ago

proxy.golang.org: github.com/PCMDI/pcmdi_metrics

Documentation: https://pkg.go.dev/github.com/PCMDI/pcmdi_metrics#section-documentation
License: bsd-3-clause
Latest release: v3.9.1+incompatible
published 11 months ago

Versions: 32
Dependent Packages: 0
Dependent Repositories: 0

Rankings

Dependent packages count: 5.4%

Average: 5.6%

Dependent repos count: 5.8%

Last synced: 6 months ago

conda-forge.org: pcmdi_metrics

model metrics tools

Homepage: http://github.com/PCMDI/pcmdi_metrics
License: BSD-3-Clause
Latest release: 2.5.1
published over 3 years ago

Versions: 7
Dependent Packages: 0
Dependent Repositories: 0

Rankings

Forks count: 28.5%

Dependent repos count: 34.0%

Stargazers count: 35.6%

Average: 37.3%

Dependent packages count: 51.2%

Last synced: 6 months ago

Dependencies

.github/workflows/build_workflow.yml actions

actions/cache v2 composite
actions/checkout v2 composite
actions/setup-python v2 composite
conda-incubator/setup-miniconda v2 composite
fkirc/skip-duplicate-actions master composite
pre-commit/action v2.0.3 composite

pcmdi_metrics

Science Score: 85.0%

Keywords

Keywords from Contributors

Repository

Basic Info

Statistics

Topics

Metadata Files

README.md

PCMDI Metrics Package (PMP)

Documentation

Getting Started

References

Contact

Acknowledgement

License

Release Notes and History

Current Core Development Team

All Contributors

Owner

Citation (CITATION.cff)

GitHub Events

Total

Last Year

Committers

All Time

Past Year

Top Committers

Committer Domains (Top 20 + Academic)

Issues and Pull Requests

All Time

Past Year

Top Authors

Issue Authors

Pull Request Authors

Top Labels

Issue Labels

Pull Request Labels

Packages

proxy.golang.org: github.com/pcmdi/pcmdi_metrics

Rankings

proxy.golang.org: github.com/PCMDI/pcmdi_metrics

Rankings

conda-forge.org: pcmdi_metrics

Rankings

Dependencies