Data

Tools

Indexes

Applications

Experiments

Open Source Science

matbench-discovery

An evaluation framework for machine learning models simulating high-throughput materials discovery.

https://github.com/janosh/matbench-discovery

Science Score: 49.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ✓
    DOI references
    Found 2 DOI reference(s) in README
  • ✓
    Academic publication links
    Links to: arxiv.org, nature.com
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (14.6%) to scientific vocabulary

Keywords

bayesian-optimization convex-hull high-throughput-search interatomic-potential machine-learning materials-discovery
Last synced: 6 months ago · JSON representation

Repository

An evaluation framework for machine learning models simulating high-throughput materials discovery.

Basic Info
Statistics
  • Stars: 178
  • Watchers: 12
  • Forks: 43
  • Open Issues: 6
  • Releases: 8
Topics
bayesian-optimization convex-hull high-throughput-search interatomic-potential machine-learning materials-discovery
Created over 3 years ago · Last pushed 6 months ago
Metadata Files
Readme Changelog Contributing License Citation

readme.md

Logo
Matbench Discovery

Matbench Discovery is an interactive leaderboard which ranks ML models on multiple tasks designed to simulate high-throughput discovery of new stable inorganic crystals, finding their ground state atomic positions and predicting their thermal conductivity.

We rank 20+ models covering multiple methodologies including graph neural network (GNN) interatomic potentials, GNN one-shot predictors, iterative Bayesian optimizers and random forests with shallow-learning structure fingerprints.

Our results show that ML models have become robust enough to deploy them as triaging steps to more effectively allocate compute in high-throughput DFT relaxations. This work provides valuable insights for anyone looking to build large-scale materials databases.

To cite Matbench Discovery, use:

Janosh Riebesell, Rhys E. A. Goodall, Philipp Benner, Yuan Chiang, Bowen Deng, Mark Asta, Gerbrand Ceder, Alpha A. Lee, Anubhav Jain, and Kristin A. Persson. "Matbench Discovery -- A Framework to Evaluate Machine Learning Crystal Stability Predictions." arXiv, August 28, 2023. https://doi.org/10.1038/s42256-025-01055-1.

We welcome new models additions to the leaderboard through GitHub PRs. See the contributing guide for details and ask support questions via GitHub discussion.

For detailed results and analysis, check out https://nature.com/articles/s42256-025-01055-1.

Disclaimer: We evaluate how accurately ML models predict several material properties like thermodynamic stability, thermal conductivity, and atomic positions, in all cases using PBE DFT as reference data. Although these properties are important for high-throughput materials discovery, the ranking cannot give a complete picture of a model's overall ability to drive materials research. A high ranking does not constitute endorsement by the Materials Project.

Owner

  • Name: Janosh Riebesell
  • Login: janosh
  • Kind: user
  • Location: GitHub

Working on computational chemistry with pre-trained ML force fields

GitHub Events

Total
  • Issues event: 44
  • Watch event: 71
  • Delete event: 73
  • Issue comment event: 163
  • Push event: 468
  • Pull request review comment event: 158
  • Pull request event: 171
  • Pull request review event: 104
  • Fork event: 34
  • Create event: 70
Last Year
  • Issues event: 44
  • Watch event: 71
  • Delete event: 73
  • Issue comment event: 163
  • Push event: 468
  • Pull request review comment event: 158
  • Pull request event: 171
  • Pull request review event: 104
  • Fork event: 34
  • Create event: 70

Issues and Pull Requests

Last synced: 6 months ago

All Time
  • Total issues: 63
  • Total pull requests: 245
  • Average time to close issues: 27 days
  • Average time to close pull requests: 4 days
  • Total issue authors: 29
  • Total pull request authors: 26
  • Average comments per issue: 2.13
  • Average comments per pull request: 0.8
  • Merged pull requests: 213
  • Bot issues: 0
  • Bot pull requests: 15
Past Year
  • Issues: 27
  • Pull requests: 160
  • Average time to close issues: 16 days
  • Average time to close pull requests: 4 days
  • Issue authors: 19
  • Pull request authors: 24
  • Average comments per issue: 1.81
  • Average comments per pull request: 0.84
  • Merged pull requests: 136
  • Bot issues: 0
  • Bot pull requests: 6
View more stats
Top Authors
Issue Authors
  • pbenner (22)
  • lan496 (5)
  • janosh (4)
  • DeNeutoy (4)
  • TheCutestCat (2)
  • hongshuh (2)
  • PrincessHakuryu (1)
  • yzchen08 (1)
  • jackwebersdgr (1)
  • zzz-sl (1)
  • Andrew-S-Rosen (1)
  • bkmi (1)
  • ltalirz (1)
  • rydeveraumn (1)
  • YutackPark (1)
Pull Request Authors
  • janosh (183)
  • CompRhys (38)
  • pre-commit-ci[bot] (19)
  • yury-lysogorskiy (8)
  • YutackPark (8)
  • DeNeutoy (7)
  • pbenner (6)
  • zmyybc (4)
  • WangYuHang-WYH (4)
  • chiang-yuan (4)
  • yanghan-microsoft (2)
  • EricZQu (2)
  • yanghan234 (2)
  • Asecretboy (2)
  • anyangml (2)
Top Labels
Issue Labels
bug (23) data (9) discussion (6) pkg (5) help (4) site (4) question (3) reproducibility (3) enhancement (2) preprint (1) documentation (1) help wanted (1) new model (1) analysis (1) geo opt (1) slurm (1)
Pull Request Labels
site (73) analysis (49) fix (45) new model (41) enhancement (30) data (28) docs (21) housekeeping (17) ux (14) qa (13) refactor (12) mlff (12) reproducibility (11) MLIP (compliant) (9) geo opt (8) types (7) MLIP (non-compliant) (7) pkg (7) preprint (7) documentation (6) viz (5) breaking (5) model update (4) ci (3) discussion (2) bug (2) symmetry (2) phonons (2) duplicate (1) slurm (1)

Packages

  • Total packages: 2
  • Total downloads:
    • pypi 736 last-month
  • Total dependent packages: 0
    (may contain duplicates)
  • Total dependent repositories: 0
    (may contain duplicates)
  • Total versions: 19
  • Total maintainers: 1
proxy.golang.org: github.com/janosh/matbench-discovery
  • Versions: 9
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent packages count: 5.4%
Average: 5.6%
Dependent repos count: 5.8%
Last synced: 6 months ago
pypi.org: matbench-discovery

A benchmark for machine learning energy models on inorganic crystal stability prediction from unrelaxed structures

  • Homepage: https://janosh.github.io/matbench-discovery
  • Documentation: https://matbench-discovery.readthedocs.io/
  • License: MIT License Copyright (c) 2022 Janosh Riebesell Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. The software is provided "as is", without warranty of any kind, express or implied, including but not limited to the warranties of merchantability, fitness for a particular purpose and noninfringement. In no event shall the authors or copyright holders be liable for any claim, damages or other liability, whether in an action of contract, tort or otherwise, arising from, out of or in connection with the software or the use or other dealings in the software.
  • Latest release: 1.3.1
    published over 1 year ago
  • Versions: 10
  • Dependent Packages: 0
  • Dependent Repositories: 0
  • Downloads: 736 Last month
Rankings
Dependent packages count: 6.6%
Average: 18.9%
Downloads: 19.5%
Dependent repos count: 30.6%
Maintainers (1)
janosh
Last synced: 6 months ago

Dependencies

.github/workflows/gh-pages.yml actions
.github/workflows/link-check.yml actions
.github/workflows/test.yml actions
site/package.json npm
  • @iconify/svelte ^3.1.4 development
  • @rollup/plugin-yaml ^4.1.1 development
  • @sveltejs/adapter-static ^2.0.3 development
  • @sveltejs/kit ^1.25.0 development
  • @sveltejs/vite-plugin-svelte ^2.4.5 development
  • @typescript-eslint/eslint-plugin ^6.7.0 development
  • @typescript-eslint/parser ^6.7.0 development
  • d3-scale-chromatic ^3.0.0 development
  • elementari ^0.2.2 development
  • eslint ^8.49.0 development
  • eslint-plugin-svelte ^2.33.1 development
  • hastscript ^8.0.0 development
  • highlight.js ^11.8.0 development
  • js-yaml ^4.1.0 development
  • katex ^0.16.8 development
  • mdsvex ^0.11.0 development
  • prettier ^3.0.3 development
  • prettier-plugin-svelte ^3.0.3 development
  • rehype-autolink-headings ^7.0.0 development
  • rehype-katex-svelte ^1.2.0 development
  • rehype-slug ^6.0.0 development
  • remark-math 3.0.0 development
  • svelte ^4.2.0 development
  • svelte-check ^3.5.1 development
  • svelte-multiselect ^10.1.0 development
  • svelte-preprocess ^5.0.4 development
  • svelte-toc ^0.5.6 development
  • svelte-zoo ^0.4.9 development
  • svelte2tsx ^0.6.21 development
  • tslib ^2.6.2 development
  • typescript 5.2.2 development
  • vite ^4.4.9 development
pyproject.toml pypi
  • jinja2 *
  • matplotlib *
  • numpy *
  • pandas [output_formatting]>=2.0.0
  • plotly *
  • pymatgen *
  • pymatviz [export-figs]
  • scikit-learn *
  • scipy *
  • tqdm *
  • wandb *
.github/workflows/release.yml actions
  • actions/checkout v4 composite
  • actions/setup-python v5 composite
  • pypa/gh-action-pypi-publish release/v1 composite