<!-- These are examples of badges you might also want to add to your README. Update the URLs accordingly. -->

⚠️mp-time-split has been moved to matbench-genmetrics.mp_time_split as a namespace package⚠️

mp-time-split

Use Materials Project time-splits for generative modeling benchmarking.

While methods for cross-validating accuracy of materials informatics models is well estabilished (see for example Matbench), evaluating the performance of generative models such as FTCP or imatgen, and many others is less straightforward. Recently, Xie et al. introduced new benchmark datasets and metrics in CDVAE for several state-of-the-art algorithms. This repository acts as a supplement to CDVAE benchmarks, delivering a new benchmark dataset (Materials_Project_Time_Split_52 or MPTS-52) with time-based (5 $\times$ train/val) +train/test splits suitable for cross-validated hyperparameter optimization and subsequent benchmarking via the test split.

MPTS-52 is most comparable to MP-20 from Xie et al., with the difference that up to 52 atoms are allowed and possibly a difference in the unique elements, as no elemental filtering was applied (e.g. removal of radioactive elements).

Quick Start

Installation

bash conda env create -n mp-time-split -c conda-forge mp-time-split conda activate mp-time-split

Example

```python from mptimesplit.core import MPTimeSplit

mpt = MPTimeSplit(target="energyabovehull") mpt.load(dummy=False)

for fold in mpt.folds: traininputs, valinputs, trainoutputs, valoutputs = mpt.gettrainandvaldata( fold )

finaltraininputs, testinputs, finaltrainoutputs, testoutputs = mpt.gettestdata() ```

Output

python print(train_inputs.iloc[0], train_outputs)

{note} Data (e.g. `train_inputs`, `train_outputs`) is sorted by earliest publication year. The Materials Project ID number is used as the pandas `Series` index. The "mp-" and "mvc" suffixes are dropped, except that "-" is retained in case of "mvc-" to make it negative in the index and distinguish between "mp-" and "mvc-" type entries. This is to make it easy to sort the entries based on the index. Ensure that you apply `abs` to the index prior to sorting, otherwise the `mvc-` entries will be in the wrong order. See [how-do-i-do-a-time-split-of-materials-project-entries-e-g-pre-2018-vs-post-2018](https://matsci.org/t/how-do-i-do-a-time-split-of-materials-project-entries-e-g-pre-2018-vs-post-2018/42584/2?u=sgbaird) for more information. More detailed information (e.g. BibTeX references) can be accessed via `mpt.data`.

For additional examples, see the notebooks directory.

Installation

Anaconda

Create an environment named (-n) mp-time-split with the Anaconda package mp-time-split intalled from the conda-forge channel (-c). bash conda env create -n mp-time-split -c conda-forge mp-time-split

PyPI

Optionally create and activate a conda environment (recommended to use an isolated environment of some kind): bash conda env create -n mp-time-split python==3.9.* # 3.7.* or 3.8.* also OK conda activate mp-time-split Install the mp-time-split package from PyPI. bash pip install mp-time-split

environment.yml

In order to set up the necessary environment:

1. review and uncomment what you need in environment.yml and create an environment mp-time-split with the help of conda: conda env create -f environment.yml
2. activate the new environment with: conda activate mp-time-split

Local Installation

NOTE: The conda environment will have mp-time-split installed in editable mode. Some changes, e.g. in setup.cfg, might require you to run pip install -e . again.

Optional and needed only once after git clone:

1. install several pre-commit git hooks with: ```bash pre-commit install

   You might also want to run pre-commit autoupdate

   `` and checkout the configuration under.pre-commit-config.yaml. The-n, --no-verifyflag ofgit commit` can be used to deactivate pre-commit hooks temporarily.

2. install nbstripout git hooks to remove the output cells of committed notebooks with: bash nbstripout --install --attributes notebooks/.gitattributes This is useful to avoid large diffs due to plots in your notebooks. A simple nbstripout --uninstall will revert these changes.

Then take a look into the scripts and notebooks folders.

Project Organization

├── AUTHORS.md <- List of developers and maintainers. ├── CHANGELOG.md <- Changelog to keep track of new features and fixes. ├── CONTRIBUTING.md <- Guidelines for contributing to this project. ├── Dockerfile <- Build a docker container with `docker build .`. ├── LICENSE.txt <- License as chosen on the command-line. ├── README.md <- The top-level README for developers. ├── configs <- Directory for configurations of model & application. ├── data │ ├── external <- Data from third party sources. │ ├── interim <- Intermediate data that has been transformed. │ ├── processed <- The final, canonical data sets for modeling. │ └── raw <- The original, immutable data dump. ├── docs <- Directory for Sphinx documentation in rst or md. ├── environment.yml <- The conda environment file for reproducibility. ├── models <- Trained and serialized models, model predictions, │ or model summaries. ├── notebooks <- Jupyter notebooks. Naming convention is a number (for │ ordering), the creator's initials and a description, │ e.g. `1.0-fw-initial-data-exploration`. ├── pyproject.toml <- Build configuration. Don't change! Use `pip install -e .` │ to install for development or to build `tox -e build`. ├── references <- Data dictionaries, manuals, and all other materials. ├── reports <- Generated analysis as HTML, PDF, LaTeX, etc. │ └── figures <- Generated plots and figures for reports. ├── scripts <- Analysis and production scripts which import the │ actual PYTHON_PKG, e.g. train_model. ├── setup.cfg <- Declarative configuration of your project. ├── setup.py <- [DEPRECATED] Use `python setup.py develop` to install for │ development or `python setup.py bdist_wheel` to build. ├── src │ └── mp_time_split <- Actual Python package where the main functionality goes. ├── tests <- Unit tests which can be run with `pytest`. ├── .coveragerc <- Configuration for coverage reports of unit tests. ├── .isort.cfg <- Configuration for git hook that sorts imports. └── .pre-commit-config.yaml <- Configuration of pre-commit git hooks.

Note

This project has been set up using PyScaffold 4.2.2 and the dsproject extension 0.7.post1.dev8+g43a905e.

To create the same starting point for this repository, as of 2022-06-01 on Windows you will need the development versions of PyScaffold and extensions, however this will not be necessary once certain bugfixes have been introduced in the next stable releases: bash pip install git+https://github.com/pyscaffold/pyscaffold.git git+https://github.com/pyscaffold/pyscaffoldext-dsproject.git git+https://github.com/pyscaffold/pyscaffoldext-markdown.git

The following pyscaffold command creates a starting point for this repository: bash putup xtal2png --github-actions --markdown --dsproj Alternatively, you can edit a file interactively and update and uncomment relevant lines, which saves some of the additional setup: bash putup --interactive xtal2png