Releases | Open Source Science

openpmd-api - New Backends, Extension, Perf. & Memory

This release fixes backend and build bugs of the 0.16.0 release.

Thank you to Franz Poeschel @franzpoeschel and Axel Huebl @ax3l for contributing to this release!

- C++
Published by ax3l 12 months ago

openpmd-api - New Backends, Extension, Perf. & Memory

ADIOS2 Joined arrays, API simplification, HDF5 subfiling, TOML backend, wildcard file extensions, Performance & Memory: This release adds support for additional I/O features in all backends, and additionally a completely new TOML backend, useful for openPMD-formatted configuration files in scientific workflows. The ADIOS1 backend, previously deprecated, has been removed in favor of ADIOS2.

The openPMD-api has been simplified by no longer requiring explicit specification for scalar components via RecordComponent::SCALAR in its object model. The addition of wildcard filename extensions (e.g. simData.%E or simData_%T.%E) simplifies the generic implementation of backend-independent logic, such as post-processing routines.

Thanks to Franz Pöschel @franzpoeschel, Axel Huebl @ax3l, David Grote @dpgrote, Davide Terzani @titoiride, Ilian Kara-Mostefa @IlianCS, Junmin Gu @guj, Luca Fedeli @lucafedeli88, Nils Schild @DerNils-git, Paweł Ordyna @pordyna and Ulrik Günther @skalarproduktraum for contributing to this release!

New Contributors

@dpgrote made their first contribution in https://github.com/openPMD/openPMD-api/pull/1442
@skalarproduktraum made their first contribution in https://github.com/openPMD/openPMD-api/pull/1364
@pordyna made their first contribution in https://github.com/openPMD/openPMD-api/pull/1477
@IlianCS made their first contribution in https://github.com/openPMD/openPMD-api/pull/1510
@lucafedeli88 made their first contribution in https://github.com/openPMD/openPMD-api/pull/1532
@titoiride made their first contribution in https://github.com/openPMD/openPMD-api/pull/1558

- C++
Published by ax3l over 1 year ago

openpmd-api - Python, ADIOS2 and HDF5 Fixes

This release fixed regressions in the Python frontend as well as the ADIOS2 and HDF5 backends. Supported macOS versions are now 11.0+ and Python versions are 3.8+.

Thanks to Franz Poeschel, Axel Huebl, David P. Grote, Ulrik Günther and Ilian Kara-Mostefa for contributing to this release!

- C++
Published by ax3l over 2 years ago

openpmd-api - Build Regressions

This release fixes build regressions and minor documentation updates for the 0.15.0 release.

Thanks to Axel Huebl and Franz Poeschel for contributing to this release!

- C++
Published by ax3l almost 3 years ago

openpmd-api - C++17, Error Recovery, ADIOS2 BP5, Append & Read-Linear Modes, Performance & Memory

This release adds error recovery mechanisms, in order to access erroneous datasets, created e.g. by crashing simulations. The BP5 engine of ADIOS2 v2.9 is fully supported by this release, including access to its various features for more fine-grained control of memory usage. Various I/O performance improvements for HDF5 are activated by default. Runtime configuration of openPMD and its backends, e.g. selection of backends and compression, is now consistently done via JSON, and alternatively via TOML for better readability. The data storage/retrieval API now consistently supports all common C++ pointer types (raw and smart pointers), implementing automatic memory optimizations for ADIOS2 BP5 if using unique pointers.

The miminum required C++ version is now C++17. Supported Python versions add Python 3.10 and 3.11.

Thanks to Axel Huebl, Franz Poeschel, Jean Luc Bez, Junmin Gu, Bernhard Manfred Gruber, and Nils Schild for contributing to this release!

- C++
Published by ax3l almost 3 years ago

openpmd-api - Improve Series Parsing, Python & Fix Backend Bugs

This release improves reading back iterations that overflow the specified zero-pattern. ADIOS1, ADIOS2 and HDF5 backend stability and performance were improved. Python bindings got additional wheel platform support and various smaller issues were fixed.

Thanks to Axel Huebl, Franz Poeschel, and Jean Luca Bez for contributing to this release!

- C++
Published by ax3l over 3 years ago

openpmd-api - Increased Compatibility & Python Install Bug

This release fixes various read/parsing bugs and increases compatibility with upcoming versions of ADIOS and old releases of Intel icpc. An installation issue for pip-based installs from source in the last release was fixed and Python 3.10 support added. Various documentation and installation warnings have been fixed.

Thanks to Axel Huebl and Franz Poeschel for contributing to this release!

- C++
Published by ax3l almost 4 years ago

openpmd-api - Read Bugs, C++17 Mixing and HDF5 Performance

This release makes reads more robust by fixing small API, file-based parsing and test bugs. Building the library in C++14 and using it in C++17 will not result in incompatible ABIs anymore. HDF5 1.10.1+ performance was improved significantly.

Thanks to Axel Huebl, Franz Poeschel and Jean Luca Bez for contributing to this release!

- C++
Published by ax3l about 4 years ago

openpmd-api - Various Reader Fixes

This releases fixes regressions in reads, closing files properly, avoiding inefficient parsing and allowing more permissive casts in attribute reads. (Inofficial) support for HDF5 vlen string reads has been fixed.

Thanks to Franz Poeschel and Axel Huebl for contributing to this release!

- C++
Published by ax3l over 4 years ago

openpmd-api - ADIOS2 Close Regressions & ADIOS1 Build

Fix a regression with file handling for ADIOS2 when using explicit close logic, especially with interleaved writes to multiple iterations. Also fix an issue with ADIOS1 builds that potentially picked up headers from older, installed openPMD-api versions.

Thanks to Franz Poeschel and Axel Huebl for contributing to this release!

- C++
Published by ax3l over 4 years ago

openpmd-api - Resize, Dask, openpmd-pipe and new ADIOS2 Iteration Encoding

This release adds support for resizable data sets. For data-processing, support for Dask (parallel) and Pandas (serial) are added and lazy reader parsing of iterations is now supported. ADIOS2 adds an experimental variable-based iteration encoding. An openPMD Series can now be flushed from non-Series objects and write buffers can be requested upfront to avoid unnecessary data copies in some situations.

Thanks to Axel Huebl, Franz Poeschel, Erik Schnetter, Junmin Gu, John Kirkham, and Dmitry Ganyushin for contributing to this release!

- C++
Published by ax3l over 4 years ago

openpmd-api - Fix AppleClang & DPC++ Build

Fix a missing include that fails builds with Apple's clang and Intel's dpcpp compilers.

Thanks to Axel Huebl for contributing to this release!

- C++
Published by ax3l over 4 years ago

openpmd-api - Fix Various Read Issues

This release fixes various bugs related to reading: a chunk fallback for constant components, skip missing patch records, a backend bug in each ADIOS2 & HDF5, and we made the Python load_chunk method more robust.

Thanks to Axel Huebl and Franz Poeschel for contributing to this release!

- C++
Published by ax3l over 4 years ago

openpmd-api - Fix Patch Read & Python store_chunk

This release fixes a regression with particle patches, related to Iteration::open() and ::close() functionality. Also, issues with the Python store_chunk method are addressed.

Thanks to Axel Huebl and Franz Poeschel for contributing to this release!

- C++
Published by ax3l almost 5 years ago

openpmd-api - Fix openPMD-ls & Iteration open/close

This release fixes regressions in the series "ls" functionality and tools, related to Iteration::open() and ::close() functionality. We also add support to read back complex numbers with JSON.

Thanks to Franz Poeschel and Axel Huebl for contributing to this release!

- C++
Published by ax3l almost 5 years ago

openpmd-api - Streaming Support, Python, Benchmarks

This release adds first support for streaming I/O via ADIOS2's SST engine. More I/O benchmarks have been added with realistic application load patterns. Many Python properties for openPMD attributes have been modernized, with slight breaking changes in Iteration and Mesh data order. This release requires C++14 and adds support for Python 3.9. With this release, we leave the "alpha" phase of the software and declare "beta" status.

Thanks to Axel Huebl, Franz Poeschel, Junmin Gu, Richard Pausch, and Sergei Bastrakov for contributing to this release!

- C++
Published by ax3l about 5 years ago

openpmd-api - Complex Numbers, Close & Backend Options

This release adds data type support for complex numbers, allows to close iterations and adds first support for backend configuration options (via JSON), which are currently implemented for ADIOS2. Further installation options have been added (homebrew and CLI tool support with pip). New free standing functions and macro defines are provided for version checks.

Thanks to Axel Huebl, Franz Poeschel, and Lígia Diana Amorim for contributing to this release!

- C++
Published by ax3l over 5 years ago

openpmd-api - HDF5-1.12, Azimuthal Examples & Tagfile

This release adds support for the latest HDF5 release. Also, we add versioned Doxygen and a tagfile for external docs to our online manual.

Thanks to Axel Huebl for contributing to this release!

- C++
Published by ax3l almost 6 years ago

openpmd-api - Robust Independent I/O

This release improves MPI-parallel I/O with HDF5 and ADIOS. ADIOS2 is now the default backend for handing .bp files.

Thanks to Axel Huebl, Junmin Gu and Ligia Diana Amorim for contributing to this release!

- C++
Published by ax3l almost 6 years ago

openpmd-api - Improved HDF5 Handling

More robust HDF5 file handling and fixes of local includes for more isolated builds.

Thanks to Axel Huebl, Junmin Gu, Ligia Diana Amorim, Richard Pausch, and Igor Andriyash for contributing to this release!

- C++
Published by ax3l about 6 years ago

openpmd-api - Improved Error Messages

Thrown errors are now prefixed by the backend in use and ADIOS1 series reads are more robust.

Thanks to Axel Huebl, Ligia Diana Amorim, Richard Pausch, and Ksenia Bastrakova for contributing to this release!

- C++
Published by ax3l about 6 years ago

openpmd-api - ADIOS2 Open Speed and NVCC Fixes

This releases improves the initial time spend when parsing data series with the ADIOS2 backend. Compile problems when using the CUDA NVCC compiler in downstream projects have been fixed. We adopted a Code of Conduct in openPMD.

Thanks to Axel Huebl, Franz Poeschel, William F. Godoy, Ligia Diana Amorim, and Remi Lehe for contributing to this release!

- C++
Published by ax3l about 6 years ago

openpmd-api - ADIOS2 Preview, Python & MPI Improved

This release adds a first (preview) implementation of ADIOS2 (BP4). Python 3.8 support as well as improved pip builds on macOS and Windows have been added. ADIOS1 and HDF5 now support non-collective (independent) store and load operations with MPI. More HPC compilers, such as IBM XL, ICC and PGI have been tested. The manual has been improved with more details on APIs, examples, installation, and backends.

Thanks to Axel Huebl, Franz Poeschel, Ligia Diana Amorim, and Glenn Richardson for contributing to this release!

- C++
Published by ax3l about 6 years ago

openpmd-api - Improved Builds and Packages

This release improves PyPI releases with proper declaration of build dependencies (use pip 19.0+). For Makefile-based projects, an openPMD.pc file to be used with pkg-config is added on install. RecordComponent now supports a makeEmpty method to write a zero-extent, yet multi-dimensional record component. We are now building as shared library by default.

Thanks to Axel Huebl, and Franz Poeschel for contributing to this release!

- C++
Published by ax3l over 6 years ago

openpmd-api - Python mpi4py and Slice Support

We implemented MPI support for the Python frontend via mpi4py and added []-slice access to Record_Component loads and stores. A bug requiring write permissions for read-only series was fixed and memory provided by users is now properly checked for being contiguous. Introductory chapters in the manual have been greatly extended.

Thanks to Axel Huebl, and Fabian Koller for contributing to this release!

- C++
Published by ax3l almost 7 years ago

openpmd-api - Bug Fixes in Multi-Platform Builds

This release fixes several issues with libstdc++ (e.g. on OSX), cross-compiles and modern compilers.

Thanks to Axel Huebl, and Ray Donnelly for contributing to this release!

- C++
Published by ax3l almost 7 years ago

openpmd-api - JSON Support, Interface Simplification and Stability

This release introduces serial JSON (.json) support. Our API has been unified with slight breaking changes such as a new Python module name (import openpmd_api from now on) as well as re-ordered store/loadChunk argument orders.

Please see our new "upgrade guide" section in the manual on how to update existing scripts. Additionally, many little bugs have been fixed. Official Python 3.7 support and a parallel benchmark example have been added.

Thanks to Axel Huebl, Franz Poeschel, Sergei Bastrakov, and Ray Donnelly for contributing to this release!

- C++
Published by ax3l almost 7 years ago

openpmd-api - Reading Varying Iteration Padding

Support reading series with varying iteration padding (or no padding at all) as currently used in PIConGPU.

Thanks to Axel Huebl for contributing to this release!

- C++
Published by ax3l about 7 years ago