adios2

https://github.com/ornladios/adios2 - v2.10.2

What's Changed

Bump version to v2.10.2 by @vicentebolea in #4383
Backport #4379 by @vicentebolea in #4381
fix(cmake): increase min required cmake when using bloscs2 by @vicentebolea in #4379
Fix missing int64_t definition by @eisenhauer in #4365
Update the memory selection layout when there is a layout mismatch by @anagainaru in #4346
Setting the memory space for Spans to Host by @anagainaru in #4343
Compress the lossless part of MGARD data with Zstd. by @pnorbert in #4339
Fix an issue when reading blocks in streaming mode by @pnorbert in #4332
Fix in python API: by @pnorbert in #4331
Update the script to build adios with Kokkos for Frontier by @anagainaru in #4303
Fixing parser/lexer related errors by @anagainaru in #4293
[Bug fix] Read side computation of derived variables should not be triggered when data is stored by @anagainaru in #4247
Update ReadMe.md by @vicentebolea in #4243
[Bug fix] Switch from ROCM_VERSION_MAJOR to HIP_VERSION_MAJOR by @anagainaru in #4222
Limit the exposed symbols related to derived variables by @anagainaru in #4207
Patch for xrootd with derived variables by @anagainaru in #4200
Add macro defines for BP3, BP4 and BP5 by @franzpoeschel in #4191
Overload NdCopy instead of adding a new argument for the 2.10 release by @anagainaru in #4377
Prepare release branch by @vicentebolea in #4378
bp5dbg parse records and check sizes during it for mmd.0 and md.0. No… by @pnorbert in #3759
Backport #3759 to Release 2.10 branch. by @pnorbert in #4341
ci: backport ci changes from master by @vicentebolea in #4356
Backport 4305 by @vicentebolea in #4349
bp5dbg parse records and check sizes during it for mmd.0 and md.0. No… by @pnorbert in #3759
Backport #3759 to Release 2.10 branch. by @pnorbert in #4341
kokkos: support ROCM >=6 by @vicentebolea in #4214
Dill Upstream in release_210 by @eisenhauer in #4198

Full Changelog: https://github.com/ornladios/ADIOS2/compare/v2.10.1...v2.10.2

- C++
Published by vicentebolea over 1 year ago

https://github.com/ornladios/adios2 - v2.10.1

What's Changed

pugixml: correct target name by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/4135
Dill Upstream by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/4136
Fix python char by @pnorbert in https://github.com/ornladios/ADIOS2/pull/4144
Test hpdf5 fix on 2.10 by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/4177
Deactivate campaign recording by default. Use .adios-campaign/ as dir… by @pnorbert in https://github.com/ornladios/ADIOS2/pull/4174
EVPath Upstream #4181
Add API call to reset memory selection #4169
Typo in documentation #4166
Adding hooks to gather performance #4161
Minor docs tweaks #4153
ReadModeFlatten #4147
pugixml,cmake: fix linkage when pugixml #4142
Fixes for FreeBSD, including upstream #4138
Kill outdated reference to bindings.Mode.ReadFlattenSteps #4185
Bump version to v2.10.1 by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/4187

Full Changelog: https://github.com/ornladios/ADIOS2/compare/v2.10.0...v2.10.1

- C++
Published by vicentebolea about 2 years ago

https://github.com/ornladios/adios2 - v2.10.0

What's new in 2.10?

This is a major release with new features and lots of bug fixes. The main new feature is the new Python API.

Python

Before, ADIOS had two separate APIs for Python. The low-level ("Full") API was written with Pybind11 and directly mimicked the C++ API. The high-level API was another, smaller, and more pythonesque API that allowed for easier scripting with Python. The main problems with these two were that they were independent, and that the high-level API was not complete. Once a developer needed a feature only available in the full API, they had to start from scratch writing a script with the full API.

In 2.10, there is officially one Python API, written in Python, which in turn uses the old Pybind11 classes. The new API combines the high-level features of the old high-level API and the full feature set of the low-level bindings.

Note:

Old scripts that used the full API can still run without almost any modification, just change the import line from import adios2 to import adios2.bindings as adios2

Old scripts that used the high-level API must be modified to make them work with the new API, see Transition from old API to new API: Transition from old API to new API

See Python API

New/updated notable features

BP5 is supported on Windows now
- SST and DataMan staging engines are GPU-Aware now
- SYCL support added for Intel GPUs (besides CUDA and HIP for NVidia and AMD GPUs)
- the SST/libfabric data transport now works on Frontier (besides the MPI data transport)

New experimental features

Campaign management: https://adios2.readthedocs.io/en/latest/advanced/campaign_management.html

What's Changed

ci: migrate to almalinux, cuda base image by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3484
ci: update kokkos to 3.7 by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3490
codeowners: added code owners file by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3496
Hdf5 vol support will require latest hdf5 1.14 by @guj in https://github.com/ornladios/ADIOS2/pull/3516
Bump numpy from 1.21.6 to 1.22.0 in /docs by @dependabot in https://github.com/ornladios/ADIOS2/pull/3535
Bump cryptography from 38.0.2 to 39.0.1 in /docs by @dependabot in https://github.com/ornladios/ADIOS2/pull/3534
Release 29 by @dmitry-ganyushin in https://github.com/ornladios/ADIOS2/pull/3545
Enable Linux Emulation build (Power8, IBM XL) by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3537
Setting code owners for the GPU files by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3561
Release 29 by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3573
Allowing Put/Get to receive Kokkos Views with HIP backend by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3571
Prototype DAOS engine by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3576
CODEOWNERS: update codeowner file by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3581
shellcheck: add shellcheck ci check by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3582
Build scripts for ADIOS2 with CUDA and Kokkos backend by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3579
Disable emu tests by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3583
cmake: set correct soname by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3585
TestBPChangingShapeWithinStep without and with various compressors by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3577
add release files by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3570
Heat Transfer Example: MPI Datatype by @ax3l in https://github.com/ornladios/ADIOS2/pull/3593
test,install: paramaterize adios2_DIR by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3586
ci,ascent: enable ascent builds by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3610
ci: use nvcc_wrapper in adiosKokkos by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3623
Adding SYCL support to the Kokkos backend by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3621
Bump requests from 2.28.1 to 2.31.0 in /docs by @dependabot in https://github.com/ornladios/ADIOS2/pull/3628
Fix flaky MSMPI installation in windows builds by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3627
docs: update whatsnew by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3634
ci,olcf,crusher: enable Crusher CI by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3620
ci,crusher: minor tweaks by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3635
Ascent,CI: do not report sync script build by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3637
replaced uint with size_t by @guj in https://github.com/ornladios/ADIOS2/pull/3640
crusher,ci: set unique env per pipeline by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3644
Update documentation for the GPU-aware functionality by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3639
work around with include on windows by @guj in https://github.com/ornladios/ADIOS2/pull/3652
Bug fix for Kokkos applications without GPU backend by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3654
An effort to make profiler more user friendly by @guj in https://github.com/ornladios/ADIOS2/pull/3660
Bump cryptography from 39.0.1 to 41.0.0 in /docs by @dependabot in https://github.com/ornladios/ADIOS2/pull/3650
ci: remove power8 emulated builds by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3667
ci,ascent: disable install tests by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3673
Tweaks for BP5 on windows. by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3682
Fix memory leak when there are Joined Arrays in streaming mode by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3684
Fix for the issue #3646. by @dmitry-ganyushin in https://github.com/ornladios/ADIOS2/pull/3687
cmake,evpath: namespace modules install dir by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3689
Ci rework gcc builds by @scottwittenburg in https://github.com/ornladios/ADIOS2/pull/3631
added gitattributes file by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3698
Update files with new EOL by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3699
build(deps): bump cryptography from 41.0.0 to 41.0.2 in /docs by @dependabot in https://github.com/ornladios/ADIOS2/pull/3697
ci,mpich: disable Sz|Zfp flaky tests by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3702
build(deps): bump pygments from 2.14.0 to 2.15.0 in /docs by @dependabot in https://github.com/ornladios/ADIOS2/pull/3704
ci,cmake: cleanup scripts subdirectory by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3625
cmake: moved logic to enable h5vol to detectoptions.cmake by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3701
build(deps): bump certifi from 2022.12.7 to 2023.7.22 in /docs by @dependabot in https://github.com/ornladios/ADIOS2/pull/3712
ci: increase windows timeout defsync and DataWrite by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3726
format,ci: update clang-format to clang16 by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3722
Enable Flush Test for BP5 by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3683
Use ccache in github actions builds by @scottwittenburg in https://github.com/ornladios/ADIOS2/pull/3705
ci: Fix conditional expression syntax to fix cache save by @scottwittenburg in https://github.com/ornladios/ADIOS2/pull/3730
ci: work around bug in actions/cache by @scottwittenburg in https://github.com/ornladios/ADIOS2/pull/3731
build(deps): bump cryptography from 41.0.2 to 41.0.3 in /docs by @dependabot in https://github.com/ornladios/ADIOS2/pull/3733
Version constants plus functions to retrieve list of ... by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3676
ci: add libcurl4 to all ubuntu images by @scottwittenburg in https://github.com/ornladios/ADIOS2/pull/3734
ci: update release instructions by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3738
ci: add downloads readthedocs by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3739
cmake: fix evpath plugins install path by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3737
Remote Data access via Get() or Transport by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3740
readthedocs: remove sys pkgs; use current ADIOS2 by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3743
Fix: Static Builds Against C-Blosc2 by @ax3l in https://github.com/ornladios/ADIOS2/pull/3715
CI: add two static builds by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3749
Upstream ffs by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3755
Update Readme.md with community section by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3700
cmake,readthedocs: update python enable instructions by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3713
cmake: correct info.h installation path by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3758
Korvo upstream by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3761
cmake: resolve cmake python deprecation warning by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3764
Update kwsys by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3765
ci: add mgard dependency to spack builds by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3753
Misc fixes by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3768
Add automated vulnerability code analysis using CodeQL. by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3606
cmake: correct ADIOS2Blosc2PREFER_SHARED behavior by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3767
match type of timestep for h5 engine to size_t (same as adios Variabl… by @guj in https://github.com/ornladios/ADIOS2/pull/3776
ci: Create static minimal build by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3760
Remove unused SelectionType values by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3784
Update ADIOS2 HDF5 VOL with basic set of capability flags by @jhendersonHDF in https://github.com/ornladios/ADIOS2/pull/3632
gha,ci: update checkout to v4 by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3789
ci: Add HDF5 to a windows build by @scottwittenburg in https://github.com/ornladios/ADIOS2/pull/3732
Fixup local var reading by block with test, master branch by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3787
Add MinBlocksInfo to BP5 writer engine by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3794
yaml-cpp: support 0.8.0 version by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3799
Delete experimental examples by @spyridon97 in https://github.com/ornladios/ADIOS2/pull/3800
Remove BP5 BetweenStepPairs variable that hides Engine.h counterpart by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3803
WIP: Reader-side Profiling by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3778
Fix the ADIOS_USE_{} variable names to use ADIOS2 by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3807
Pass Array Order of reader to remote server for proper Get() operation by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3810
Fix abs/rel step in BP5 DoCount by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3812
FFS upstream master-branch by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3815
added support to query BP5 files by @guj in https://github.com/ornladios/ADIOS2/pull/3809
install: export adios2 device variables by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3819
ci: fix codeql ignore path by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3772
Add Memory Selection to SST, and prototype a mechanism for running st… by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3823
Remote test redux by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3830
Tweak Remote class and test multi-threaded file remote access by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3834
MPI: add timeout for conf test for MPI DataPlane by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3848
cmake: update minimum cmake to 3.12 by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3849
MPI DataPlane: do not call MPI_Init internally by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3847
Fix destdir install test by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3850
Organize and add examples by @spyridon97 in https://github.com/ornladios/ADIOS2/pull/3820
Blosc2 USE ON: Fix Module Fallback by @ax3l in https://github.com/ornladios/ADIOS2/pull/3774
BP5Serializer: handle local variables that use operators by @caitlinross in https://github.com/ornladios/ADIOS2/pull/3859
Update examples by @spyridon97 in https://github.com/ornladios/ADIOS2/pull/3856
bug fix: syntax error in json output by @guj in https://github.com/ornladios/ADIOS2/pull/3857
Remove Slack as a contact mechanism by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3866
Remote Server Improvements by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3862
Fix hip example compilation error by @spyridon97 in https://github.com/ornladios/ADIOS2/pull/3865
new attempt to commit query support of local array. by @guj in https://github.com/ornladios/ADIOS2/pull/3868
cmake: link to MPI::MPI_Fortran using INTERFACE by @caitlinross in https://github.com/ornladios/ADIOS2/pull/3853
Backports from release_29 by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3858
Backport 3877 to master by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3882
Fix MPI Data plane cohort handling by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3588
Testing writing null blocks without affecting the read by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3878
docs: update new_release.md by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3892
cmake: find threads package first by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3893
Add tutorials by @spyridon97 in https://github.com/ornladios/ADIOS2/pull/3880
examples,hip: use find_package(hip) once in proj by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3896
Add Remote "-status" command to see if a server is running and where by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3911
Defining and computing derived variables by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3816
Fix printing issue in perfstub thirdparty code by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3913
ci,crusher: increase supported num branches by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3902
Change install directory of Gray scott files by @spyridon97 in https://github.com/ornladios/ADIOS2/pull/3905
ci: add shellcheck coverage to source and testing by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3915
Change install directory of Gray scott files again by @spyridon97 in https://github.com/ornladios/ADIOS2/pull/3917
SST with GPU support by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3918
ci: Set openmpi and openmp parameters to make an efficient use of the CI by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3908
Cleanup server output a bit by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3914
ReadMe.md: Mention 2.9.2 release by @eschnett in https://github.com/ornladios/ADIOS2/pull/3921
ci: use mpich built with ch3:sock to speed up tests by @scottwittenburg in https://github.com/ornladios/ADIOS2/pull/3883
Dataman with GPU support by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3928
CI Contract Build examples with external ADIOS by @spyridon97 in https://github.com/ornladios/ADIOS2/pull/3906
Revert "Fix printing issue in perfstub thirdparty code" by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3943
Perf stubs upstream by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3944
Fix the HIP CMAKE issues to run on Frontier by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3931
Set the compiler for the Kokkos DataMan example by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3946
Fix MATLAB bindings for MacOS by @tomgade09 in https://github.com/ornladios/ADIOS2/pull/3950
Delete VTK code in examples by @spyridon97 in https://github.com/ornladios/ADIOS2/pull/3952
Enable shell check scripts for existing scripts by @spyridon97 in https://github.com/ornladios/ADIOS2/pull/3940
Move useful docs from ADIOS2-examples to ADIOS2 by @spyridon97 in https://github.com/ornladios/ADIOS2/pull/3939
Use adios2modereadRandomAccess in matlab open to make it work for BP5 by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3956
Adapt libfabric dataplane of SST to Cray CXI provider by @franzpoeschel in https://github.com/ornladios/ADIOS2/pull/3672
Update readme for heat transfer example with new location and build i… by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3904
ci: fix false negative asan checks by @scottwittenburg in https://github.com/ornladios/ADIOS2/pull/3957
adios2::core::ADIOS: Initialize new IO objects with config file by @spyridon97 in https://github.com/ornladios/ADIOS2/pull/3955
MDR Operator by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3826
fix for ASAN issue related to JoinedDimArray handling in BP5 deserializer by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3963
Have HDF5 write raise error if operator(s) requested by @rupertnash in https://github.com/ornladios/ADIOS2/pull/3951
Http connector by @dmitry-ganyushin in https://github.com/ornladios/ADIOS2/pull/3890
Add pip packaging by @scottwittenburg in https://github.com/ornladios/ADIOS2/pull/3960
pip: also run pypi upload job on workflow_dispatch by @scottwittenburg in https://github.com/ornladios/ADIOS2/pull/3967
Campaign manager by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3932
update copyright.txt to include recent kitware authors by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3965
Bug fix building ADIOS2 without the campaign example by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3969
Several fixes for ASAN issues by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3959
SST ASAN buffer fix by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3972
python: Implement Python high level API in python by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3938

Changes added in the latest final release

Fix the C example stepping and reading data. Simplify code for shape/… by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3980
Fix read request length on BP5 Local arrays with operators (backport to release 2.10) by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3993
Fix 1: bpls handle when BP5 throws logic error on minmax for unsuppor… by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3994
python,cmake: update version script to support rc by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3992
Python API: Adios() new argument 'config_file', Stream() new argument… by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3984
Backport recent ASAN fixes to 2.10 release branch by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3985
Merge upstream dill chanages to release_210 by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/4005
Backport master commits by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/4029
Backport master PR into release210 by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/4042
backports from master by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/4059
Backports from master by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/4128
Bump version to v2.10.0 by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/4130

New Contributors

@jhendersonHDF made their first contribution in https://github.com/ornladios/ADIOS2/pull/3632
@spyridon97 made their first contribution in https://github.com/ornladios/ADIOS2/pull/3800
@tomgade09 made their first contribution in https://github.com/ornladios/ADIOS2/pull/3950
@rupertnash made their first contribution in https://github.com/ornladios/ADIOS2/pull/3951

Full Changelog: https://github.com/ornladios/ADIOS2/compare/v2.9.2...v2.10.0

- C++
Published by vicentebolea about 2 years ago

https://github.com/ornladios/adios2 - v2.10.0-rc1

What's Changed

ci: migrate to almalinux, cuda base image by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3484
ci: update kokkos to 3.7 by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3490
codeowners: added code owners file by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3496
Hdf5 vol support will require latest hdf5 1.14 by @guj in https://github.com/ornladios/ADIOS2/pull/3516
Bump numpy from 1.21.6 to 1.22.0 in /docs by @dependabot in https://github.com/ornladios/ADIOS2/pull/3535
Bump cryptography from 38.0.2 to 39.0.1 in /docs by @dependabot in https://github.com/ornladios/ADIOS2/pull/3534
Release 29 by @dmitry-ganyushin in https://github.com/ornladios/ADIOS2/pull/3545
Enable Linux Emulation build (Power8, IBM XL) by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3537
Setting code owners for the GPU files by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3561
Release 29 by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3573
Allowing Put/Get to receive Kokkos Views with HIP backend by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3571
Prototype DAOS engine by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3576
CODEOWNERS: update codeowner file by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3581
shellcheck: add shellcheck ci check by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3582
Build scripts for ADIOS2 with CUDA and Kokkos backend by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3579
Disable emu tests by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3583
cmake: set correct soname by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3585
TestBPChangingShapeWithinStep without and with various compressors by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3577
add release files by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3570
Heat Transfer Example: MPI Datatype by @ax3l in https://github.com/ornladios/ADIOS2/pull/3593
test,install: paramaterize adios2_DIR by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3586
ci,ascent: enable ascent builds by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3610
ci: use nvcc_wrapper in adiosKokkos by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3623
Adding SYCL support to the Kokkos backend by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3621
Bump requests from 2.28.1 to 2.31.0 in /docs by @dependabot in https://github.com/ornladios/ADIOS2/pull/3628
Fix flaky MSMPI installation in windows builds by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3627
docs: update whatsnew by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3634
ci,olcf,crusher: enable Crusher CI by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3620
ci,crusher: minor tweaks by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3635
Ascent,CI: do not report sync script build by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3637
replaced uint with size_t by @guj in https://github.com/ornladios/ADIOS2/pull/3640
crusher,ci: set unique env per pipeline by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3644
Update documentation for the GPU-aware functionality by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3639
work around with include on windows by @guj in https://github.com/ornladios/ADIOS2/pull/3652
Bug fix for Kokkos applications without GPU backend by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3654
An effort to make profiler more user friendly by @guj in https://github.com/ornladios/ADIOS2/pull/3660
Bump cryptography from 39.0.1 to 41.0.0 in /docs by @dependabot in https://github.com/ornladios/ADIOS2/pull/3650
ci: remove power8 emulated builds by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3667
ci,ascent: disable install tests by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3673
Tweaks for BP5 on windows. by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3682
Fix memory leak when there are Joined Arrays in streaming mode by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3684
Fix for the issue #3646. by @dmitry-ganyushin in https://github.com/ornladios/ADIOS2/pull/3687
cmake,evpath: namespace modules install dir by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3689
Ci rework gcc builds by @scottwittenburg in https://github.com/ornladios/ADIOS2/pull/3631
added gitattributes file by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3698
Update files with new EOL by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3699
build(deps): bump cryptography from 41.0.0 to 41.0.2 in /docs by @dependabot in https://github.com/ornladios/ADIOS2/pull/3697
ci,mpich: disable Sz|Zfp flaky tests by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3702
build(deps): bump pygments from 2.14.0 to 2.15.0 in /docs by @dependabot in https://github.com/ornladios/ADIOS2/pull/3704
ci,cmake: cleanup scripts subdirectory by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3625
cmake: moved logic to enable h5vol to detectoptions.cmake by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3701
build(deps): bump certifi from 2022.12.7 to 2023.7.22 in /docs by @dependabot in https://github.com/ornladios/ADIOS2/pull/3712
ci: increase windows timeout defsync and DataWrite by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3726
format,ci: update clang-format to clang16 by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3722
Enable Flush Test for BP5 by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3683
Use ccache in github actions builds by @scottwittenburg in https://github.com/ornladios/ADIOS2/pull/3705
ci: Fix conditional expression syntax to fix cache save by @scottwittenburg in https://github.com/ornladios/ADIOS2/pull/3730
ci: work around bug in actions/cache by @scottwittenburg in https://github.com/ornladios/ADIOS2/pull/3731
build(deps): bump cryptography from 41.0.2 to 41.0.3 in /docs by @dependabot in https://github.com/ornladios/ADIOS2/pull/3733
Version constants plus functions to retrieve list of ... by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3676
ci: add libcurl4 to all ubuntu images by @scottwittenburg in https://github.com/ornladios/ADIOS2/pull/3734
ci: update release instructions by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3738
ci: add downloads readthedocs by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3739
cmake: fix evpath plugins install path by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3737
Remote Data access via Get() or Transport by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3740
readthedocs: remove sys pkgs; use current ADIOS2 by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3743
Fix: Static Builds Against C-Blosc2 by @ax3l in https://github.com/ornladios/ADIOS2/pull/3715
CI: add two static builds by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3749
Upstream ffs by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3755
Update Readme.md with community section by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3700
cmake,readthedocs: update python enable instructions by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3713
cmake: correct info.h installation path by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3758
Korvo upstream by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3761
cmake: resolve cmake python deprecation warning by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3764
Update kwsys by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3765
ci: add mgard dependency to spack builds by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3753
Misc fixes by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3768
Add automated vulnerability code analysis using CodeQL. by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3606
cmake: correct ADIOS2Blosc2PREFER_SHARED behavior by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3767
match type of timestep for h5 engine to size_t (same as adios Variabl… by @guj in https://github.com/ornladios/ADIOS2/pull/3776
ci: Create static minimal build by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3760
Remove unused SelectionType values by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3784
Update ADIOS2 HDF5 VOL with basic set of capability flags by @jhendersonHDF in https://github.com/ornladios/ADIOS2/pull/3632
gha,ci: update checkout to v4 by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3789
ci: Add HDF5 to a windows build by @scottwittenburg in https://github.com/ornladios/ADIOS2/pull/3732
Fixup local var reading by block with test, master branch by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3787
Add MinBlocksInfo to BP5 writer engine by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3794
yaml-cpp: support 0.8.0 version by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3799
Delete experimental examples by @spyridon97 in https://github.com/ornladios/ADIOS2/pull/3800
Remove BP5 BetweenStepPairs variable that hides Engine.h counterpart by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3803
WIP: Reader-side Profiling by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3778
Fix the ADIOS_USE_{} variable names to use ADIOS2 by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3807
Pass Array Order of reader to remote server for proper Get() operation by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3810
Fix abs/rel step in BP5 DoCount by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3812
FFS upstream master-branch by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3815
added support to query BP5 files by @guj in https://github.com/ornladios/ADIOS2/pull/3809
install: export adios2 device variables by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3819
ci: fix codeql ignore path by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3772
Add Memory Selection to SST, and prototype a mechanism for running st… by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3823
Remote test redux by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3830
Tweak Remote class and test multi-threaded file remote access by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3834
MPI: add timeout for conf test for MPI DataPlane by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3848
cmake: update minimum cmake to 3.12 by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3849
MPI DataPlane: do not call MPI_Init internally by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3847
Fix destdir install test by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3850
Organize and add examples by @spyridon97 in https://github.com/ornladios/ADIOS2/pull/3820
Blosc2 USE ON: Fix Module Fallback by @ax3l in https://github.com/ornladios/ADIOS2/pull/3774
BP5Serializer: handle local variables that use operators by @caitlinross in https://github.com/ornladios/ADIOS2/pull/3859
Update examples by @spyridon97 in https://github.com/ornladios/ADIOS2/pull/3856
bug fix: syntax error in json output by @guj in https://github.com/ornladios/ADIOS2/pull/3857
Remove Slack as a contact mechanism by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3866
Remote Server Improvements by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3862
Fix hip example compilation error by @spyridon97 in https://github.com/ornladios/ADIOS2/pull/3865
new attempt to commit query support of local array. by @guj in https://github.com/ornladios/ADIOS2/pull/3868
cmake: link to MPI::MPI_Fortran using INTERFACE by @caitlinross in https://github.com/ornladios/ADIOS2/pull/3853
Backports from release_29 by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3858
Backport 3877 to master by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3882
Fix MPI Data plane cohort handling by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3588
Testing writing null blocks without affecting the read by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3878
docs: update new_release.md by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3892
cmake: find threads package first by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3893
Add tutorials by @spyridon97 in https://github.com/ornladios/ADIOS2/pull/3880
examples,hip: use find_package(hip) once in proj by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3896
Add Remote "-status" command to see if a server is running and where by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3911
Defining and computing derived variables by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3816
Fix printing issue in perfstub thirdparty code by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3913
ci,crusher: increase supported num branches by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3902
Change install directory of Gray scott files by @spyridon97 in https://github.com/ornladios/ADIOS2/pull/3905
ci: add shellcheck coverage to source and testing by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3915
Change install directory of Gray scott files again by @spyridon97 in https://github.com/ornladios/ADIOS2/pull/3917
SST with GPU support by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3918
ci: Set openmpi and openmp parameters to make an efficient use of the CI by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3908
Cleanup server output a bit by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3914
ReadMe.md: Mention 2.9.2 release by @eschnett in https://github.com/ornladios/ADIOS2/pull/3921
ci: use mpich built with ch3:sock to speed up tests by @scottwittenburg in https://github.com/ornladios/ADIOS2/pull/3883
Dataman with GPU support by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3928
CI Contract Build examples with external ADIOS by @spyridon97 in https://github.com/ornladios/ADIOS2/pull/3906
Revert "Fix printing issue in perfstub thirdparty code" by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3943
Perf stubs upstream by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3944
Fix the HIP CMAKE issues to run on Frontier by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3931
Set the compiler for the Kokkos DataMan example by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3946
Fix MATLAB bindings for MacOS by @tomgade09 in https://github.com/ornladios/ADIOS2/pull/3950
Delete VTK code in examples by @spyridon97 in https://github.com/ornladios/ADIOS2/pull/3952
Enable shell check scripts for existing scripts by @spyridon97 in https://github.com/ornladios/ADIOS2/pull/3940
Move useful docs from ADIOS2-examples to ADIOS2 by @spyridon97 in https://github.com/ornladios/ADIOS2/pull/3939
Use adios2modereadRandomAccess in matlab open to make it work for BP5 by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3956
Adapt libfabric dataplane of SST to Cray CXI provider by @franzpoeschel in https://github.com/ornladios/ADIOS2/pull/3672
Update readme for heat transfer example with new location and build i… by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3904
ci: fix false negative asan checks by @scottwittenburg in https://github.com/ornladios/ADIOS2/pull/3957
adios2::core::ADIOS: Initialize new IO objects with config file by @spyridon97 in https://github.com/ornladios/ADIOS2/pull/3955
MDR Operator by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3826
fix for ASAN issue related to JoinedDimArray handling in BP5 deserializer by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3963
Have HDF5 write raise error if operator(s) requested by @rupertnash in https://github.com/ornladios/ADIOS2/pull/3951
Http connector by @dmitry-ganyushin in https://github.com/ornladios/ADIOS2/pull/3890
Add pip packaging by @scottwittenburg in https://github.com/ornladios/ADIOS2/pull/3960
pip: also run pypi upload job on workflow_dispatch by @scottwittenburg in https://github.com/ornladios/ADIOS2/pull/3967
Campaign manager by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3932
update copyright.txt to include recent kitware authors by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3965
Bug fix building ADIOS2 without the campaign example by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3969
Several fixes for ASAN issues by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3959
SST ASAN buffer fix by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3972
python: Implement Python high level API in python by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3938

New Contributors

@jhendersonHDF made their first contribution in https://github.com/ornladios/ADIOS2/pull/3632
@spyridon97 made their first contribution in https://github.com/ornladios/ADIOS2/pull/3800
@tomgade09 made their first contribution in https://github.com/ornladios/ADIOS2/pull/3950
@rupertnash made their first contribution in https://github.com/ornladios/ADIOS2/pull/3951

Full Changelog: https://github.com/ornladios/ADIOS2/compare/v2.9.2...v2.10.0-rc1

- C++
Published by vicentebolea over 2 years ago

https://github.com/ornladios/adios2 - ADIOS v2.9.2 Production Release

Breaking changes

BP5 is the default engine wince 2.9.0 that behaves differently from BP3/BP4. Read the notes in the 2.9.0 release.

What's Changed

Organize the processes into groups so that the two steps of metadata … by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3750
bp5dbg parse records and check sizes during it for mmd.0 and md.0. No… by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3759
Fix ChunkV maintaining CurOffset when downsizing current chunk in All… by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3769
BP5Deserialize: modify changes to keep abi compt by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3773
Backport readthedocs fixes by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3783
Fix local values block reading by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3782
Set AWS version requirement to 1.10.15 and also turn it OFF by defaul… by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3804
bpls -l with scalar string variable: print the value (since min/max … by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3805
FFS upstream by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3814
Backport from master by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3861
Fix data length calculation for hash by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3875
Replace LookupWriterRec's linear search on RecList with an unordered_… by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3877
Merge pull request #3823 from eisenhauer/SstMemSel by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3871
bp5: make RecMap an static anon namespaced var by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3881
Merge pull request #3588 from vicentebolea/fix-mpi-dp by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3885
Merge pull request #3878 from anagainaru/test-null-blocks by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3886
Bump version to v2.9.2 by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3888
Fix destdir install test by @vicentebolea in #3850
cmake: update minimum cmake to 3.12 by @vicentebolea in #3849
MPI: add timeout for conf test for MPI DataPlane by @vicentebolea in #3848
MPI DataPlane: do not call MPI_Init internally by @vicentebolea in #3847
install: export adios2 device variables by @vicentebolea in #3819
Fix abs/rel step in BP5 DoCount by @eisenhauer in #3812
yaml-cpp: support 0.8.0 version by @vicentebolea in #3799
gha,ci: update checkout to v4 by @vicentebolea in #3789
Blosc2 USE ON: Fix Module Fallback by @ax3l in #3774
cmake: correct ADIOS2Blosc2PREFER_SHARED behavior by @vicentebolea in #3767
ci: Create static minimal build by @vicentebolea in #3760
cmake: correct info.h installation path by @vicentebolea in #3758
ci: add downloads readthedocs by @vicentebolea in #3739
cmake: fix evpath plugins install path by @vicentebolea in #3737
Fix: Static Builds Against C-Blosc2 by @ax3l in #3715
Fix: Static Builds Against C-Blosc2 by @ax3l in #3715
ci: add downloads readthedocs by @vicentebolea in #3739
ci: Create static minimal build by @vicentebolea in #3760
cmake: correct info.h installation path by @vicentebolea in #3758
cmake: correct ADIOS2Blosc2PREFER_SHARED behavior by @vicentebolea in #3767
gha,ci: update checkout to v4 by @vicentebolea in #3789
Fix abs/rel step in BP5 DoCount by @eisenhauer in #3812

Full Changelog: https://github.com/ornladios/ADIOS2/compare/v2.9.1...v2.9.2

- C++
Published by vicentebolea over 2 years ago

https://github.com/ornladios/adios2 - ADIOS v2.9.1 Production Release

Pull Requests backported

| PR | Title | | ----- | ---------------------------------------------------------------- | | #3676 | Version constants plus functions to retrieve list of ... | | #3697 | build(deps): bump cryptography from 41.0.0 to 41.0.2 in /docs | | #3698 | added gitattributes file | | #3704 | build(deps): bump pygments from 2.14.0 to 2.15.0 in /docs | | #3712 | build(deps): bump certifi from 2022.12.7 to 2023.7.22 in /docs | | #3687 | Fix for the issue #3646. | | #3699 | Update files with new EOL | | #3690 | Merge pull request #3689 from vicentebolea/fix-evpath-modules | | #3689 | cmake,evpath: namespace modules install dir | | #3684 | Fix memory leak when there are Joined Arrays in streaming mode | | #3652 | work around with include on windows | | #3650 | Bump cryptography from 39.0.1 to 41.0.0 in /docs | | #3673 | ci,ascent: disable install tests | | #3681 | Fix: std::min w/ windows.h in C-Blosc2 | | #3657 | Allow Span in files opened for Append | | #3623 | ci: use nvcc_wrapper in adiosKokkos | | #3619 | Backports bp5 bugfix | | #3605 | Adding missing HIP specific functionality | | #3607 | Backport: Heat Transfer Example: MPI Datatype #3593 | | #3593 | Heat Transfer Example: MPI Datatype | | #3591 | Implement BP5 (and BP4) reader-side memory selection, do testing |

Full Changelog: https://github.com/ornladios/ADIOS2/compare/v2.9.0...v2.9.1

- C++
Published by vicentebolea almost 3 years ago

https://github.com/ornladios/adios2 - ADIOS v2.9.0 Production Release

Summary

This is a major release with new features and lots of bug fixes.

General

GPU-Aware I/O enabled by using Kokkos. Device pointers can be passed to Put()/Get() calls directly. Kokkos 3.7.x required for this release. Works with CUDA, HIP and Kokkos applications. https://adios2.readthedocs.io/en/latest/advanced/gpu_aware.html#gpu-aware-i-o
GPU-compression. MGARD and ZFP operators can compress data on GPU if they are built for GPU. MGARD operator can be fed with host/device pointers and will move data automaticaly. ZFP operator requires matching data and compressor location.
Joined Array concept (besides Global Array and Local Array), which lets writers dump Local Arrays (no offsets no global shape) that are put together into a Global Array by the reader. One dimension of the arrays is selected for this join operation, while other dimensions must be the same for all writers. https://adios2.readthedocs.io/en/latest/components/components.html?highlight=Joined#shapes

File I/O

Default File engine is now BP5. If for some reason this causes problems, manually specify using "BP4" for your application.
BP5 engine supports multithreaded reading to accelerate read performance for low-core counts.
BP5 Two level metadata aggregation and reduction reduced memory impact of collecting metadata and therefore is more scalable in terms of numbers of variables and writers than BP4.
Uses Blosc-2 instead of Blosc for lossless compression. The new compression operator is backward compatible with old files compressed with blosc. The name of the operator remains "blosc".

Staging

UCX dataplane added for SST staging engine to support networks under the UCX consortium
MPI dataplane added for SST staging engine. It relies on MPI intercommunicators to connect multiple independent MPI applications for staging purposes. Applications must enable multithreaded MPI for this dataplane.

Experimental features

Preliminary support for data structs. A struct can have single variables of basic types, and 1D fixed size arrays of basic types. Supported by BP5, SST and SSC engines.

What's Changed

Refactor SSC to allow easy addition of new sub-routines by @JasonRuonanWang in https://github.com/ornladios/ADIOS2/pull/3092
BP5Deserializer GenerateReadRequests fix rank fix by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3096
Added naive engine mode for SSC by @JasonRuonanWang in https://github.com/ornladios/ADIOS2/pull/3109
move sodium option to its right place by @JasonRuonanWang in https://github.com/ornladios/ADIOS2/pull/3112
Bump pillow from 9.0.0 to 9.0.1 in /docs by @dependabot in https://github.com/ornladios/ADIOS2/pull/3103
added bp file reading in MgardPlus by @JasonRuonanWang in https://github.com/ornladios/ADIOS2/pull/3113
added zero block test for ssc by @JasonRuonanWang in https://github.com/ornladios/ADIOS2/pull/3111
fixed a bug in ssc which blocks MPI communications by @JasonRuonanWang in https://github.com/ornladios/ADIOS2/pull/3114
clean up SSC logging information for better debugging experience by @JasonRuonanWang in https://github.com/ornladios/ADIOS2/pull/3115
SSC: temporary workaround for crusher MPI bug by @JasonRuonanWang in https://github.com/ornladios/ADIOS2/pull/3117
DILL upstream by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3118
Fix windows mpi builds by @chuckatkins in https://github.com/ornladios/ADIOS2/pull/3142
remove template instantiation in header files by @JasonRuonanWang in https://github.com/ornladios/ADIOS2/pull/3144
MgardPlus: check for empty mesh file by @JasonRuonanWang in https://github.com/ornladios/ADIOS2/pull/3157
pr3153 into master by @chuckatkins in https://github.com/ornladios/ADIOS2/pull/3158
blosc unknown parameter should be warning by @JasonRuonanWang in https://github.com/ornladios/ADIOS2/pull/3163
Fix a template instantiation issue with GCC 10 in dataman serializer by @JasonRuonanWang in https://github.com/ornladios/ADIOS2/pull/3165
ENET upstream by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3166
Fix helloBPTimeWriter Python example. by @manauref in https://github.com/ornladios/ADIOS2/pull/3167
removed VariableCompound class as it has never worked since added by @JasonRuonanWang in https://github.com/ornladios/ADIOS2/pull/3168
move Span class to dedicated files by @JasonRuonanWang in https://github.com/ornladios/ADIOS2/pull/3169
Add template-free C++ API by @JasonRuonanWang in https://github.com/ornladios/ADIOS2/pull/3171
on-demand step delivery by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3170
added missing open modes in C API by @JasonRuonanWang in https://github.com/ornladios/ADIOS2/pull/3174
Add struct variable support by @JasonRuonanWang in https://github.com/ornladios/ADIOS2/pull/3179
added DefineStructVariable test by @JasonRuonanWang in https://github.com/ornladios/ADIOS2/pull/3183
moved Variable::Shape() to VariableBase by @JasonRuonanWang in https://github.com/ornladios/ADIOS2/pull/3184
removed VariableBase::GetShape by @JasonRuonanWang in https://github.com/ornladios/ADIOS2/pull/3185
Remove templates in SSC writer by @JasonRuonanWang in https://github.com/ornladios/ADIOS2/pull/3187
removed most templated functions in ssc reader by @JasonRuonanWang in https://github.com/ornladios/ADIOS2/pull/3188
added ssc test for struct variables by @JasonRuonanWang in https://github.com/ornladios/ADIOS2/pull/3189
added IO::m_StructDefinitions to hold temporary struct definitions from parsing step metadata by @JasonRuonanWang in https://github.com/ornladios/ADIOS2/pull/3191
fixed a bug in ssc test that fails when mpi size is large by @JasonRuonanWang in https://github.com/ornladios/ADIOS2/pull/3192
Add MinMax in non-template API by @JasonRuonanWang in https://github.com/ornladios/ADIOS2/pull/3193
Tweak OnDemand test by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3190
Add BlocksInfo in SSC naive by @JasonRuonanWang in https://github.com/ornladios/ADIOS2/pull/3194
finish up polishing tests and examples by @JasonRuonanWang in https://github.com/ornladios/ADIOS2/pull/3196
moved ToBlocksInfoMin to private by @JasonRuonanWang in https://github.com/ornladios/ADIOS2/pull/3197
added BlocksInfo API for VariableNT, and implementation for VariableStruct in SSC by @JasonRuonanWang in https://github.com/ornladios/ADIOS2/pull/3198
make struct size parameter compulsory to ensure padding correctness by @JasonRuonanWang in https://github.com/ornladios/ADIOS2/pull/3199
added freeze function to StructDefinition by @JasonRuonanWang in https://github.com/ornladios/ADIOS2/pull/3200
added allowReorganize for InquireStructVariable by @JasonRuonanWang in https://github.com/ornladios/ADIOS2/pull/3201
Add SSC struct definition serialization by @JasonRuonanWang in https://github.com/ornladios/ADIOS2/pull/3202
Update documentation with correct StepDistributionMode values by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3212
Add access to internal m_BetweenStepPairs member by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3216
Bp5 read multithreaded by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3220
Bp5 multithreaded read, dynamic version by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3233
Throw an error when GathervArrays() is asked to gather more than 2^31… by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3239
Stop using Dims in some performance critical areas by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3240
Merge bp5 flush read from 281 by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3243
Fix cuda build by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3246
If BP5 parameter Threads is 0 (default), calculate the number of thre… by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3247
fix api for gcc11 by @robertu94 in https://github.com/ornladios/ADIOS2/pull/3250
BP5 Two level metadata aggregation and reduction: work in progress saved by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3251
Remove DebugMode from all external APIs by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3204
FFS upstream by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3259
Fix C binding problem by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3264
BP5 subformats by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3260
Ensure Init of Blocks Info by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3268
Kill reader-created variables in parent IO upon engine close (for BP5) by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3270
Changed parameters for DAOS API functions. by @dmitry-ganyushin in https://github.com/ornladios/ADIOS2/pull/3266
Add backward compatible read with BLOSC compression to BP4 files crea… by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3277
Necessary revision to get VOL work HDF5 1.13 & BP4 by @guj in https://github.com/ornladios/ADIOS2/pull/3279
Destructor close by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3278
Destroy only created vars on BeginStep and again on Reader close by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3272
VOL: added BP5 read support for H5 style files (no steps) by @guj in https://github.com/ornladios/ADIOS2/pull/3282
Bump version to v2.8.2 on master by @chuckatkins in https://github.com/ornladios/ADIOS2/pull/3281
Fix a typo. by @hyoklee in https://github.com/ornladios/ADIOS2/pull/3283
fix bpls: if minblockinfo is available, don't just still use allsteps… by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3288
CI: adds CUDA build by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3276
Fix bpls dumping local arrays from BP5 files. Also renamed WasLocalVa… by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3289
SST: Add MPI SST dataplane by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3095
MPI_DP: Optimize scalable component by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3292
Optimize processing attributes, by using a new AttributeBase.Equals()… by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3299
Span MinMax in BP5 by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3306
ci: fix macos install test missing deps by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3308
Enable memory selection in ssc reader by @JasonRuonanWang in https://github.com/ornladios/ADIOS2/pull/3313
zfp: fix version compatibility for 1.0+ by @chuckatkins in https://github.com/ornladios/ADIOS2/pull/3312
Rework BP5 Attribute handling by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3304
fix compile error in dataspace engine by @JasonRuonanWang in https://github.com/ornladios/ADIOS2/pull/3317
When reading with different order (column <-> row order), do not reve… by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3315
Fix MinInfo Leak, add tests by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3316
TESTS, Examples: Add comment for MPITHREADMULTIPLE by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3293
Modifiable attributes for staging and BP5 by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3318
cmake: make find_package(ZFP) quiet by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3319
Fix compilation problem on crusher by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3321
SST with MPI dataplane: only print MPI initialization warning on rank 0 by @cwsmith in https://github.com/ornladios/ADIOS2/pull/3322
Reinstall Norbert's BP5Deserializer opts by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3327
CI: update macOS version for github OS builds by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3325
utils: Fix bpls linking in Windows by @Biswa96 in https://github.com/ornladios/ADIOS2/pull/3242
Restore inadverent changes to Attr handling by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3333
CUDA: remove cuda_drivers linking dep by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3334
CI: major rework for image generations of GHA by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3331
CI: add cuda_lambda to Kokkos deployment by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3337
AdiosView object that allows Get/Put to receive Kokkos::View by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3320
CMAKE: CMAKECXXEXTENSIONS=OFF by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3341
CMAKE: Use Ninja for gcc[9,10,11] by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3338
CI: provide catalyst to gcc11 by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3340
Untemplate BP5Writer::PutCommon by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3343
Handshake fix: rank 0 should receive msg from last rank, not from -1. by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3345
Support Structs in BP5 marshalling (BP5file and SST), modify an SSC t… by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3350
CMAKE: Support cmake 3.24 by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3351
Fix race condition in OnDemand timestep distribution by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3355
EVpath upstream updates by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3360
SST reader tries to use the writer's DataPlane or fail by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3356
Handle error on SST contact file create by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3364
Fix race, restore multi-reader OnDemand test by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3361
ParaView Catalyst In Situ engine plugin by @caitlinross in https://github.com/ornladios/ADIOS2/pull/3346
Fix race condition in OnDemand delivery by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3369
Added BLOSC2 operator using the c-blosc-2 library. It works identical… by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3374
Remove closed file from map of transports so that we don't attempt to close it again by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3375
Raise the limit of open filesi to max in BP file engines by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3376
Clean-up by @dmitry-ganyushin in https://github.com/ornladios/ADIOS2/pull/3380
repeat setup from Init in Execute (Caitlin's fix) by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3384
Fix conversion warning in CompressPNG.cpp by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3389
BP5 read direct to application memory (1 dimensional case) by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3387
WIP: Fix untyped conversion in PNG operator test by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3390
CI: Workaround for Windows builds 20221120.1 by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3394
Convert Transport parameters's keys to lower case for easier lookup. by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3388
Add new parameters to bpls to control engine, engine params and trans… by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3399
Change (at least) some sprintf() to snprintf() by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3401
Added a simple example for Fides: a single hexagon cell output by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3404
Add Python SstWriter example by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3403
Revert "CI: Workaround for Windows builds 20221120.1" by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3405
Move Ssc-internal struct manipulation to Ssc-Internal Maps by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3406
Rename some portions of Struct interface by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3409
GPU buffers will never bypass the ADIOS internal buffers by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3410
Adjust DefineStruct bindings by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3414
install: post install test checks for jail root builds by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3415
Clean the GPU code to allow multiple backends by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3411
CMAKE: Check usability of MPIPortopen by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3407
Dill upstream by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3420
Testing for GPU backend for correct/incorrect memory spaces by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3421
Allowing the option of CUDA memory space only if CUDA is enabled by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3423
SST: Add UCX SST Dataplane by @sameehj in https://github.com/ornladios/ADIOS2/pull/3416
Added the AWSSDK transport using the AWS SDK S3 API. by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3425
When we use AWSSDK, bpls should not check if the file it exists loca… by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3431
Enforce minimum UCX version by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3432
Kill warnings in UCX dataplane by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3435
Extend FindUCX to work if pkgconfig is missing or ignored by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3437
Fix MGARD operator by @JieyangChen7 in https://github.com/ornladios/ADIOS2/pull/3438
Blosc by @lizdulac in https://github.com/ornladios/ADIOS2/pull/3430
Do not use the deprecated ctime(), use instead localtime_r() and strf… by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3441
Fix BP5 Append: fix metadata index position calculation for AppendAft… by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3447
sst,mpi: renable ADIOS2SSTHAVE_MPI by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3449
Fig bug in the IME file transport by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3451
Reader-side struct API changes by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3428
Testing for MGARD operator with GPU buffers by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3434
Tweak min blocks interface to avoid leak in Shape() by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3458
Fix UCX version checking by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3464
Added support for Joined Arrays in the BP4 format and engine. by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3466
Warning fix when finding the MGARD library by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3467
Fix bug in BP3/BP4 attribute merging process. When a global attribute… by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3468
Add JoinedArray test in staging-common, limit to BP4 currently by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3469
Bpls daos extention by @dmitry-ganyushin in https://github.com/ornladios/ADIOS2/pull/3474
Fixed: When appending to BP5 with a different number of processes, bp… by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3472
Better error when libfabric defaults to or is specified a bad FABRIC_IFACE by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3475
release: Bump version to v2.9.0-rc1 by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3476
Fix misplaced *s in sst.rst by @tobyjamez in https://github.com/ornladios/ADIOS2/pull/3471
Fix build error on Summit for PGI with CUDA backend by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3481
Make adios_iotest work with SST again: add CurrentStep() to SstWriter… by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3482
Fix Matlab build which depends on the C bindings not the C++ bindings. by @pnorbert in https://github.com/ornladios/ADIOS2/pull/3489
BP5 JoinedArray support by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3488
Add JoinedArrays to SST in BP5 marshaling by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3491
Kokkos backend in ADIOS2 by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3446
Update the documentation for the GPU backend by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3495
kokkos,cmake: remove kokkosview example by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3498
Remove deprecated bindings for release by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3494
ci,kokkos,cuda: remove warnings by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3500
cmake: do not install empty sst dirs by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3501
Workaround for lammps and Scorpion to have a green CI by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3502
Update fortran.rst by @mathrack in https://github.com/ornladios/ADIOS2/pull/3506
Change default marshalling for SST to BP5 by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3513
Set the CMAKE GPU architectures to what is used for Kokkos by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3517
Move the buffer copy logic outside operators by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3514
Add adios2modereadRandomAccess to fortran bindings by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3522
Fix plugin engine example by @caitlinross in https://github.com/ornladios/ADIOS2/pull/3499
update docs/README.md for using conda env by @caitlinross in https://github.com/ornladios/ADIOS2/pull/3427
Dependbot backports by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3536
Bug fix in BP5 when memory selection is used with GPU buffers by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3539
Fix bug when writing null blocks by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3542
Throw an exception in Python bindings on Get() buffer type mismatch by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3544
Throw an exception if an unimplemented function is called in BP5 by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3543
Fix for the issue 3503. by @dmitry-ganyushin in https://github.com/ornladios/ADIOS2/pull/3519
H5 subfile support by @guj in https://github.com/ornladios/ADIOS2/pull/3530
Refactoring and clean-up group API by @dmitry-ganyushin in https://github.com/ornladios/ADIOS2/pull/3557
Change default file engine to BP5 by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3526
I3454 missing mpi functions in doc by @dmitry-ganyushin in https://github.com/ornladios/ADIOS2/pull/3558
Add documentation for Joined Arrays by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3554
Collect Shape at EndStep (and restore DataWrite test) by @eisenhauer in https://github.com/ornladios/ADIOS2/pull/3562
Bug fix for the Kokkos backend used with a Kokkos application by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3560
Throw an error when unsupported parameters are used with the ZFP CUDA backend by @anagainaru in https://github.com/ornladios/ADIOS2/pull/3567
Bump version to v2.9.0 by @vicentebolea in https://github.com/ornladios/ADIOS2/pull/3568

New Contributors

@manauref made their first contribution in https://github.com/ornladios/ADIOS2/pull/3167
@robertu94 made their first contribution in https://github.com/ornladios/ADIOS2/pull/3250
@hyoklee made their first contribution in https://github.com/ornladios/ADIOS2/pull/3283
@cwsmith made their first contribution in https://github.com/ornladios/ADIOS2/pull/3322
@Biswa96 made their first contribution in https://github.com/ornladios/ADIOS2/pull/3242
@sameehj made their first contribution in https://github.com/ornladios/ADIOS2/pull/3416
@lizdulac made their first contribution in https://github.com/ornladios/ADIOS2/pull/3430
@tobyjamez made their first contribution in https://github.com/ornladios/ADIOS2/pull/3471
@mathrack made their first contribution in https://github.com/ornladios/ADIOS2/pull/3506

Full Changelog: https://github.com/ornladios/ADIOS2/compare/v2.8.0...v2.9.0

- C++
Published by vicentebolea about 3 years ago

https://github.com/ornladios/adios2 - ADIOS v2.9.0-rc1 Production Release

- C++
Published by vicentebolea over 3 years ago

https://github.com/ornladios/adios2 - ADIOS v2.8.3 Production Patch Release

This is a patch release that fixes two specific issues: * #3287 Allow the inline engine to work with ParaView Catalyst Live * #3291 Fix bpls issues with local arrays

- C++
Published by chuckatkins almost 4 years ago

https://github.com/ornladios/adios2 - ADIOS v2.8.2 Production Patch Release

Summary

This is a patch release to fix a compatibility issue with BP4 and BLOSC compressed data arrays. It is essentially v2.8.1 with the addition of PR #3275.

- C++
Published by chuckatkins almost 4 years ago

https://github.com/ornladios/adios2 - ADIOS v2.8.1 Production Patch Release

Summary

This is a patch release to address many of the issues identified with the v2.8.0 release after it was published. For the new BP5 engine the fixes are - SelectSteps were always showing the first step in the file even if not selected - AppendAfterStep created a corrupted index file - bpls did not show the actual values of Global Values and Local Values (shown as 1D arrays) - bpls was not able to dump data of variables beyond the first step

Changes / Fixes

PR #3237 PR #3236 PR #3234 PR #3228 PR #3227 PR #3221 PR #3213 PR #3211 PR #3207 PR #3206 PR #3186 PR #3176 PR #3162 PR #3161 PR #3156 PR #3155 PR #3145 PR #3153

- C++
Published by chuckatkins almost 4 years ago

https://github.com/ornladios/adios2 - ADIOS v2.8.0 Production Release

Summary

This is a major production release with a variety of new features and bug fixes.

Changes to the API

adios2::Mode::ReadRandomAccess mode is introduced for reading files with access to all steps. BeginStep/EndStep calls are NOT allowed. SetStepSelection() can be used to access specific steps in the file.
adios2::Mode::Read mode now requires using BeginStep/EndStep loop to access steps serially one by one. Variable inquiry fails outside BeginStep/EndStep sections. You need to modify your Open() statement to use the random-access mode if your code is accessing all steps in an existing file. Technically, this is only true for the new BP5 files, you can still read BP3/BP4/HDF5 files with your existing code but keep in mind that your code is not going to work when moving to BP5.
adios2::ADIOS::EnterComputationBlock(), adios2::ADIOS::ExitComputationBlock() are hints to ADIOS that a process is in a computing (i.e. non-communicating) phase. BP5 asynchronous I/O operations can schedule writing during such phases to avoid interfering with the application’s own communication.
GPU-aware I/O supports passing device-memory data pointers to the Put/Get functions, and ADIOS2 will automatically download/upload data from/to the device during I/O. Alternatively, an extra member function of the Variable class, SetMemorySpace(const adios2::MemorySpace mem) can explicitly tell ADIOS2 whether the pointer points to device memory or host memory.

Notable new features

BP5

BP5 is a new data format and there is a new engine, BP5, that writes and reads in this format. See https://adios2.readthedocs.io/en/latest/engines/engines.html for runtime options. There are four main advantages using this over the previous engine BP4. - Lower memory consumption. Deferred Puts use user buffer for I/O wherever possible thus saving on a memory copy. Aggregation uses a fixed-size shared-memory segment on each compute node instead of using MPI to send data from one process to another. Memory consumption can save 50% of extra memory usage compared to BP4 in some cases. - Faster metadata management improves write/read performance where hundreds or more variables are added to the output. - Improved functionality around appending many output steps into the same file. Better performance than writing new files each step. Restart can append to an existing series by truncating unwanted steps. Readers can filter out unwanted steps to only see and process a limited set of steps. Just like as in BP4, existing steps cannot be corrupted by appending new steps. Existing steps can be read by another application while new steps are written and the reader can wait for incoming steps, i.e., in situ processing through files works identically to using staging engines. - Asynchronous writing is an option with BP5, which can further improve performance in applications. This is a simple runtime option to turn it on/off, there is no need to tinker with I/O calls in the source code. This asynchronous I/O implementation does NOT need a multithreaded MPI library.

GPU Support

IO: ADIOS can now write and read data to and from CUDA device memory by passing a valid device pointer to the appropriate Put and Get API calls. Current support is limited to CUDA on NVIDIA GPUs but future releases will support both AMD and Intel GPUs.
Compression: The ZFP floating point compression operator has additional configurations options and now supports compression and decompression operations on NVIDIA GPUs for variable data on either host or device memory.

Further details on GPU-Aware I/O can be found in the ADIOS user guide

Plugin architecture for Engines and Operators

Engines and Operators can now be developed outside of ADIOS and loaded at runtime. For engine plugins, the engine should inherit from the PluginEngineInterface class. The engine library should also contain EngineCreate() and EngineDestroy() symbols that ADIOS will be able to use to create and destroy your engine. Similarly, Operator plugins should inherit from the PluginOperatorInterface class and the library should contain OperatorCreate() and OperatorDestroy() symbols. To help ADIOS find user plugin libraries, the ADIOS2_PLUGIN_PATH environment variable can be set. Further details on writing and using plugins can be found in the ADIOS user guide.

Compression operators backward-compatibility feature

From version 2.8.0 onward Operators manage their own metadata and their future versions will provide information on the (compression) libraries used in older datasets in case they cannot decompress the data with the current compression library it is built with. Users then may be able to rebuild the future ADIOS version with an older, compatible compression library that can read back their data. The downside of the redesign of Operators is that datasets written with lossy compression cannot be read back with ADIOS 2.8.0 and newer versions, so you need to build and use 2.7.1 for that.

Full Changelog: https://github.com/ornladios/ADIOS2/compare/v2.7.1...v2.8.0

- C++
Published by chuckatkins about 4 years ago

https://github.com/ornladios/adios2 - NWChem-ADIOS2-R-Vis

This is an intermediate release for the sole purpose of having a tagged easily reproducible build for the "Interleaving R with Simulations and Visualization for streaming on Large Distributed Systems via the ADIOS2 Data Transport Framework" paper submitted to the Journal of Statistical Software

- C++
Published by chuckatkins almost 5 years ago

https://github.com/ornladios/adios2 - ADIOS v2.7.1 Production Patch Release

Summary

This is a minor patch release to address outstanding bugs that arose after the 2.7.0 release was deployed.

Bug fixes

#2582 Installation of bp2h5 wrappers fails on Windows due to symlinks
- Fixed by #2583
#2585 Update documentation for using both serial and MPI libraries from a single build.
#2018 Blosc compression doesn't support variables over 2GiB
- Fixed by #2592
#2596 Python shouldn't be required if testing and Python bindings are disabled

- C++
Published by chuckatkins over 5 years ago

https://github.com/ornladios/adios2 - ADIOS v2.7.0 Production Release

Summary

This is a major production release with a variety of new features and bug fixes in the areas of staging, HDF5 compatibility, and writer aggregation.

Features

New features

Hierarchical traversing for reading ADIOS files using the new Group object
- This is an optional, alternative way to go over the variables and attributes in a file/stream in a similar fashion as one traverses HDF5 files.
Default aggregation mode has changed to 1 sub-file per compute node (from one sub-file per process)
DataMan engine supports compression (used for wide-area-network data transfers)
SSC engine supports flexible I/O pattern
Step selection for reading
Simplified wrapper bp2h5 for the adios2_reorganize utility for converting ADIOS BP files to HDF5 files.

HDF5 Virtual Object Layer (VOL) Plugin for ADIOS

Added a VOL plugin to enable ADIOS I/O through the HDF5 API. Basic functionalities are covered.
Enables existing HDF5 clients to create and access ADIOS files with minimal effort.

API Changes

Double pointer Get API for the Inline engine
AggregatorRatio option to set number of sub-files as a ratio of number of processes. NumAggregators option is used to set a fixed number of sub-files. See options for https://adios2.readthedocs.io/en/latest/engines/engines.html#bp4

- C++
Published by chuckatkins over 5 years ago

https://github.com/ornladios/adios2 - ADIOS v2.6.0 Production Release

Summary

This is a major release with a wide variety of features in all areas including staging, file I/O, build system, and many others.

Features

Build System

When enabling MPI the build system now generates both serial and parallel libraries. One installation of ADIOS is needed only for building both parallel and serial applications
CMake targets now support components for each supported language, i.e. instead of just adios2 there are now adios2::cxx11 and adios2::cxx11_mpi
All third party dependencies can now be provided externally rather than use internal copies

Engines

BP4 Engine: Burst buffer support for writing. File on the final target file system can still be read step-by-step in situ by other apps. Appending to an existing file, however, is NOT supported.
New engine SSC: MPI-based engine for code coupling and in situ processing
DataMan engine: new reliable mode, that guarantees that every step arrives at the reader
Inline engine: is now being elevated from experimental to beta.
SST engine:
- With the non-RDMA transport, IP Port range now defaults to "any", rather than "26000-26100". (The desired range can be specified with the ADIOS2PORTRANGE environment variable, see sstconntool documentation for more information.)
- SST now has a "push mode" where written data is pro-actively sent to readers without waiting for specific requests. This "push mode" is either speculative (based on an assumption that data will be used) or learned (based on actual usage).
  - Speculative push mode is enabled automatically in certain circumstances, managed by SST engine parameters SpeculativePreloadMode and SpecAutoNodeThreshold.
  - Learned preload mode is triggered when the reader and writer indicate that their data consumption/production patterns are static by calling LockReaderSelections and LockWriterDefinitions, respectively.

API Changes (do not break compilation)

DebugMode now deprecated from the public constructor and factory interfaces. In previous releases this flag controlled enabling or disabling additional error checking. Through extensive bench-marking of API calls it was determined that the savings was negligible so the appropriate error checking (null pointer, bounds, etc.) is now always performed within API calls and throwing exceptions (in C++) or returning error codes (C and Fortran) where appropriate.
Python available_variables() now has a filter list argument: available_variables(['type']) will return faster but only provides names and types of variables. For backward compatibility, available_variables() without arguments is identical to a call with the full list available_variables(['Type', 'Min', 'Max', 'SingleValue','Shape','AvailableStepsCount']). The filter names are case-insensitive.
YAML configuration file. adios2::ADIOS constructor can now accept a configuration file using the YAML format and it must have the *.yaml or *.yml extension. See section in docs.

- C++
Published by chuckatkins about 6 years ago

https://github.com/ornladios/adios2 - ADIOS v2.5.0 Production Release

Summary

ADIOS v2.5.0, September 30th 2019 This release should be considered stable and suitable for deployment. We highly recommend migration to this version for long term support and future API compatibility. This is a major update which includes the introduction of a new directory-based file format, BP4, for the default Engine targeting streaming through files applications.

Features

Virtual Engines (File, FileStream, InSituAnalysis, InSituVisualization, CodeCoupling)
BP4 file engine has a constant append time of new steps. Faster than creating new output at every step.
Default is now the BP4 file engine that supports reading streaming concurrently while the writer is still producing new steps
Zero-copy access to the engine buffer memory in Write mode. Uses Variable::Span per Engine Put block, based on upcoming C++20 std::span. Available in BP3 (since v2.4.0) and BP4 (since v2.5.0) Engines C++ APIs only.

API Breakage

Python high-level API: ability to retrieve single global values as numpy's 0-dimension array This breaks API compatibility with v2.4.0 Python high-level API

- C++
Published by chuckatkins over 6 years ago

https://github.com/ornladios/adios2 - ADIOS v2.4.0 Production Release

Summary

ADIOS v2.4.0, June 30th 2019. This release should be considered stable and suitable for deployment. We highly recommend migration to this version for long term support and future API compatibility.

Features

Automatic conversion to fixed width types (int -> int32_t), in all bindings to facilitate portable types and bpls output
MPI library "serial" functionality. ADIOS2 library compiled with MPI can run with code not invoking MPI_Init and MPI_Finalize running in serial. Requires linking with MPI for symbols.
Stable high-level APIs in Python and C++ supporting local variables
Zero-copy access to the engine buffer memory in Write mode. Uses Variable<T>::Span per Engine Put block, based on upcoming C++20 std::span. Available in BP3 Engine C++ APIs only.
New functionality in Fortran and C APIs for safer handling of subroutines/functions outputting strings
Addition of no-op backends to disable selected functionality and / or evaluate the performance impact of various layers
- null Engine : Bypass all library actions as soon as possible, often in the high-level language bindings before the core library is even reached
- nullcore Engine: Implemented in the core of the library allowing the front-end data management and variable operations to occur but to transparently ignore all write and buffering requests.
- null Transport : For use with the BP engines, this will bypass all disk I/O operations. All of the data management and buffering will still occur but there will be no actual data written to a file.
Optimization of data exchanges in the SST engine when a fixed data exchange pattern is specified via LockWriterDefinitions/LockReaderSelections
PreciousFirstTimestep engine mode allows saving mesh and other one-time-output in a stream, so that late-comer readers can properly initialize from the first input step. Available in SST engine only.
Added support for MemorySelection for reading into non-contiguous memory (e.g. ghost cells). Available in BP3 Engine only.
Fixed several bugs: substreams aggregation, endian interoperability, local variables and bpls
Complete redesign of the user guide documentation in readthedocs
Added several new lossless compression plugins in BP buffer: bzip2, libpng, blosc. Available in BP and SST engines only.
Availability of tools interface with connection to TAU 2.28 or greater.

Changes

LatestAvailable reading mode has been removed from BeginStep() and is now specified at Open() with AlwaysProvideLatestTimestep engine parameter. This allows for more optimized queueing behavior in staging engines. This breaks API compatibility with v2.3.x
IO.LockDefinitions() function has been removed and replaced with Engine.LockWriterDefinitions() and Engine.LockReaderSelections() to improve clarity and separate functionality. As before, calling these functions informs ADIOS2 that the write/read pattern between two applications in staging is unchanging from timestep to timestep (after the calls). This allows certain optimizations, such as eliminating some metadata exchanges and pre-loading data to the readers. This breaks API compatibility with v2.3.x
Data queueing in the SST engine has been redesigned, adding more explicit support for retaining timesteps for late-arriving readers (see the ReserveQueueLimit engine parameter), but also changing the way timesteps are handled when there are no readers present. In the no-readers case, specifying QueueFullPolicy:Blocking will no longer cause SST to pause until a reader arrives. This behavior is a change from v2.3.x

- C++
Published by chuckatkins almost 7 years ago

https://github.com/ornladios/adios2 - ADIOS v2.4.0-RC1 Release Candidate

Summary

This is a release candidate for the upcoming v2.4.0 release in June 30th 2019. The v2.4.0-rc1 release should be considered stable and suitable for testing deployment as a module on HPC platforms. We recommend users to migrate to this version to test for the upcoming v2.4.0 and report any outstanding issue.

Features

Moved completely to fix width types in all bindings to facilitate portable types and bpls output
Stable high-level APIs for Python and C++ supporting local variables
Added NULL Engine to measure Engine overall overhead
Variable<T>::Span provide access to the engine buffer memory in Write mode (C++ APIs) similar to C++20 std::span of a std::vector
New functionality in Fortran and C APIs for handling strings in a safer mode
Fixed several bugs: Substreams aggregation, endian interoperability, bpls
Update documentation: adios2.readthedocs.io

- C++
Published by williamfgc about 7 years ago

https://github.com/ornladios/adios2 - ADIOS v2.3.1 Production Release

Summary

This is a bug-fix release on top of the v2.3.0 release. The v2.3.1 release should be considered stable and suitable for deployment as a module on HPC platforms.

Features

Bug fixes beyond the v2.3.0 release include: * Various CMake fixes * #1203 #1204 * Statically generate adios2-config at install time so cmake and python are no longer required * #1172 #1191 * Various bpls bug fixes * #1175 #1182 #1211 * Add missing attributes to high level API * #1146 #1197 * Various SST bug fixes * #1157 #1167 #1152 #1151 #1149 #1186 #1183 #1201 #1207 #1213 * HDF5 Single value attributes * #1159 * MGARD tweaks * #1196

- C++
Published by chuckatkins over 7 years ago

https://github.com/ornladios/adios2 - ADIOS v2.3.0 Production Release

Summary

This is the first production release of the ADIOS v2.x branch. At this point, all public APIs can be considered fairly stable. This also marks the official transition from ADIOS v1.x to v2.x, thus v1.x should now be considered deprecated.

Features

Major additions from the previous v2.2 Beta release include:

Additional testing added reaching over 300 test covering both low level unit tests and higher level mini-app use-cases.
APIs for local array handling (each rank is an independent chunk of data instead of a smaller piece in a global index space)
The dissemination of per-rank write block information so a reader can know which pieces of a variable were written by which rank and optimize their read selection accordingly.
An extensive list of bug fixes.

- C++
Published by chuckatkins over 7 years ago

https://github.com/ornladios/adios2 - ADIOS v2.2.0 Performance Beta Release

Summary

This is the fourth pre-release of the ADaptable Input Output System, ADIOS, version 2 (v2.2.0). The primary focus of this beta release is increased performance through I/O aggregation. This version is not intended for production quality purposes.

Features

Major additions from the previous v2.1.1 Beta release include process aggregation and multi-block I/O.

Aggregation

So far ADIOS 2.x wrote one output file per each writing process. This provides good performance on parallel file systems for up to the point where the file system starts to struggle to handle too many files at once. Aggregation here means that a subset of processes collect the data from all the processes and only they write files. We have implemented the aggregation algorithm known from ADIOS 1.x with some modifications to use less memory than in ADIOS 1.x. At the API level, the user only needs to set the number of “substreams” different from the number of MPI processes. Figure 1 shows an example of how 4 processes can write to a single file. This implementation saturates the MPI network bandwidth while writing to disk in an asynchronous manner. In addition, the new modular architecture of ADIOS2 allows for future experimentation of different aggregation methods to chase performance on upcoming exascale systems.

Multi-Block Writing

ADIOS 2.x now allows writing independent variable pieces to can be later retrieved as a single selection. This is better illustrated in Figure 2, in which each sub-block of data, in blue, is written independently as it becomes available. Whereas a read request can be made for a certain selection, thus pulling data for multiple sub-blocks. This pattern is largely used to support experimental and observational data in which independent I/ O. This feature is naturally integrated into the ADIOS2 API, in which each Variable Put/Get block call is inserted in a container.

- C++
Published by chuckatkins almost 8 years ago

https://github.com/ornladios/adios2 - ADIOS v2.1.1 Staging Beta Release

This is the third pre-release of the ADaptable Input Output System, ADIOS, version 2 (v2.1.1). This is a beta release with staging support and a stable API intended for users and developers to begin integration and testing of ADIOS 2 into their applications, libraries, and workflows. This version is not intended for production quality purposes.

Major additions from the previous v2.1.0 Beta release include:

SST Engine for staging data memory-to-memory across applications
Updated Fortran, C and Python bindings APIs
Several bugs corrected for default write and read BPFile engine
Added option for flexible number of substreams (subfiles) based on MPI aggregation for NxM writes (N=mpi processes, M = user defined number of files)
Added High-Level APIs to each supported language. These APIs are based on the language native IO (C++: adios2::fstream, C: adios2_FILE*, Fortran: file handler, Python: io stream object)
User Guide documentation moved to readthedocs: http://adios2-adaptable-io-system-version-2.readthedocs.io/en/latest/

| Feature | ADIOS 1.13 Support | ADIOS 2.1.1 Support | | ------------- |:-------------:|:-------------:| | Write API | | | | Local Variable | :whitecheckmark: (Offsets, LocalDimensions) | :whitecheckmark: (Start, Count) | | Global Variable | :whitecheckmark: (Global) | :whitecheckmark: (Variable) | | Joined Dimensions | :whitecheckmark: | :whitecheckmark: | | Dimension Variable | :whitecheckmark: | deprecated | | Constant Dimension Variable | :x: | :whitecheckmark: | | Selection | :whitecheckmark: | partial | | Memory Selection | :whitecheckmark: | :x: | | Attributes | :whitecheckmark: | :whitecheckmark: | | Transforms | :whitecheckmark: | :x: | | Config XML | :whitecheckmark: | :whitecheckmark: | | Threaded Operations | :x: | :whitecheckmark: | | Time Aggregation | :whitecheckmark: | :whitecheckmark: | | Collective Metadata | :whitecheckmark: | :whitecheckmark: | | BP3 Format | :whitecheckmark: | :whitecheckmark: (bitmap stats deprecated) | | BP4 Format | --- | :x: | | I/O Modes | :whitecheckmark: (Transports) | beta | | BP3 SubFile N-to-N | :whitecheckmark: (POSIX) | :whitecheckmark: | | BP3 SubFile N-to-M | :whitecheckmark: (MPIAggregate) | :whitecheckmark: initial support | | HDF5 | :whitecheckmark: | :whitecheckmark: | | WAN (DataMan) | :x: | :whitecheckmark: | | Read API | :whitecheckmark: | :whitecheckmark: | | Utils | | all compatible with ADIOS1.12 | | bpmeta | :whitecheckmark: | :x: | | bpls | :whitecheckmark: | partial bpls2 | | bpdump | :whitecheckmark: | :x: | | Language Support | | C++ | :whitecheckmark: through C | :whitecheckmark: native | | Python | :whitecheckmark: Cython | :whitecheckmark: PyBind11 | | C | :whitecheckmark: | :whitecheckmark: | | Fortran | :whitecheckmark: | :whitecheckmark: | | Other | | Automated testing | :x: | :whitecheckmark: (via CTest and GoogleTest) | | Windows support | :x: | :whitecheck_mark: (via CTest and GoogleTest) |

- C++
Published by chuckatkins about 8 years ago

https://github.com/ornladios/adios2 - ADIOS v2.1.0 Beta Release

This is the second pre-release of the ADaptable Input Output System, ADIOS, version 2 (v2.1.0). This is a beta release with a stable API intended for users and developers to begin integration and testing of ADIOS 2 into their applications, libraries, and workflows. This version is not intended for production quality purposes.

Major additions from the v2.0.0 Alpha release include:

Re-designed Read and Write APIs for uniformity between the two
Native read and write support for the ADIOS BP format
Read and write support for parallel and serial HDF5
Full support for C, Fortran, and Python APIs in addition to the native C++ interface.
Initial staging implementations with SST for HPC environments and DataMan for Wide Area Networks.

| Feature | ADIOS 1.12 Support | ADIOS 2.1.0 Support | | ------------- |:-------------:|:-------------:| | Write API | | | | Local Variable | :whitecheckmark: (Offsets, LocalDimensions) | :whitecheckmark: (Start, Count) | | Global Variable | :whitecheckmark: (Global) | :whitecheckmark: (Variable) | | Joined Dimensions | :x: | :whitecheckmark: | | Dimension Variable | :whitecheckmark: | deprecated | | Constant Dimension Variable | :x: | :whitecheckmark: | | Selection | :whitecheckmark: | partial | | Memory Selection | :whitecheckmark: | :x: | | Attributes | :whitecheckmark: | :whitecheckmark: | | Transforms | :whitecheckmark: | :x: | | Config XML | :whitecheckmark: | :whitecheckmark: | | Threaded Operations | :x: | :whitecheckmark: | | Time Aggregation | :whitecheckmark: | :whitecheckmark: | | Collective Metadata | :whitecheckmark: | :whitecheckmark: | | BP1 Format | :whitecheckmark: | :whitecheckmark: (bitmap stats deprecated) | | BP2 Format | --- | :x: | | I/O Modes | :whitecheckmark: (Transports) | partial | | BP1 File N-to-N | :whitecheckmark: (POSIX) | :whitecheckmark: | | BP1 File N-to-M | :whitecheckmark: (MPIAggregate) | :x: | | HDF5 | :whitecheckmark: | :whitecheckmark: | | WAN (DataMan) | :x: | :whitecheckmark: | | Read API | :whitecheckmark: | :whitecheckmark: | | Utils | | all compatible with ADIOS1.12 | | bpmeta | :whitecheckmark: | :x: | | bpls | :whitecheckmark: | partial bpls2 | | bpdump | :whitecheckmark: | :x: | | Language Support | | C++ | :whitecheckmark: through C | :whitecheckmark: native | | Python | :whitecheckmark: Cython | :whitecheckmark: PyBind11 | | C | :whitecheckmark: | :whitecheckmark: | | Fortran | :whitecheckmark: | :whitecheckmark: Still to do | | Other | | Automated testing | :x: | :whitecheckmark: (via CTest and GoogleTest) | | Windows support | :x: | :whitecheck_mark: (via CTest and GoogleTest) |

- C++
Published by chuckatkins over 8 years ago

https://github.com/ornladios/adios2 - ADIOS v2.0.0 Alpha Release

Initial release of the ADaptable Input Output System, ADIOS, version 2.0.0. This is an alpha release intended to be distributed so users and developers can test the preliminary list of supported features, provide feedback on the source code, documentation, API, and development flows, etc., and report bugs. This version is not intended for production quality purposes.

Please see Getting Started for instructions on how to configure and build this release, along with the User Guide.

|Summary | | |:------------- |----------------------------| | Supported 64-bit OS | Linux | | | Mac OS X El Capitan | | | Windows 10 | | Minimum Compiler Versions | gcc v4.8.1 | | | clang 3.3 | | | Visual Studio 2015 | | | Intel 16 | | Distributed 3rd Party | PyBind11 | | | pugixml | | | GoogleTest | | | KWSys | | | JSON for Modern C++ |

Feature Support

This is list of partial features support in ADIOS 2.0.0 that are included in ADIOS 1.12. The goal of this list is to provide a simple overview to those familiar with ADIOS. It also provides a roadmap of the features from ADIOS 1.12 to be implemented in the next beta release in December 2017.

| Feature | ADIOS 1.12 Support | ADIOS 2.0.0 Support | | ------------- |:-------------:|:-------------:| | Write API | | | | Local Variable | :whitecheckmark: (Offsets, LocalDimensions) | :whitecheckmark: (Start, Count) | | Global Variable | :whitecheckmark: (Global) | :whitecheckmark: (Variable) | | Joined Dimensions | :x: | :whitecheckmark: | | Dimension Variable | :whitecheckmark: | deprecated | | Constant Dimension Variable | :x: | :whitecheckmark: | | Selection | :whitecheckmark: | partial | | Memory Selection | :whitecheckmark: | :x: | | Attributes | :whitecheckmark: | :x: | | Transforms | :whitecheckmark: | :x: | | Config XML | :whitecheckmark: | :whitecheckmark: | | Threaded Operations | :x: | :whitecheckmark: | | Time Aggregation | :whitecheckmark: | :whitecheckmark: | | Collective Metadata | :whitecheckmark: | :x: (see Utils bpmeta) | | BP1 Format | :whitecheckmark: | :whitecheckmark: (bitmap stats deprecated) | | BP2 Format | --- | (experimental) | | I/O Modes | :whitecheckmark: (Transports) | partial | | BP1 File N-to-N | :whitecheckmark: (POSIX) | :whitecheckmark: | | BP1 File N-to-M | :whitecheckmark: (MPIAggregate) | :x: | | HDF5 | :whitecheckmark: | :whitecheckmark: | | WAN (DataMan) | :x: | :whitecheckmark: | | Read API | :whitecheckmark: | partial wrapper around ADIOS1 | | Utils | | all compatible with ADIOS1.12 | | bpmeta | :whitecheckmark: | :x: | | bpls | :whitecheckmark: | :x: | | bpdump | :whitecheckmark: | :x: | | Language Support | | C++ | :whitecheckmark: through C | :whitecheckmark: native | | Python | :whitecheckmark: Cython | :whitecheckmark: PyBind11 | | C | :whitecheckmark: | :x: | | Fortran | :whitecheckmark: | :x: | | Other | | Automated testing | :x: | :whitecheckmark: (via CTest and GoogleTest) | | Windows support | :x: | :whitecheck_mark: (via CTest and GoogleTest) |

- C++
Published by chuckatkins almost 9 years ago