glamod-marine-processing - v7.1.0

v7.1.0 (2024-11-25)

Contributors to this version: Ludwig Lierhammer (@ludwiglierhammer)

Announcements

• make preparations for zenodo DOI assignment (#4, #42, #54)
• adding readthedocs documentation (#42)
• Now under Apache v2.0 license (#42)
• Thanks to NOC, Cookiecutter and GNU parallel (#53, #54)
• glamod-marine-processing has migrated its development branch name from master to main.
• final GLAMOD marine processing data release 7.0 version (#63)

New features and enhancements

• obs_suite: add quality checks for both Wind speed and wind direction in level1e script (#20, #22)
• pre_proc: add pre-processing for ICOADS data (#24)
• post_proc: add post processing for C-RAID level1a data (#26)
• obs_suite: optionally, set list of decks to process (#25)
• obs_suite: optionally, set both release period init and end year (#25)
• obs_suite: running with C-RAID data (#25)
• obs_suite: optionally, set both source and destination level, release and dataset (#67)
• obs_suite: optionally, set both path to NOC correction data and NOC version in level1b (#67)
• obs_suite: optionally, set path to Pub47 data in level1d (#67)
• obs_suite and qc_suite: optionally, run jobs in parallel with gnu_parallel (#41)
• post_proc: optionally, post-processing for ICOADS data (#46)
• post_proc: optionally, merge data from additional directories (invalid, excluded) to a new deck dataset (#52)

Internal changes

• obs_suite: take data paths from already created configuration files (#67)
• obs_suite: configuration files for C-RAID (#25)
• obs_suite: adjust to cdm_reader_mapper version v0.4.0 and further versions (#21, #28)

Breaking changes

• delete metadata suite, config suite and not-used scripts/modules (#14, #16)
• obs_suite: date information is NOT mandatory in filenames anymore (#25)
• obs_suite: pass tables if no correction or quality control file are available (#25)
• obs_suite: adjust both process deck lists and processing init/end years to release7.0 requirements (#27)
• obs_suite: new ICOADS_R3.0.2T deck list after level 1a (#40)
• obs_suite: starting with year 2014 (#40)
• set BASTION do default machine (bastion01.core.ichec.ie) (#37)
• obs_suite: use duplicate checker from cdm_reader_mapper instead of NOc correction files in level1b (#37)
• obs_suite: create only one task for level2 (#45)
• obs_suite: rename Pub47 data from {year}-{month}-01.csv to pub47-{year}-{month}.csv in level1d script (#48)
• obs_suite: set release period to 2015 to 2023 (#49)
• obs_suite: if no qc files available: set report_quality from 2 (not checked) to 0 (passed) in level1e script (#50)
• obs_suite: if reportid is not available in any observations: remove reportid from header (and vice versa) in level1e script (#50)
• obs_suite: update configuration file structure of previous GLAMOD data releases (#67)
• qc_suite: no need for NOC correction files (#39)
• qc_suite: set minimum QC end year from 2022 to 1948 (#52)
• qc_suite: update job list for release 7.0 (#52)

Bug fixes

• obs_suite: fixing observation suite level1e tests (#17)
• obs_suite: QC mapping from v7.0.0 is now running by setting values of location_quality and report_time_quality to str (#18)
• obs_suite: use only observation reports that are also available in the header file (#44, #45)
• qc_suite: take qc source data from level1d files instead of level1a (#47)
• qc_suite: update deck list for quality control (#47)
• qc_suite: ignore reports with invalid date time information (#52, #58)
• obs_suite: allow mixed date time formats in level1c (#62)

- Python
Published by ludwiglierhammer about 1 year ago

glamod-marine-processing - v7.0.0

renaming release name to vX.Y.Z

- Python
Published by ludwiglierhammer over 1 year ago

glamod-marine-processing - release_7.0.0

Contributors to this version: Ludwig Lierhammer (:user:ludwiglierhammer)

Breaking changes ^^^^^^^^^^^^^^^^ * delete emtpy and not used files, functions and folders (:pull:3) * create requirements for each suite (:pull:3) * rebuild to a installable python package (:pull:3) * install package and requirements via a pyproject.toml file (:pul::3) * change QC mapping in obs_suite level1e (:issue:7, :pull:8):

• if location_quality is equal 2 set both report_quality and quality_flag to 1
• if `report_time_quality is equal 4 or 5 set both report_quality and quality_flag to 1

New features and enhancements ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ * add some information files: AUTHORS.rst, CHANGES.rst, CONTRIBUTING.rst and LICENSE (:pull:3) * make us of pre-commit (:pull:3) * make use of an command-line interface to create suite PYTHON and SLURM scripts (:pull:3, :pull:5) * add new release 7.0 configuration files (:pull:3) * set some default directories and SLURM settings for both HPC systems KAY and MeluXina (:pull:3)

Internal changes ^^^^^^^^^^^^^^^^ * reduce complexity of some functions (:pull:3) * adding observational testing suite (:issue:5, :pull:5) * load data from cdm-testdata (:pull:11)

- Python
Published by ludwiglierhammer over 1 year ago

glamod-marine-processing - Release 6

Release 6 of the GLAMOD marine. Consistent with Marine User Guide v8

- Python
Published by andreas-wernecke over 2 years ago