Recent Releases of wfexs-backend
wfexs-backend - 1.0.1
What's Changed
- 193 tag version 100 by @jmfernandez in https://github.com/inab/WfExS-backend/pull/194
- Replaced aioftp by ftputil by @jmfernandez in https://github.com/inab/WfExS-backend/pull/198
- Bump docker/build-push-action from 6.16.0 to 6.17.0 in the github-actions group by @dependabot in https://github.com/inab/WfExS-backend/pull/199
- Release of 1.0.1 by @jmfernandez in https://github.com/inab/WfExS-backend/pull/200
- Updated constraints (triggered on 2025-05-19T08:32:43+00:00 by b715fe7369e0d307d621bdc9a377a9d7215a969e) by @github-actions in https://github.com/inab/WfExS-backend/pull/202
- Fixed issue in Dockerfile recipe intermediate state building by @jmfernandez in https://github.com/inab/WfExS-backend/pull/203
- Stable 1.0.x by @jmfernandez in https://github.com/inab/WfExS-backend/pull/204
Full Changelog: https://github.com/inab/WfExS-backend/compare/1.0.0...1.0.1
- Python
Published by jmfernandez about 1 year ago
wfexs-backend - Release 1.0.0 (Labor Day release)
What's Changed (automatically generated)
- Fixed several path handling issues in several parts of RO-Crate gener… by @jmfernandez in https://github.com/inab/WfExS-backend/pull/138
- 140 apptainer installer scripts do not work any more by @jmfernandez in https://github.com/inab/WfExS-backend/pull/141
- First test integrating 143 integrate pip audit in the ci workflows by @jmfernandez in https://github.com/inab/WfExS-backend/pull/144
- 143 integrate pip audit in the ci workflows by @jmfernandez in https://github.com/inab/WfExS-backend/pull/146
- Updated constraints due security reasons (triggered on 2025-01-20T12:09:38+00:00 by 0df685cdf1772423c3b6e2e93dce22d81fcbbd60) by @github-actions in https://github.com/inab/WfExS-backend/pull/147
- Workflow discovery code has been revamped by @jmfernandez in https://github.com/inab/WfExS-backend/pull/149
- Updated constraints (triggered on 2025-02-18T03:52:57+00:00 by c8f794a66cd6bdc737561f7c4f41e6c72a7656dd) by @github-actions in https://github.com/inab/WfExS-backend/pull/150
- Added uncommited pending changes which adjust max duration time per t… by @jmfernandez in https://github.com/inab/WfExS-backend/pull/153
- Updated constraints due security reasons (triggered on 2025-03-03T12:10:10+00:00 by 8847e21a40391b18b67b2308e0c7396f31e7f61c) by @github-actions in https://github.com/inab/WfExS-backend/pull/151
- Updated constraints (triggered on 2025-03-03T11:13:57+00:00 by 8847e21a40391b18b67b2308e0c7396f31e7f61c) by @github-actions in https://github.com/inab/WfExS-backend/pull/154
- 155 add support for null params by @jmfernandez in https://github.com/inab/WfExS-backend/pull/156
- Updated constraints (triggered on 2025-03-05T02:57:45+00:00 by dd0946fa0bc8b8941301f8a0a67c27813de7c6c4) by @github-actions in https://github.com/inab/WfExS-backend/pull/157
- Fixed issue #158, where computing the digestion of a serialized JSON … by @jmfernandez in https://github.com/inab/WfExS-backend/pull/159
- 160 add official support and tests for pypy by @jmfernandez in https://github.com/inab/WfExS-backend/pull/161
- Updated constraints (triggered on 2025-03-09T03:52:47+00:00 by 23309f22b6da0e02ad3b9d27f7e97ba429447abf) by @github-actions in https://github.com/inab/WfExS-backend/pull/162
- 160 add official support and tests for pypy (additional improvements) by @jmfernandez in https://github.com/inab/WfExS-backend/pull/163
- Fixed heuristics determining type of container tags by @jmfernandez in https://github.com/inab/WfExS-backend/pull/166
- The mypy caching mechanism used in GitHub Actions was not working for… by @jmfernandez in https://github.com/inab/WfExS-backend/pull/167
- 160 add official support and tests for pypy (improve CI) by @jmfernandez in https://github.com/inab/WfExS-backend/pull/168
- Updated constraints (triggered on 2025-03-11T15:35:15+00:00 by 41ac5b817d324fd605fee9dadbd6c2cce777507b) by @github-actions in https://github.com/inab/WfExS-backend/pull/169
- 170 explicitly handle currently implicit fetcher redirects at the cache level by @jmfernandez in https://github.com/inab/WfExS-backend/pull/171
- Fixed issue #173, detecting the minimal version is fulfilled by @jmfernandez in https://github.com/inab/WfExS-backend/pull/174
- Implemented support for Nextflow workflow inspection feature requeste… by @jmfernandez in https://github.com/inab/WfExS-backend/pull/176
- 177 support usage of files and directories from mirrored archives and datasets by @jmfernandez in https://github.com/inab/WfExS-backend/pull/179
- Implemented proof of concept which detects parameters with default va… by @jmfernandez in https://github.com/inab/WfExS-backend/pull/182
- 183 fix pip audit issues while installing pypy preconditions by @jmfernandez in https://github.com/inab/WfExS-backend/pull/184
- 183 fix pip audit issues while installing pypy preconditions by @jmfernandez in https://github.com/inab/WfExS-backend/pull/186
- Bump the github-actions group with 3 updates by @dependabot in https://github.com/inab/WfExS-backend/pull/185
- 188 set log level only for wfexs related modules by @jmfernandez in https://github.com/inab/WfExS-backend/pull/189
- Updated constraints (triggered on 2025-04-10T23:16:31+00:00 by 21c91849be95ae6baea02b5489cce1f9ccbbbcef) by @github-actions in https://github.com/inab/WfExS-backend/pull/190
- 172 add a more complete example of sarek workflow by @paulaidt in https://github.com/inab/WfExS-backend/pull/191
- Updated constraints due security reasons (triggered on 2025-04-28T12:10:36+00:00 by 76a28d90ca5d8ca8e0506e74d76bfa8418b44620) by @github-actions in https://github.com/inab/WfExS-backend/pull/178
- Bump docker/build-push-action from 6.15.0 to 6.16.0 in the github-actions group by @dependabot in https://github.com/inab/WfExS-backend/pull/192
New Contributors
- @dependabot made their first contribution in https://github.com/inab/WfExS-backend/pull/185
Full Changelog: https://github.com/inab/WfExS-backend/compare/1.0.0rc2...1.0.0
- Python
Published by jmfernandez about 1 year ago
wfexs-backend - 1.0.0rc0
This is the first release candidate release, and the first one having an official docker image, available at https://github.com/inab/WfExS-backend/pkgs/container/wfexs-backend .
What's Changed
- Docker and Singularity container recipes are considered mature enough.
- Added GitHub CI workflow to automatically generate a docker image on each new tag.
- Added instructions about how to used WfExS-backend in different containerized scenarios (see https://github.com/inab/WfExS-backend/blob/1.0.0rc0/WFEXS-in-CONTAINERS.md).
- Added "disclosable", "cacheable" and "clonable" flags to parameters from the stage definition. These definitions should tell the orchestrator whether the datasets associated to those input parameters contents can be transferred to generated RO-Crates or exported, whether a copy can be kept in the common cache, or whether a copy can be placed in the working directory.
- Now it is allowed to label a dataset cache entry as "clonable" or non "clonable", so it restricts the copies of its contents.
- More publications/posters for the README by @stain in https://github.com/inab/WfExS-backend/pull/112
- Several documentation and small fixes.
Full Changelog: https://github.com/inab/WfExS-backend/compare/1.0.0b0...1.0.0rc0
- Python
Published by jmfernandez almost 2 years ago
wfexs-backend - 1.0.0b0
What's Changed
- Added metadata file locking mechanism at the working directory level, so it is possible to check and update metadata associated to the different stages in an ordered way.
- Added capability to execute workflows in background, reporting the different rough states of the execution: queued, running, finished or died.
- Added capability to detect workflow executions which have died.
- Added a mechanism to override the maximum step processing time declared in a Nextflow workflow. Some real life workflows (like the ones from nf-core) have in place these limits, which can kill workflow executions because it was taking too long.
- Added several core tests to check both basic bootstrapping and workflow instantiation are working.
- Added core test to check parallel workflow executions do not corrupt metadata.
- Many minor fixes.
Full Changelog: https://github.com/inab/WfExS-backend/compare/1.0.0a2...1.0.0b0
- Python
Published by jmfernandez almost 2 years ago
wfexs-backend - 1.0.0a2
What's Changed
- Major internal changes in order to switch from bare path handling using
os.pathtopathlib.Path. Fetcher interfaces now support either bare path handling or os.PathLike instances. - Major internal changes in order to simplify the parameters being passed to workflow engines and container factories.
- Both representation of nextflow profiles and "content with metadata" inputs have been properly integrated into generation and consumption of Workflow Run RO-Crates.
- Fixed several metadata corruption corner case, which were leading to lose the stage state and overwriting previous execution states.
- Working directories should now be relocatable now, in the sense that embedded paths in their status metadata are now represented as relative routes.
- Avoided issue in cwltool which arises when WfExS CWL engine tries generating the graphical representation of the workflow, just forcing pydot 2.0 in newly created cwltool virtual environments.
- Added more robust support for cwltool versions, searching for the nearest compatible one to the requested one (newer versions of cwltool do not support python 3.7).
- Added the capability to use custom python interpreters for cwltool, so WfExS CWL engine can use a different one for cwltool from the used for WfExS core.
- Added initial docker and singularity recipes to build a container of WfExS-backend, along with all its dependencies (and embedded copies of docker, apptainer, gocryptfs and encfs clients).
- Many other improvements and fixes.
Full Changelog: https://github.com/inab/WfExS-backend/compare/1.0.0a0...1.0.0a2
- Python
Published by jmfernandez almost 2 years ago
wfexs-backend - 1.0.0a0
What's Changed
Full Changelog: https://github.com/inab/WfExS-backend/compare/0.99.0...1.0.0a0
- Python
Published by jmfernandez about 2 years ago
wfexs-backend - v0.99.0
What's Changed
- Reimplemented the export APIs and flow, and added deposition capabilities to sites like Zenodo, B2SHARE, Dataverse, etc...
- Added test battery for all the implemented export plugins.
- Both fetch and export plugins are now discovered, which opens the door to install them separately from WfExS as a side dependency.
- Lots of fixes.
- Version bump to v0.99.0.
Full Changelog: https://github.com/inab/WfExS-backend/compare/0.10.1...0.99.0
- Python
Published by jmfernandez about 2 years ago
wfexs-backend - 0.10.1
The main milestones achieved are the generation of compliant Workflow Run RO-Crates, and the rework of how containers are materialized and cached on stage and presented on execution.
There have been also lots of fixes for different corner cases.
Full Changelog: https://github.com/inab/WfExS-backend/compare/0.9.3...0.10.1
- Python
Published by jmfernandez over 2 years ago
wfexs-backend - Almost there...
This release has support for complex workflows as well as the generation of RO-Crate metadata containing provenance (either prospective or retrospective)
- Python
Published by jmfernandez over 3 years ago
wfexs-backend - Changed indirect set of dependencies pulled by jsonschema
Removed indirect GPLv3+ dependency which was fetched as an optional jsonschema dependency.
Also, updated the version tag and copyright year in headers.
- Python
Published by jmfernandez about 4 years ago
wfexs-backend - Reimplemented dependency on nextcloud export feature
Due project license collisions on nextcloud-api-wrapper library used by nextcloud export pusher (WfExS-backend is Apache 2.0, nextcloud-api-wrapper is GPLv3+), that export pusher has been reimplemented around pyncclient-ext (MIT license), written by @jmfernandez to provide a subclass of pyncclient (MIT license) with the needed features.
- Python
Published by jmfernandez about 4 years ago
wfexs-backend - Properly updated CITATION.cff version
Previous release did not have properly synchronized CITATION.cff with the repo itself. So, at the code level there is no change.
- Python
Published by jmfernandez about 4 years ago
wfexs-backend - First release with DOI
This WfExS-backend tagged version is the first one with an associated DOI.
It is also the first release which supports exporting results outside the staged working directory.
- Python
Published by jmfernandez about 4 years ago
wfexs-backend - v0.4.9-beta
Since the last official release there have been several improvements. Main ones are:
- Support for Docker and Podman both on cwltool and Nextflow.
- Triaged issue related to the use of crypt4gh private keys which were encrypted using scrypt, when they are used in a Python instance which does not support it.
- Added JSON Schemas to validate both configuration, staging and credential files.
- First steps towards having described and stored the full provenance of workflow execution.
- More configuration scenarios now support secure staging directories.
- Added support to several protocols and CURIEs, like FILE, FTP, SFTP, PRIDE and TRS.
- Almost all scenarios involving workflows in git repositories are now supported, as well as several versions of TRS protocol.
- Tons of bugfixes and corner cases.
- Python
Published by jmfernandez over 4 years ago