Releases | Open Source Science

https://github.com/aliceo2group/datadistribution - v1.6.3

- C++
Published by lkrcal over 2 years ago

https://github.com/aliceo2group/datadistribution - v1.6.2

- C++
Published by lkrcal over 2 years ago

https://github.com/aliceo2group/datadistribution - v1.6.1

- C++
Published by lkrcal almost 3 years ago

https://github.com/aliceo2group/datadistribution - v1.6.0

What's Changed

Several updates to DD by @davidrohr in https://github.com/AliceO2Group/DataDistribution/pull/175

Changes include: - Terminate TfBuilder in case of gRPC failure, since that EPN will be silently dead otherwise. - Terminate TfBuilder if it cannot connect to one of the FLPs, otherwise we can loose an FLP in a run. - Adapt to new FairMQ API. - Disable deprecation warnings to enable compilation with new boost.

Full Changelog: https://github.com/AliceO2Group/DataDistribution/compare/v1.6.0...v1.6.0-fix.1

- C++
Published by cuveland about 3 years ago

https://github.com/aliceo2group/datadistribution - v1.5.5

What's Changed

Set InfoLogger message levels in addition to severity https://github.com/AliceO2Group/DataDistribution/pull/174

- C++
Published by lkrcal about 3 years ago

https://github.com/aliceo2group/datadistribution - v1.5.4

What's Changed

Add RDHv7 to the list of known versions https://github.com/AliceO2Group/DataDistribution/pull/173

- C++
Published by lkrcal about 3 years ago

https://github.com/aliceo2group/datadistribution - v1.5.3

What's Changed

Fix warning causing a compilation error when using GCC 12 by @ktf in https://github.com/AliceO2Group/DataDistribution/pull/171
GCC 12 compatibility by @cuveland in https://github.com/AliceO2Group/DataDistribution/pull/172

Full Changelog: https://github.com/AliceO2Group/DataDistribution/compare/v1.5.2...v1.5.3

- C++
Published by cuveland over 3 years ago

https://github.com/aliceo2group/datadistribution - v1.5.2

What's Changed

Adapt to libfmt 9.1.0 by @davidrohr in https://github.com/AliceO2Group/DataDistribution/pull/170

Full Changelog: https://github.com/AliceO2Group/DataDistribution/compare/v1.5.1...v1.5.2

- C++
Published by cuveland over 3 years ago

https://github.com/aliceo2group/datadistribution - v1.5.1

ucx: check for completion before going to sleep
[OMON-593] Do not generate PM metrics requiring reading smaps file
Remove go_package line from AliECS grpc interface

- C++
Published by ironMann almost 4 years ago

https://github.com/aliceo2group/datadistribution - v1.5.0

Improved cancellation of Initializing states, and exit on control system signals before reaching READY states.
Option to drop a percentage of TimeFrames on FLPs, before building them to EPNs, i.e., not sending them over network
Consolidated copyright notices
Comes with bump of ucx to v1.13.1 (from v1.12.1)

NOTE: Synchronized upgrade on EPN and FLP is recommended.

- C++
Published by ironMann almost 4 years ago

https://github.com/aliceo2group/datadistribution - v1.4.12

stfsender: optimize stf-copy and improve handling of memory pressure
tfscheduler: implement tf scheduling throttling based on setting from AliECS (R3C-848)

- C++
Published by ironMann almost 4 years ago

https://github.com/aliceo2group/datadistribution - v1.4.11

stfsender: allocate a buffer and copy stfs before sending (optional)
stfsender: enable copying in standalone mode
adopt to FairMQ API changes
tfscheduler: optimize incomplete tf bookkeeping
stfbuilder: decrease size of o2 header region
ucx: fail from ucp_wait on error
tfbuilder: monitor token success ratio
tfscheduler: implement exclusive and shared tokens
stfsender: fix termination of flp-only run
tfscheduler: configurable number of tokens per StfSender
tfbuilder: remove not needed monitoring metric

- C++
Published by ironMann almost 4 years ago

https://github.com/aliceo2group/datadistribution - v1.4.10

tfscheduler: improve flp token selection bias

- C++
Published by ironMann almost 4 years ago

https://github.com/aliceo2group/datadistribution - v1.4.9

file sink: accept fractional percentage for raw tf acceptance parameter
sched: fix reporting of token loop metric

- C++
Published by ironMann almost 4 years ago

https://github.com/aliceo2group/datadistribution - v1.4.8

Raw TF metadata: adopt the same run_type as CTFs (raw,calibration,other)
monitoring: align metric time to a larger interval step (influxdb performance)
monitoring: enable process metric monitoring. Via --monitoring-process-interval parameter

- C++
Published by ironMann almost 4 years ago

https://github.com/aliceo2group/datadistribution - v1.4.7

tfscheduler: fix process lingering after termination request
monitoring: make it easier to spot non-synchronized and stalled data flow on flps
stfsender: start the grpc earlier in the configure procedure
optional: token requests for stf transfers; improves transfers rates

- C++
Published by ironMann almost 4 years ago

https://github.com/aliceo2group/datadistribution - v1.4.6

tfbuilder: implement timeouts and retries for StfDataRequest grpcs (EPN-49)

- C++
Published by cuveland almost 4 years ago

https://github.com/aliceo2group/datadistribution - v1.4.5

file sink: adopt lhc period from AliECS for EOS metadata
tfbuilder: on stop, push all TFs, then End-of-Stream
tfbuilder: fix delay during terminating
tfbuilder: improve ucx throughput for small stfs (optional)
monitoring: consolidate and remove unused metrics
end of stream: add current time in DataProcessingHeader
monitoring: don’t publish size in rate metric if no datapoints in interval

- C++
Published by ironMann about 4 years ago

https://github.com/aliceo2group/datadistribution - v1.4.4

tfbuilder: fix broken merge.stffetchms metric
grpc: verify partition id on terminate requests (EPN-130)
stfsender: publish number of rejected (late) stfs
tfbuilder: fix first orbit and run number in DD-info message when reading raw files

- C++
Published by ironMann about 4 years ago

https://github.com/aliceo2group/datadistribution - v1.4.3

stfsender: enforce processing of complete meta messages, and warn on broken data model on flp
allow transmission of null data payloads (FIT compression workflows)
monitoring: publish duration of FLP-workflow-qc processing
tfbuilder: prevent network congestion with multi-ucx_worker option

- C++
Published by ironMann about 4 years ago

https://github.com/aliceo2group/datadistribution - v1.4.2

stfsender: fix deadlock when FLP-standalone memory pressure test is active
tfscheduler: publish TF rates of ITS threshold scans
tfbuilder: parallelize stf request grpc calls
tfbuilder: add blocking mode for rdma (optional, saves 0.5 CPU core)
data-model: make new hdr scheme default (O2-2814)

- C++
Published by ironMann about 4 years ago

https://github.com/aliceo2group/datadistribution - v1.4.1

tfscheduler: add run nunber to infologger
monitor: prevent rate logging without run number
tfscheduler: log fix TfBuilder UCX Disconnection error: gRPC error when connecting StfSender
monitoring: prevent numeric max and mins
tfscheduler: do not count dropped late stfs into the dropped TFs
tfbuilder: optimize RDMA loop: move key to pre, and monitoring to post

- C++
Published by ironMann about 4 years ago

https://github.com/aliceo2group/datadistribution - v1.4.0

Improve STF meta notification by using ucx active messages
Optimize RMA worker/endpoint utilization. TfBuilder needs only a small number of workers to optimally perform rma-get.
Improvements in connection monitoring and reporting
TfScheduler: fix typo causing delayed incomplete TF building (bugfix)

- C++
Published by ironMann about 4 years ago

https://github.com/aliceo2group/datadistribution - v1.3.13

grpc: init grpc early to allow loading of control plugins with grpc
ucx: transport performance optimization of RDMA-read scheduling

- C++
Published by ironMann about 4 years ago

https://github.com/aliceo2group/datadistribution - v1.3.12

tfscheduler: fix under-reporting of incomplete tfs
tfscheduler: report global accept/rejects rates and sizes
tfscheduler: improve scheduling of incomplete tfs on completion and timeout
tfbuilder: handle the case when all stfs are not available any longer or grpc requests failed
consul: decrease connection timeout for retries to 5 seconds

- C++
Published by ironMann about 4 years ago

https://github.com/aliceo2group/datadistribution - v1.3.11

grpc: speed up connection establishing
tfscheduler: stop propagating tfs with no detector data to epn
tfscheduler: monitor rpc duration (optional)
stfsender: remove unclaimed stfs after a timeout (EPN-49)
tfbuilder: prevent activating TfBuilder if grpc::StfDataRequest() cannot be made in InitTask() (EPN-49)

- C++
Published by ironMann about 4 years ago

https://github.com/aliceo2group/datadistribution - v1.3.10

fix for compilation with CS8 GCC (FLP)

- C++
Published by ironMann about 4 years ago

https://github.com/aliceo2group/datadistribution - v1.3.9

DataDistribution v1.3.9:

FIX: tfbuilder rpc: revert deadline for grpc:StfDataRequest()
tfbuilder: add stf selection method options (consul): random (default), linear, stfsize.
monitoring: add option to monitor grpc call duration (disabled by default)
tfscheduler: use thread pool for async drop of stfs
tfbuilder: monitor total time for all grpc StfDataRequest calls per TF
tfbuilder: optimize stf in flight accounting
tfbuilder: decrease min number of ucx threads to 4

- C++
Published by ironMann about 4 years ago