Releases | Open Source Science

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a68

[integration]

*ConfigurationSystem

CHANGE: (#8295) Move getDIRACPlatform to DIRACCommon

- Python
Published by github-actions[bot] 6 months ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a67

[integration]

*DataManagementSystem

NEW: (#8293) Add composite indices on (lastUpdate, assignment) for both Jobs and Operations tables in FTS3DB

*Test

CHANGE: (#8292) Speed up CI/pytest runtime

*Core

CHANGE: (#8290) Move TimeUtilities to DIRACCommon CHANGE: (#8290) Move StateMachine to DIRACCommon CHANGE: (#8290) Move JDL utilities to DIRACCommon CHANGE: (#8290) Move List utilities to DIRACCommon CHANGE: (#8289) Move ClassAdLight to DIRACCommon

*WorkloadManagement

CHANGE: (#8290) Move JobStatus to DIRACCommon CHANGE: (#8290) Move JobsStateMachine to DIRACCommon CHANGE: (#8290) Move JobStatusUtility to DIRACCommon CHANGE: (#8290) Move JobManifest to DIRACCommon CHANGE: (#8290) Move JobDBUtils to DIRACCommon FIX: (#8290) Correctly update JDL defaults when CS changes

*Workload Management

CHANGE: (#8289) Move JobDBUtilsto DIRACCommon CHANGE: (#8289) Move ParametricJob to DIRACCommon

*Configuration/Core

NEW: (#8288) Included a VO (using top level groups as VO) option when getting user subjects from IAM. Behaviour with single VO IAM should be unchanged.

Address GHSA-mm38-q5fj-99j4

*RucioFileCatalogClient

FIX Remove some potentially Sensitive Information (DN, account)

- Python
Published by github-actions[bot] 6 months ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a66

- Python
Published by github-actions[bot] 6 months ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a65

[integration]

*Core

NEW: (#8287) DIRACCommon package containing stateless utilities extracted from DIRAC. This allows external projects like DiracX to use DIRAC's SOK/SERROR return values and error codes without importing DIRAC and triggering global state initialization. The package maintains full backward compatibility - all existing imports continue to work unchanged.

*ConfigurationSystem

CHANGE: (#8284) force successful sync of remote CS for agents

*WorkloadManagementSystem

FIX: (#8283) Do not import DB at the top of the utility module

Please follow the template:

*Integration Tests

NEW: (#8281) Add diracx logs into DIRAC CI

- Python
Published by github-actions[bot] 6 months ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a64

[integration]

*WorkloadManagement

FIX: (#8282) Use of closed file in ZstdCompatibleTarFile when downloading sandboxes NEW: (#8282) add sandbox download functionality in Dirac runLocal API

- Python
Published by github-actions[bot] 7 months ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a63

[integration]

*WorkloadManagementSystem

NEW: (#8276) Add zstandard compression support for sandbox files with backward compatibility CHANGE: (#8244) move the logic for jobs deleting and killing in an utility

*Integration Tests

FIX: (#8248) Add in CI and in integration_test.py legacy adaptors.

- Python
Published by github-actions[bot] 7 months ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a62

[integration]

*ConfigurationSystem

FIX: (#8259) Stops VOMS2CSSynchronizer crashing on robot DNs that don't follow the CERN pattern. FEAT: (#8242) Cache getUsernameForDN and getVOForGroup

*WorkloadManagementSystem

CHANGE: (#8254) JobMonitoring: removed several unused RPC calls CHANGE: (#8252) removed SandboxStore RPC call unassignEntities CHANGE: (#8252) removed SandboxStore RPC call assignSandboxesToEntities CHANGE: (#8252) removed SandboxStore RPC calls getFreeDiskSpace and getTotalSpace CHANGE: (#8240) add site metadata for WMSHistory (Monitoring)

*ResourceStatusSystem

CHANGE: (#8250) Added a token expiry option to dirac-rss-set-status and dirac-admin-allow/ban-site commands.

*Resources

CHANGE: (#8247) add spool option to SSHCE+Condor FIX: (#8241) missing parenthesis in htcondor submission script

*Core

FIX: (#8245) dirac-apptainer-exec can bind optional paths

*TransformationSystem

NEW: (#8242) Cleanup unused TransformationAgent caches at the beginning of a cycle FEAT: (#8242) cache isSameSEInList

- Python
Published by github-actions[bot] 8 months ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.76

[rel-v8r0]

*ConfigurationSystem

FIX: (#8255) Stops VOMS2CSSynchronizer crashing on robot DNs that don't follow the CERN pattern.

*Resources

CHANGE: (#8214) SLURM plugin now supports the WholeNode options

*ResourceStatusSystem

CHANGE: (#8213) Added a token expiry option to dirac-rss-set-status and dirac-admin-allow/ban-site commands.

- Python
Published by github-actions[bot] 8 months ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a61

[integration]

*WorkloadManagementSystem

FIX: (#8237) The JobWrapper correctly resolves the destination SE lists

*Core

FIX: (#8235) do not randomized urls sorted by preferredURLPatterns

*WorkloadManagement

NEW: (#8234) add a warning log when the first pilot retrieval attempt fails

*Resources

NEW: (#8230) StorageBase.getParameters returns all parameters FIX: (#8225) Condor command line call compatible with v24

*Accounting

CHANGE: (#8216) removed few dangerous RPC calls

*MonitoringSystem

CHANGE: (#8216) removed useless RPC calls

- Python
Published by github-actions[bot] 8 months ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a60

[integration]

*DataManagementSystem

FIX: (#8228) don't remove twice in the RemoveFile operation

*WorkloadManagement

FIX: (#8227) Download input data in the job directory

*FrameworkSystem

FIX: (#8222) do not import MySQLdb unless necessary

- Python
Published by github-actions[bot] 8 months ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a59

[integration]

*Resources

CHANGE: (#8219) SLURM plugin now supports the WholeNode options CHANGE: (#8211) remove dependency from HTCondorComputingElement to PilotManager

*WorkloadManagementSystem

FIX: (#8217) move DB import in the function NEW: (#8215) define setJobStatus in the Client to allow keyword arguments

*WorkloadManagement

FIX: (#8212) AREXCE not detecting pilots in an accepted/staged states

*Core

FIX: (#8207) dirac-apptainer-exec : cd in /mnt to find locally-mounted files (e.g. pilot.cfg)

- Python
Published by github-actions[bot] 9 months ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.75

[rel-v8r0]

*FrameworkSystem

FIX: (#8210) adapt AuthServer to latest version of authlib

*WorkloadManagement

FIX: (#8208) AREXCE not detecting pilots in an accepted/staged states

*WorkloadManagementSystem

FIX: (#8205) added deprecation notice for PilotManagerHandler.killPilot

- Python
Published by github-actions[bot] 9 months ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a58

[integration]

*Resources

NEW: (#8206) allow to mount local folder in rw mode in SingularityCE

*PilotManagerHandler

CHANGE: (#8203) Moved killpilot to diracAdmin itself to avoid migrating it to DiracX while migrating PilotManagerHandler This is useful later because in DiracX we can't use it for now: it needs a proxy to contact the CS + we would need to import MySQLdb in DiracX (see error below): from DIRAC.Core.Utilities.MySQL import MySQL File "/opt/conda/lib/python3.11/site-packages/DIRAC/Core/Utilities/MySQL.py", line 155, in <module> import MySQLdb ModuleNotFoundError: No module named 'MySQLdb' This error occurred while importing killPilotsInQueues from DiracX via an import.

*ConfigurationSystem

NEW: (#8202) Add PreferredURLPattern for URL sorting

- Python
Published by github-actions[bot] 9 months ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a57

[integration]

*WorkloadManagement

FIX: (#8200) Remove accidental dependnecies in JobStateUtility

- Python
Published by github-actions[bot] 9 months ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a56

[integration]

*Resources

FIX: (#8197) SE: check the protocol only after checking the plugins when checking the occupancy

*Core

FIX: (#8193) consistent use of utcfromtimestamp in the Graph FIX: (#8187) Platform detection on recent EL9 broken

*WorkloadManagementSystem

FIX: (#8192) getJobParameters for the multi-VO case FIX: (#8192) stalledJobAgents talks directly to DBs

*MonitoringSystem

NEW: (#8190) WebAppHandler and Client, for WMS and TS related DIRAC WebApp calls

*WorkloadManagement

FIX: (#8189) Don't clean sandboxes which are on S3 FIX: (#8184) PushJobAgent dumps the proxy to get access to DiracX token NEW: (#8183) Support bulk calls to JobMonitoring.getInputData

*Tests

CHANGE: (#8188) use a jwks instead of a single key

- Python
Published by github-actions[bot] 9 months ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.74

[rel-v8r0]

*Core

FIX: (#8185) Platform detection on recent EL9 broken

*Subsystem

FIX: (#8175) SiteDirector: the architecture script can be set in the options

*WorkloadManagement

FIX: (#8168) JobAgent endlessly looping in checkSubmittedJobs

- Python
Published by github-actions[bot] 10 months ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a55

[integration]

- Python
Published by github-actions[bot] 10 months ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a54

[integration]

*WorkloadManagement

FIX: (#8180) disable watchdgo wallclock check for remote executions FIX: (#8179) Watchdog does not kill payload properly and runs endlessly FIX: (#8170) JobAgent endlessly looping in checkSubmittedJobs FIX: (#8163) Typo in SingularityComputingElement

*Subsystem

FIX: (#8177) SiteDirector: the architecture script can be set in the options

*TransformationSystem

FIX: (#8172) adapt getTableSummaryWeb to the new data access layer response format NEW: (#8165) Cast return of getTransformationFilesCount in the client FIX: (#8162) Explicitly cast to string when returning dates

*WorklodManagement

FIX: (#8171) Optimize SandboxMetadataDB.unassignEntities FIX: (#8171) Optimize JobLoggingDB.deleteJob FIX: (#8171) Optimize JobDB.getJobsAttributes FIX: (#8171) Optimize PilotAgentsDB.__getPilotID

*Resources

CHANGE: (#8153) remove PilotBundle

*GraphUtilities

FIX: (#8151) prevent any local-UTC offset being introduced in plots

- Python
Published by github-actions[bot] 10 months ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.73

[rel-v8r0]

*WorkloadManagement

FIX: (#8161) Typo in SingularityComputingElement

- Python
Published by github-actions[bot] 11 months ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a53

[integration]

*WorkloadManagement

NEW: (#8159) cgroup2 limit support

*FrameworkSystem

CHANGE: (#8157) Improve performance of TheImpersonator FIX: (#8154) Caching the proxy strength to avoid a DB call NEW: (#8144) add a randomized connection pooling for diracx

*WorkloadManagementSystem

FIX: (#8156) we can kill a list of pilots instead of going one by one

*Core

NEW: (#8155) Add caches to asn1_utils for better performance

*RequestManagementSystem

FIX: (#8152) Printing DiracX ForwardDISET requests

- Python
Published by github-actions[bot] 11 months ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.72

[rel-v8r0]

*Resources

FIX: (#8118) do not try to use a malformed StorageElement instance in SingularityCE FIX: (#8114) Fix AREX CE pilot logs in alwaysIncludeProxy case

*WorkloadManagement

FIX: (#8118) report the message of the Exception instead of the Exception itself in JobAgent.submitJob NEW: (#8110) cgroup2 limit support

*WorkloadManagementSystem

FIX: (#8113) avoid repeating optimization when job goes to STAGING

*WorkloadManagemnt

FIX: (#8103) Clear any non-UTF encodable environment variables in pilots

*ConfigurationSystem

NEW: (#8093) backport the export of the Sub in DiracX

*Core

FIX: (#8091) Don't use string processing on X509 name objects

- Python
Published by github-actions[bot] 11 months ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a52

[integration]

*RequestManagementSystem

FIX: (#8150) RequestValidator sets correct Owner for v8 requests

- Python
Published by github-actions[bot] 11 months ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a51

[integration]

*FrameworkSystem

FIX: (#8149) DiracX tokens should not be included in the proxies used to interact with CEs FIX: (#8146) TypeError in TheImpersonator

*TransformationSystem

FIX: (#8147) bad escape in the updateTransformationParameter

*WorkloadManagement

FIX: (#8145) Running dirac-admin-update-pilot

- Python
Published by github-actions[bot] 11 months ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a50

[integration]

*Core

FIX: (#8140) Make it possible to debug dirac-jobexec failures NEW: (#8139) add a source parameter to the impersonator NEW: (#8138) DiracX token from PEM is always stored in $TMP/dx_.....

- Python
Published by github-actions[bot] 11 months ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a49

[integration]

*Core

FIX: (#8136) Proxy Pilots are sent with a token

*WorkloadManagementSystem

FIX: (#8135) the PoolXMLSlice should be created in the same directory where the job runs

- Python
Published by github-actions[bot] 11 months ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a48

[integration]

*WorkloadManagement

FIX: (#8134) replace DIRACJOBID with JOBID in JobWrapper environment FIX: (#8126) Write DiracX token in JobAgent FIX: (#8126) Remove platform validation as extensions can redefine it FIX: (#8125) add VO information to the pilotDict when killing pilots FIX: (#8124) report the message of the Exception instead of the Exception itself in JobAgent.submitJob

*Core

CHANGE: (#8131) DIRAC.initialize(): ensure host credentials are not ignored in case passed as a list and not a tuple

*ConfigurationSystem

FIX: (#8127) dirac-admin-update-pilot can work without a specified VO

*Resources

FIX: (#8124) do not try to use a malformed StorageElement instance in SingularityCE

- Python
Published by github-actions[bot] 11 months ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a47

[integration]

*WorkloadManagement

FIX: (#8123) PilotManager not using tokens to kill pilots NEW: (#8119) JobStateUpdate legacy adapter

*RequestManagementSystem

CHANGE: (#8121) No longer directly use of the JobStateUpdateClient when processing requests FIX: (#8121) Calling setJobParameter when processing requests

*TransformationSystem

CHANGE: (#8121) No longer directly use of the JobStateUpdateClient from the DataRecoveryAgent

*DBs

FIX: (#8120) Escape password when giving it to sqlalchemy

*WorkloadManagementSystem

FIX: (#8116) avoid repeating optimization when job goes to STAGING

*Resources

FIX: (#8115) Fix AREX CE pilot logs in alwaysIncludeProxy case

- Python
Published by github-actions[bot] 11 months ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a46

[integration]

*Subsystem

NEW: (#8109) added setInputData to JobState

*ConfigurationSystem

FIX: (#8108) VOMS2CSAgent newDiracName might not be defined yet

- Python
Published by github-actions[bot] 11 months ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a45

[integration]

*WorkloadManagemnt

FIX: (#8106) Clear any non-UTF encodable environment variables in pilots

*ConfigurationSystem

NEW: (#8104) backport the export of the Sub in DiracX

*Core

FIX: (#8102) Don't use string processing on X509 name objects NEW: (#8099) -S option on dirac-configure is now ignored FIX: (#8088) force M2Crypto to use the proxy instead of the host certificate if provided

*WorkloadManagement

FIX: (#8100) Move JobWrapperUtilities after import Script.parseCommandLine()

*Resources

FIX: (#8086) Catch ConnectionError when calling send on a MQ

- Python
Published by github-actions[bot] 11 months ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a44

[integration]

*Resources

FIX: (#8084) htcondor x509 unsupported version CHANGE: (#8075) Disable Bearer token for HTTPs unless upload/TPC CHANGE: (#8074) conditionally reset the rlimit for xroot NEW: (#8069) findFileByMetadata method for Rucio

*Core

FIX: (#8080) dirac-apptainer-exec should work also in the case of no proxy

*WorkloadManagement

FIX: (#8073) pass args to buildQueueDict() in the right order

*Workload Management

FIX: (#8067) Supress non-UTF8 variables from pilot environment

*ConfigurationSystem

CHANGE: (#8061) TTLCache for getProxyPrrovidersForDN

*Test

NEW: (#8005) use containerized certs creation for integration tests

- Python
Published by github-actions[bot] 12 months ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.71

[rel-v8r0]

*Resources

FIX: (#8085) Catch ConnectionError when calling send on a MQ FIX: (#8083) htcondor x509 unsupported version CHANGE: (#8072) conditionally reset the rlimit for xroot CHANGE: (#8070) Disable Bearer token for HTTPs unless upload/TPC NEW: (#8046) findFileByMetadata method for Rucio

*Core

FIX: (#8071) dirac-apptainer-exec should work also in the case of no proxy

- Python
Published by github-actions[bot] 12 months ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.70

[rel-v8r0]

*WorkloadManagement

FIX: (#8064) create valid request for delayed sandbox purging

*Workload Management

FIX: (#8063) Supress non-UTF8 variables from pilot environment

*ConfigurationSystem

CHANGE: (#8059) TTLCache for getProxyPrrovidersForDN

- Python
Published by github-actions[bot] 12 months ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a43

[integration]

*Documentation

CHANGE: (#8055) extend pilot documentation.

*docs

FIX: (#8054) update and correct the README to build the DIRAC documenation

*Subsystem

CHANGE: (#8044) default MySQL version from 8.0 to 8.4

- Python
Published by github-actions[bot] about 1 year ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.69

[rel-v8r0]

*docs

FIX: (#8053) update and correct the README to build the DIRAC documenation

*Subsystem

CHANGE: (#8039) default MySQL version from 8.0 to 8.4

- Python
Published by github-actions[bot] about 1 year ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a42

[integration]

*Core

NEW: (#8041) new command dirac-apptainer-exec for running DIRAC commands inside apptainer FIX: (#8034) Correct user mapping for DiracX from IAM

*WorkloadManagement

FIX: (#8040) get ElasticJobParametersDB index prefix from the configuration CHANGE: (#8014) Use apptainer for SingularityComputingElement CHANGE: (#8014) Drop support for SingularityComputingElement without user namespaces CHANGE: (#8014) Enchance debugging output if SingularityComputingElement fails CHANGE: (#8014) Drop support for using apptainer from outside of DIRACOS2

*ConfigurationSystem

FIX: (#8040) evaluate useCRT flag as boolean in Utilities.getElasticDBParameters()

- Python
Published by github-actions[bot] about 1 year ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.68

[rel-v8r0]

*WorkloadManagement

FIX: (#8037) get ElasticJobParametersDB index prefix from the configuration

*ConfigurationSystem

FIX: (#8037) evaluate useCRT flag as boolean in Utilities.getElasticDBParameters()

*Core

NEW: (#8036) new command dirac-apptainer-exec for running DIRAC commands inside apptainer

- Python
Published by github-actions[bot] about 1 year ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a41

[integration]

*TransformationSystem

FIX: (#8031) make the setting of inputDataBulk extendable FIX: (#8022) make 2 methods of WorkflowTaskAgent extendable

*Core

FIX: (#8029) read at most 2^14 bytes at the same time FIX: (#8002) one less flag for EnableSecurityLogging FIX: (#7988) Add locks to AuthManager caches CHANGE: (#7971) Optimise ASN1 decoding in X509Certificate FIX: (#7969) Avoid locking in MessageQueueHandler

*Resources

FIX: (#8027) explicitly disconnect Stomp before reconnecting FIX: (#8012) adapt AREX to ARC7 delegation output FIX: (#7995) adapt HTCondorCE to latest htcondor version

*WorkloadManagementSystem

FIX: (#8020) StalledJobAgent: if StartExecTime is not set, use the last recorded heartbeat FIX: (#7986) PilotBundle: compatibility with py2 and py3

*DataManagementSystem

FIX: (#8003) exit with status 1 in case of error FIX: (#7989) Apply a workaround for https://github.com/xrootd/xrootd/issues/2396

*CI

FIX: (#7999) Add PilotAgentsDB definition to tests/CI/docker-compose.yml

FIX: (#7997) pensearch configuration should consume the ca_certs parameter if it is there

*ResourceStatusSystem

CHANGE: (#7987) dirac-rss-sync: flip the default status to Active FIX: (#7983) Docs: removed Setup from Operations NEW: (#7972) Add a timeout for GocDB to avoid blocking the CacheFeederAgent

*MonitoringSystem

CHANGE: (#7978) removed ElasticSearch in favor or OpenSearch

*RequestManagementSystem

NEW: (#7975) Allow to disable DM transfer as an FTS failover

*FrameworkSystem

FIX: (#7970) Support https URLs with dirac-framework-ping-service

*Integration tests

FIX: (#7968) mount diracx in init-db container

- Python
Published by github-actions[bot] about 1 year ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.67

[rel-v8r0]

- Python
Published by github-actions[bot] about 1 year ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.66

[rel-v8r0]

*Core

FIX: (#8028) read at most 2^14 bytes at the same time

*TransformationSystem

FIX: (#8026) make the setting of inputDataBulk extendable

*Resources

FIX: (#8025) explicitly disconnect Stomp before reconnecting

- Python
Published by github-actions[bot] about 1 year ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.65

[rel-v8r0]

*TransformationSystem

FIX: (#8019) make 2 methods of WorkflowTaskAgent extendable

*WorkloadManagementSystem

FIX: (#8015) StalledJobAgent: if StartExecTime is not set, use the last recorded heartbeat

*Resources

FIX: (#8011) adapt AREX to ARC7 delegation output

*DataManagementSystem

FIX: (#8001) exit with status 1 in case of error

*WorkloadManagement

CHANGE: (#8000) Use apptainer for SingularityComputingElement CHANGE: (#8000) Drop support for SingularityComputingElement without user namespaces CHANGE: (#8000) Enchance debugging output if SingularityComputingElement fails CHANGE: (#8000) Drop support for using apptainer from outside of DIRACOS2

- Python
Published by github-actions[bot] about 1 year ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.64

[rel-v8r0]

*Resources

FIX: (#7993) adapt HTCondorCE to latest htcondor version

FIX: (#7992) pensearch configuration should consume the ca_certs parameter if it is there

- Python
Published by github-actions[bot] about 1 year ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.63

[rel-v8r0]

*Core

FIX: (#7984) Add locks to AuthManager caches

*ResourceStatusSystem

CHANGE: (#7982) dirac-rss-sync: flip the default status to Active FIX: (#7979) Docs: removed Setup from Operations

*WorkloadManagementSystem

FIX: (#7977) PilotBundle: compatibility with py2 and py3

*DataManagementSystem

FIX: (#7976) Apply a workaround for https://github.com/xrootd/xrootd/issues/2396

- Python
Published by github-actions[bot] about 1 year ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.62

[rel-v8r0]

*RequestManagementSystem

NEW: (#7973) Allow to disable DM transfer as an FTS failover

*ResourceStatusSystem

NEW: (#7967) Add a timeout for GocDB to avoid blocking the CacheFeederAgent

*FrameworkSystem

FIX: (#7965) Support https URLs with dirac-framework-ping-service

*Core

FIX: (#7964) Avoid locking in MessageQueueHandler CHANGE: (#7963) Optimise ASN1 decoding in X509Certificate CHANGE: (#7957) Introduce caches to AuthManager.getUsername

- Python
Published by github-actions[bot] about 1 year ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a40

[integration]

*Core

CHANGE: (#7961) Introduce caches to AuthManager.getUsername

- Python
Published by github-actions[bot] about 1 year ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a39

[integration]

*WorkloadManagement

FIX: (#7954) correctly log the pilot job reference during the matching process

- Python
Published by github-actions[bot] about 1 year ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a38

[integration]

*TransformationSystem

FIX: (#7953) use updatemany in TransformationDB

*WorkloadManagement

FIX: (#7949) renew delegation prior to submitting pilots FIX: (#7941) integrity check failure in RemoteRunner FIX: (#7935) Run CE cleanup step at correct point

*ConfigurationSystem

FIX: (#7947) Make writing the CS atomic

*RequestManagementSystem

FIX: (#7934) call the correct executeRPCMethod depending on diset/diracx

- Python
Published by github-actions[bot] about 1 year ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.61

[rel-v8r0]

*ConfigurationSystem

FIX: (#7946) Make writing the CS atomic

*WorkloadManagement

FIX: (#7940) integrity check failure in RemoteRunner FIX: (#7936) renew delegation prior to submitting pilots FIX: (#7924) correctly log the pilot job reference during the matching process

*TransformationSystem

FIX: (#7898) use updatemany in TransformationDB

- Python
Published by github-actions[bot] about 1 year ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a37

[integration]

*TransformationSystem

FIX: (#7927) Use parameterised query in addTransformation FIX: (#7910) Use UTC to calculate older in export_getTasksToSubmit

*WorkloadManagment

CHANGE: (#7922) Better caching performance in the Matching Limiter

*Resources

CHANGE: (#7918) Add option to include proxy on AREX token submission

*Integration tests

CHANGE: (#7915) Upgrade Indigo IAM (1.10.2)

- Python
Published by github-actions[bot] about 1 year ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.60

[rel-v8r0]

*Resources

CHANGE: (#7913) Add option to include proxy on AREX token submission

*Integration tests

CHANGE: (#7911) Upgrade Indigo IAM (1.10.2)

*TransformationSystem

FIX: (#7909) Use UTC to calculate older in export_getTasksToSubmit

*WorkloadManagment

CHANGE: (#7907) Better caching performance in the Matching Limiter

- Python
Published by github-actions[bot] about 1 year ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.59

[rel-v8r0]

*FrameworkSystem

CHANGE: (#7895) Rename TokenManager service classes to follow standard convention for tornado and diset handlers NEW: (#7888) The DISET version of the TokenManager service

*TransformationSystem

FIX: (#7894) Use parameterised query in addTransformation CHANGE: (#7874) the TransformationCleaningAgent forces jobs to KILLED

*WorkloadManagementSystem

CHANGE: (#7891) added ulimit -n 1048575 to pilotwrapper content NEW: (#7866) for parametric jobs, added the possibility to bulk insert records in JobLoggingDB

*WorkloadManagement

CHANGE: (#7886) Use plain proxy for the pilot bundle

*Core

CHANGE: (#7875) Remove lock in Logging._createLogRecord

- Python
Published by github-actions[bot] over 1 year ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a36

[integration]

*TransformationSystem

CHANGE: (#7906) the TransformationCleaningAgent forces jobs to KILLED

*WorkloadManagementSystem

NEW: (#7905) for parametric jobs, added the possibility to bulk insert records in JobLoggingDB CHANGE: (#7902) added ulimit -n 1048575 to pilotwrapper content

*FrameworkSystem

CHANGE: (#7901) Rename TokenManager service classes to follow standard convention for tornado and diset handlers

Thank you for writing the text to appear in the release notes. It will show up exactly as it appears between the two bold lines Please follow the template:

*CI

NEW: (#7899) Add PilotLogsDB to diracx environment to allow CI testing. For examples look into release.notes

*Misc

FIX: (#7893) Tidy up whitespace in cvmfs.yml

*WorkloadManagement

CHANGE: (#7892) Use plain proxy for the pilot bundle

- Python
Published by github-actions[bot] over 1 year ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a35

- Python
Published by github-actions[bot] over 1 year ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a34

[integration]

*DataManagementSystem

FIX: (#7879) Fix the condition for return type sanitation

- Python
Published by github-actions[bot] over 1 year ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a33

[integration]

*Core

CHANGE: (#7876) Remove lock in Logging._createLogRecord FIX: (#7864) Prevent -1 returned from oSocket.write being interpreted as number of bytes to avoid an infinite loop in DISET BaseTransport.sendData. FIX: (#7856) ensure processProposal always returns "closeTransport" on error FIX: (#7843) Returning DISET responses with >100000000 bytes

*ResourceStatusSystem

FIX: (#7871) some protocols or types are not in GOC

*TransformationSystem

FIX: (#7865) getTasksToSubmit consider tasks inserted by 30 seconds or more NEW: (#7843) Use getTransformationFilesAsJsonString for faster getTransformationFiles

*StompMQConnector

FIX: (#7855) add a timeout for the StompConnector to handle nonresponsive sockets which can cause logging to be blocked

*DataManagementSystem

FIX: (#7850) Fix deadlock when FTS3Agent._treatOperation fails

- Python
Published by github-actions[bot] over 1 year ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.58

- Python
Published by github-actions[bot] over 1 year ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.57

[rel-v8r0]

*TransformationSystem

FIX: (#7861) getTasksToSubmit consider tasks inserted by 30 seconds or more

*ResourceStatusSystem

FIX: (#7859) some protocols or types are not in GOC

*DataManagementSystem

FIX: (#7857) Fix the condition for return type sanitation

- Python
Published by github-actions[bot] over 1 year ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a32

[integration]

*Resources

FIX: (#7839) hide private keys from the logs FIX: (#7832) AREX interactions with tokens FIX: (#7831) Drop CloudCE proxy handling FIX: (#7829) CloudCE: truncate long messages returned from create_node

*Core

CHANGE: (#7837) minimum version of some packages

*TransformationSystem

NEW: (#7833) CHANGE: (#7833) Improve getTransformationFiles performance

- Python
Published by github-actions[bot] over 1 year ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.56

[rel-v8r0]

*StompMQConnector

FIX: (#7854) add a timeout for the StompConnector to handle nonresponsive sockets which can cause logging to be blocked

*Core

FIX: (#7852) ensure processProposal always returns "closeTransport" on error FIX: (#7848) Prevent -1 returned from oSocket.write being interpreted as number of bytes to avoid an infinite loop in DISET BaseTransport.sendData.

*DataManagementSystem

FIX: (#7847) Fix deadlock when FTS3Agent._treatOperation fails FIX: (#7844) fix wrong return type in the FTS3Agent, when using tokens

- Python
Published by github-actions[bot] over 1 year ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.55

[rel-v8r0]

*Core

FIX: (#7840) Returning DISET responses with >100000000 bytes

*TransformationSystem

NEW: (#7840) Use getTransformationFilesAsJsonString for faster getTransformationFiles

*Resources

FIX: (#7835) hide private keys from the logs

- Python
Published by github-actions[bot] over 1 year ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.54

[rel-v8r0]

*Resources

FIX: (#7824) AREX interactions with tokens FIX: (#7823) Drop CloudCE proxy handling FIX: (#7822) CloudCE: truncate long messages returned from create_node FIX: (#7818) AREXCE returns an error if a queue is not found in the ARC instance configuration

*TransformationSystem

NEW: (#7812) CHANGE: (#7812) Improve getTransformationFiles performance

FIX: (#7811) SSHCE, Try python3 before unversioned python

- Python
Published by github-actions[bot] over 1 year ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a31

[integration]

*WorkloadManagement

FIX: (#7827) SandboxStore not able to assign a sandbox to a job FIX: (#7825) JobWrapper checks existence of executable in jobIDPath FIX: (#7821) Getting pilot reference from job parameters

*Resources

FIX: (#7819) AREXCE returns an error if a queue is not found in the ARC instance configuration

- Python
Published by github-actions[bot] over 1 year ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a30

[integration]

*DataManagementSystem

NEW: (#7817) experimental token support for FTS transfers FIX: (#7781) diracdmsfindlfns: Check if requested path exists and returns error if not. Prevents users hitting #7487 FIX: (#7780) Return correct error for non-existing directory. Partial fix for #7487 NEW: (#7764) added checksum comparison in the fc/se consistency tool NEW: (#7756) FileCatalogHandler: add function exportgetFileDetails to get the (user) metadata for a list of LFNs FIX: (#7719) Remove def findDirIDsByMetadata(self, metaDict, dPath, credDict): method from MultiVODirectoryMetadata (derived) class which caused an extra VO suffix added when searching. The method is meant to be used internally only on keys which are already expanded in a MultiVO case. Add a user-level def findDirectoriesByMetadata(self, queryDict, path, credDict) to the derived class thus adding a VO suffix for a directory search. Fixes #7687. CHANGE: (#7694) dirac-dms-replicate-and-register-request: Make request chunk size configurable; default behaviour unchanged.

*FrameworkSystem

NEW: (#7817) allow to bypass cache when retrieving tokens

FIX: (#7813) SSHCE, Try python3 before unversioned python Fix a typo in the key Addler -> Adler of the return dict of putAndRegister

*TransformationSystem

FIX: (#7806) RequestTaskAgent only considers requests in final states, and consider files in intermediate state as problematic (https://github.com/DIRACGrid/DIRAC/issues/7116) NEW: (#7806) RequestTaskAgent uses getBulkRequestStatus instead of getRequestStatus RMS: (#7806) NEW: (#7806) implement getRequestStatus NEW: (#7697) InputDataAgent: new Option MultiVO, which makes the FileCatalog Query use the author of the DN, rather than the Host, to resolve MultiVO metadata correctly. Fixes #7681

*Resources

FIX: (#7803) added a 30s gfal2 timeout for downloading the SRR FIX: (#7790) SSHComputingElement fix: added check of result FIX: (#7726) Update CloudCE cloudinit.template for EL8+ CHANGE: (#7715) HTCondorCE: UseSSLSubmission: use the generated proxy file for everything, no longer need to have certificate of user present on the server. FIX: (#7713) AREX submission issue not properly handled NEW: (#7695) better error message when no matching protocol between 2 SE CHANGE: (#7689) remove BOINCCE

*ResourceStatusSystem

FIX: (#7801) use always a from address (from Operations ResourceStatus/Config/FromAddress ) when sending email notifications, to avoid "spoofing" domains restrictions NEW: (#7783) add a DIRAC to GOCDB service type conversion CHANGE: (#7774) do not consider Endpoint for StorageOccupancy FIX: (#7765) Do not use tinezone aware datetime FIX: (#7755) delete the Occupancy cache only for older entries

*WorkloadManagement

FIX: (#7797) Allow jobs to be KILLED from more states FIX: (#7797) KILLED is a final job state FIX: (#7716) PilotWrapper - check for the presence of the -l pilot option

*ConfigurationSystem

CHANGE: (#7796) VOMS2CSAgent: if a nickname is set, this nickname will always be used and no new accounts are going to be created if a DN changes or a user is in multiple VOs NEW: (#7796) VOMS2CSAgent: New option "ForceNickname", if this option is enabled no dirac user is created if no nickname attribute is set for a user CHANGE: (#7796) IAMService: use logger and return errors for users so that the VOMS2CSAgent can notify admins about issues NEW: (#7742) Configuration-system-shell: added reload and sort commands

*WorkloadManagementSystem

CHANGE: (#7792) SiteDirector will always bundle the proxy CHANGE: (#7762) removed JobDB's SiteMask and Logging FIX: (#7751) Proper killing of jobs when not matched, running or stalled FIX: (#7707) JobDB: update LastUpdateTime when the job is matched NEW: (#7699) added possibility to specify UserEnvVariable (pilot option) at CE level

*Workflow

FIX: (#7786) Avoid incorrect error strings in Workflow execute

*Core

NEW: (#7775) Support aggregating by date in MySQL.getCounters FIX: (#7750) add more safeguard when processing result queue in ProcessPool CHANGE: (#7720) register the DictCache destructor as an atexit handler NEW: (#7700) fetch nickname attribute from IAM CHANGE: (#7696) drop DIRACMYSQLCONNECTIONGRACETIME and stalled connections reuse

*Subsystem

CHANGE: (#7733) For PilotWrapper tests, use the artifacts (created in Pilot repo)

*Diracx

NEW: (#7711) populate diracx section from Iam

*Doc

FIX: (#7688) Move the explanation how to enable tokens to a place that is shown in ReadTheDocs in the end.

*Interfaces

FIX: (#7684) Document BadJobParameterError

- Python
Published by github-actions[bot] over 1 year ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.53

[rel-v8r0]

*Resources

FIX: (#7787) added a 30s gfal2 timeout for downloading the SRR

*TransformationSystem

FIX: (#7741) RequestTaskAgent only considers requests in final states, and consider files in intermediate state as problematic (https://github.com/DIRACGrid/DIRAC/issues/7116) NEW: (#7741) RequestTaskAgent uses getBulkRequestStatus instead of getRequestStatus RMS: (#7741) NEW: (#7741) implement getRequestStatus

- Python
Published by github-actions[bot] over 1 year ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.52

[rel-v8r0]

*ResourceStatusSystem

FIX: (#7800) use always a from address (from Operations ResourceStatus/Config/FromAddress ) when sending email notifications, to avoid "spoofing" domains restrictions

*WorkloadManagement

FIX: (#7794) Allow jobs to be KILLED from more states FIX: (#7794) KILLED is a final job state

*WorkloadManagementSystem

CHANGE: (#7789) SiteDirector will always bundle the proxy

*Resources

FIX: (#7788) SSHComputingElement fix: added check of result

*Workflow

FIX: (#7785) Avoid incorrect error strings in Workflow execute

*ConfigurationSystem

CHANGE: (#7784) VOMS2CSAgent: if a nickname is set, this nickname will always be used and no new accounts are going to be created if a DN changes or a user is in multiple VOs NEW: (#7784) VOMS2CSAgent: New option "ForceNickname", if this option is enabled no dirac user is created if no nickname attribute is set for a user CHANGE: (#7784) IAMService: use logger and return errors for users so that the VOMS2CSAgent can notify admins about issues

- Python
Published by github-actions[bot] over 1 year ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.51

[rel-v8r0]

*DataManagementSystem

FIX: (#7779) diracdmsfindlfns: Check if requested path exists and returns error if not. Prevents users hitting #7487 FIX: (#7778) Return correct error for non-existing directory. Partial fix for #7487 NEW: (#7740) FileCatalogHandler: add function exportgetFileDetails to get the (user) metadata for a list of LFNs

*Core

NEW: (#7772) Support aggregating by date in MySQL.getCounters

*ResourceStatusSystem

CHANGE: (#7771) do not consider Endpoint for StorageOccupancy FIX: (#7761) Do not use tinezone aware datetime NEW: (#7759) add a DIRAC to GOCDB service type conversion

- Python
Published by github-actions[bot] over 1 year ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.50

[rel-v8r0]

*ResourceStatusSystem

FIX: (#7748) delete the Occupancy cache only for older entries

*ConfigurationSystem

NEW: (#7739) Configuration-system-shell: added reload and sort commands

*Core

FIX: (#7738) add more safeguard when processing result queue in ProcessPool CHANGE: (#7653) register the DictCache destructor as an atexit handler

*Subsystem

CHANGE: (#7728) For PilotWrapper tests, use the artifacts (created in Pilot repo)

*Resources

FIX: (#7725) Update CloudCE cloudinit.template for EL8+

*DataManagementSystem

FIX: (#7708) Remove def findDirIDsByMetadata(self, metaDict, dPath, credDict): method from MultiVODirectoryMetadata (derived) class which caused an extra VO suffix added when searching. The method is meant to be used internally only on keys which are already expanded in a MultiVO case. Add a user-level def findDirectoriesByMetadata(self, queryDict, path, credDict) to the derived class thus adding a VO suffix for a directory search. Fixes #7687.

*WorkloadManagementSystem

FIX: (#7690) Proper killing of jobs when not matched, running or stalled

- Python
Published by github-actions[bot] over 1 year ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.49

[rel-v8r0]

*WorkloadManagement

FIX: (#7714) PilotWrapper - check for the presence of the -l pilot option

*Resources

FIX: (#7712) AREX submission issue not properly handled CHANGE: (#7702) HTCondorCE: UseSSLSubmission: use the generated proxy file for everything, no longer need to have certificate of user present on the server. NEW: (#7682) better error message when no matching protocol between 2 SE FIX: (#7675) HTCondorCE: fix exception when UseSSLSubmission is true. The SiteDirector environment does not have HOME. Always use /home/dirac/.globus to get userkey and usercert files.

*WorkloadManagementSystem

FIX: (#7701) JobDB: update LastUpdateTime when the job is matched NEW: (#7698) added possibility to specify UserEnvVariable (pilot option) at CE level FIX: (#7668) CVMFS_locations might be an empty list

*Core

CHANGE: (#7692) drop DIRACMYSQLCONNECTIONGRACETIME and stalled connections reuse NEW: (#7691) fetch nickname attribute from IAM FIX: (#7675) Locations.getCertificateAndKeyLocation: fix exception when HOME is not set. FIX: (#7666) Replace __del__ with weakref.finalize in DictCache

*Doc

FIX: (#7685) Move the explanation how to enable tokens to a place that is shown in ReadTheDocs in the end.

*TransformationSystem

NEW: (#7683) InputDataAgent: new Option MultiVO, which makes the FileCatalog Query use the author of the DN, rather than the Host, to resolve MultiVO metadata correctly. Fixes #7681

*DataManagementSystem

CHANGE: (#7672) dirac-dms-replicate-and-register-request: Make request chunk size configurable; default behaviour unchanged.

- Python
Published by github-actions[bot] over 1 year ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.48

[rel-v8r0]

*Resources

CHANGE: (#7656) mark ARC and ARC6 as deprecated

*WorkloadManagement

FIX: (#7655) remove exception log when output is not found

*FrameworkSystem

FIX: (#7651) close open file (DISET!)

*Core

CHANGE: (#7642) min version of rucio-client for speeding up pip resolution

*WorkloadManagementSystem

FIX: (#7639) added log headers to InputDataResolution modules

*Accounting

FIX: (#7637) AccountingDB only generate condition if needed

*ConfigurationSystem

NEW: (#7612) VOMS2CSAgent can query IAM

- Python
Published by github-actions[bot] over 1 year ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a29

[integration]

*WorkloadManagementSystem

FIX: (#7649) added log headers to InputDataResolution modules CHANGE: (#7629) add jobGroup to job parameters FIX: (#7584) ServerUtils: prevent getPilotAgentsDB from returning None FIX: (#7576) Fix potential circular import in WorkflowReader. Mostly seen in the creation of the documentation.

*WorkloadManagement

NEW: (#7643) Support Pydantic 2 FIX: (#7621) remove random shuffle in PilotWrapper CHANGE: (#7609) Perform bulk lookup of job parameters from elasticsearch CHANGE: (#7608) Make RemoteRunner more resilient to CE issues FIX: (#7594) JobMonitoring.getJobParameters should pass jobID as an int to ElasticJobParametersDB FIX: (#7590) AREX "out" and "err" need to exist before file integrity check

*Accounting

FIX: (#7640) AccountingDB only generate condition if needed

*Resources

NEW: (#7638) HTCondorCE: Added UseSSLSubmission option. Allows one to use a configured DN at given Sites for job submission, instead of proxies or tokens. Only at participating CEs and conditions apply.

*Core

FIX: (#7634) Avoid printing out clear text password in SQLAlchemy FIX: (#7591) File.secureOpenForWrite: fix exception when opening in binary mode, fixes #7581

*DataManagementSystem

NEW: (#7633) Add a protocol parameter to the getReplicas method family NEW: (#7619) prepare for FTS 3.13 release with breaking API NEW: (#7617) DataManager.putAndRegister rejects too long filename

*Deployment

FIX: (#7628) fix the path of the CVMFS sync_packages.sh script

*All

FIX: (#7616) fix pylint 3.2.0 warnings

*MonitoringSystem

FIX: (#7584) ServerUtils: prevent getMonitoringDB from returning None

- Python
Published by github-actions[bot] over 1 year ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.47

[rel-v8r0]

*DataManagementSystem

NEW: (#7632) Add a protocol parameter to the getReplicas method family

*Core

FIX: (#7631) Avoid printing out clear text password in SQLAlchemy

*Resources

NEW: (#7630) HTCondorCE: Added UseSSLSubmission option. Allows one to use a configured DN at given Sites for job submission, instead of proxies or tokens. Only at participating CEs and conditions apply.

*Deployment

FIX: (#7626) fix the path of the CVMFS sync_packages.sh script

*WorkloadManagementSystem

CHANGE: (#7599) add jobGroup to job parameters

- Python
Published by github-actions[bot] over 1 year ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.46

[rel-v8r0]

*WorkloadManagement

FIX: (#7620) remove random shuffle in PilotWrapper

*All

FIX: (#7614) fix pylint 3.2.0 warnings

*DataManagementSystem

NEW: (#7613) prepare for FTS 3.13 release with breaking API NEW: (#7604) DataManager.putAndRegister rejects too long filename

- Python
Published by github-actions[bot] almost 2 years ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.45

[rel-v8r0]

*WorkloadManagement

CHANGE: (#7606) Make RemoteRunner more resilient to CE issues CHANGE: (#7603) Perform bulk lookup of job parameters from elasticsearch

- Python
Published by github-actions[bot] almost 2 years ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.44

[rel-v8r0]

*WorkloadManagement

FIX: (#7588) AREX "out" and "err" need to exist before file integrity check

*Core

FIX: (#7583) File.secureOpenForWrite: fix exception when opening in binary mode, fixes #7581

- Python
Published by github-actions[bot] almost 2 years ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.43

[rel-v8r0]

*MonitoringSystem

FIX: (#7578) ServerUtils: prevent getMonitoringDB from returning None

*WorkloadManagementSystem

FIX: (#7578) ServerUtils: prevent getPilotAgentsDB, getVirtualMachineDB from returning None

- Python
Published by github-actions[bot] almost 2 years ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.42

[rel-v8r0]

*WorkloadManagementSystem

FIX: (#7575) Fix potential circular import in WorkflowReader. Mostly seen in the creation of the documentation. FIX: (#7572) serverUtils: jit imports

*Core

FIX: (#7568) Support M2Crypto 0.40.0+ CHANGE: (#7561) Replace the default PFN type ROOT_All with ROOT

*WorkloadManagement

FIX: (#7565) support file:/... as a location for the pilot files FIX: (#7563) make sure CVMFS_locations is a list

- Python
Published by github-actions[bot] almost 2 years ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a28

[integration]

*WorkloadManagementSystem

FIX: (#7574) serverUtils: jit imports FIX: (#7534) Pilots submitted by SiteDirector won't add the pythonVersion flag FIX: (#7521) Fix memory reporting FIX: (#7510) SandboxStore: add VO if needed

*WorkloadManagement

FIX: (#7571) support file:/... as a location for the pilot files FIX: (#7564) make sure CVMFS_locations is a list CHANGE: (#7553) Remove files from the RemoteRunner execution FIX: (#7552) JobCleaningAgent: fix exception in deleteJobsByStatus caused by mismatching job ID types NEW: (#7529) introduce JobWrapperOfflineTemplate for uses in systems without external connectivity CHANGE: (#7460) introduce JobWrapper.preprocess, process and postprocess

*test

NEW: (#7570) added pilot workflow tests to integration_tests

*Core

FIX: (#7569) Support M2Crypto 0.40.0+ CHANGE: (#7566) Replace the default PFN type ROOT_All with ROOT FIX: (#7524) Depend on packaging

*environment.yml

NEW: (#7555) add cwltool

*Resources

FIX: (#7545) TimeLeft utility was unable to get values from the cfg FIX: (#7532) support the case where HTCondor kills the jobs

*Test

FIX: (#7540) Redirect the output of popen in a file to fix #7473 FIX: (#7539) extra_module default to empty list

*FrameworkSystem

CHANGE: (#7511) ProxyDB: removed tables ProxyDBProxies and ProxyDBTokens

- Python
Published by github-actions[bot] almost 2 years ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.41

[rel-v8r0]

*WorkloadManagement

FIX: (#7550) JobCleaningAgent: fix exception in deleteJobsByStatus caused by mismatching job ID types CHANGE: (#7549) Remove files from the RemoteRunner execution

*environment.yml

NEW: (#7542) add cwltool

*Resources

FIX: (#7541) TimeLeft utility was unable to get values from the cfg FIX: (#7530) support the case where HTCondor kills the jobs

*Test

FIX: (#7537) extra_module default to empty list

*WorkloadManagementSystem

FIX: (#7531) Pilots submitted by SiteDirector won't add the pythonVersion flag

*Core

FIX: (#7523) Depend on packaging

- Python
Published by github-actions[bot] almost 2 years ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.40

[rel-v8r0]

*WorkloadManagementSystem

FIX: (#7518) Fix memory reporting FIX: (#7504) update pilot status in case the pilot failed before being able to install DIRAC, the pilot status was not updated and got stuck to 'Submitted'.

*FrameworkSystem

FIX: (#7514) ProxyManagerClient: do not check for Group

*Test

CHANGE: (#7499) use pytest-rerun instead of flaky

*Core

FIX: (#7496) plotting TypeLoader works with editable installation

- Python
Published by github-actions[bot] almost 2 years ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a27

[integration]

*Core

FIX: (#7505) plotting TypeLoader works with editable installation NEW: (#7453) Introduce an RPC stub equivalent for DiracX

*Test

CHANGE: (#7502) use pytest-rerun instead of flaky

*WorkloadManagementSystem

CHANGE: (#7498) removed GridEnv FIX: (#7497) If the SoftwareDistModule is set in the Operations Section, add it to the Job JDL to restore previous behaviour NEW: (#7453) FutureJobStateUpdate.setJobStatusBulk return a DiracX RPC stub

*WorkloadManagement

FIX: (#7493) check the VO from the task queues before submitting pilots FIX: (#7488) JobAgent.setupProxy takes owner instead of ownerDN

*RequestManagementSystem

CHANGE: (#7453) adapt ForwardDISET to DiracX stub

*FrameworkSystem

CHANGE: (#7442) removed NotificationDN

- Python
Published by github-actions[bot] almost 2 years ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.39

[rel-v8r0]

*WorkloadManagementSystem

FIX: (#7495) If the SoftwareDistModule is set in the Operations Section, add it to the Job JDL to restore previous behaviour

*FrameworkSystem

FIX: (#7489) dirac-proxy-info without the dirac group

*RequestManagementSystem

FIX: (#7479) RequestTask - download no-VOMS proxy if the owner group does not define VOMSRole

*Core

FIX: (#7478) Fix DISET calls with proxy to be used passed as an argument

*WorkloadManagement

FIX: (#7476) JobCleaningAgent - select random jobs for deletion rather than head and tail jobs. FIX: (#7474) JobID type in PushJobAgent

*Accounting

FIX: (#7466) Fix errors during insert into ac_in_* tables.

*Interfaces

CHANGE: (#7465) dfind - more explicit failure report

*DataManagementSystem

NEW: (#7370) add tools for consistency checks

- Python
Published by github-actions[bot] almost 2 years ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a26

[integration]

*FrameworkSystem

FIX: (#7491) dirac-proxy-info without the dirac group

*WorkloadManagement

FIX: (#7490) missing result in return statement

*tests

NEW: (#7484) add the state key in the test environment to fix diracx execution

- Python
Published by github-actions[bot] almost 2 years ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a25

[integration]

*Accounting

FIX: (#7486) Fix errors during insert into acin* tables.

*Core

FIX: (#7483) Fix DISET calls with proxy to be used passed as an argument

*RequestManagementSystem

FIX: (#7482) RequestTask - download no-VOMS proxy if the owner group does not define VOMSRole

- Python
Published by github-actions[bot] almost 2 years ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a24

[integration]

*WorkloadManagement

FIX: (#7480) JobCleaningAgent - select random jobs for deletion rather than head and tail jobs. FIX: (#7475) JobID type in PushJobAgent

*Interfaces

CHANGE: (#7472) dfind - more explicit failure report

*DataManagementSystem

NEW: (#7471) add tools for consistency checks

*tests

FIX: (#7470) check running containers in integration tests script

*WorkloadManagementSystem

CHANGE: (#7464) PilotAgentsDB: removed OutputReady and Broker fields

- Python
Published by github-actions[bot] about 2 years ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.38

[rel-v8r0]

*WorkloadManagement

FIX: (#7457) jobID type issue in JobAgent

*FrameworkSystem

FIX: (#7452) dirac-proxy-init printInfo without the dirac group

*ConfigurationSystem

FIX: (#7449) getQueue() overriding the CE tags

- Python
Published by github-actions[bot] about 2 years ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a23

[integration]

*WorkloadManagement

FIX: (#7458) jobID type issue in JobAgent

*ConfigurationSystem

FIX: (#7454) getQueue() overriding the CE tags

*FrameworkSystem

FIX: (#7451) dirac-proxy-init printInfo without the dirac group

- Python
Published by github-actions[bot] about 2 years ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.37

[rel-v8r0]

*WorkloadManagement

FIX: (#7444) get pilot logging info with a token from an AREXCE

*FrameworkSystem

FIX: (#7438) correctly set the duration of tokens in cache

*WorkloadManagementSystem

FIX: (#7435) JobDB: fix mismatch of string and integer jobIDs. In some cases API calls would fail because JobDB.getJobParameters and JobDB.getJobsAttributes return dictionaries with integer keys, while the function was called with a string jobID. This fixes for example the StalledJobAgent being unable to reschedule matched jobs. FIX: (#7434) StatesAccountingAgent: skip the first iteration in order to avoid double commit after a restart FIX: (#7415) Each job has its own JobReport in JobAgent FIX: (#7415) JobAgent exits when all the jobs have been processed

*MonitoringSystem

CHANGE: (#7431) moved to weekly indices for agent and service monitoring

- Python
Published by github-actions[bot] about 2 years ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a22

[integration]

*WorkloadManagement

FIX: (#7448) get pilot logging info with a token from an AREXCE

*WorkloadManagementSystem

FIX: (#7447) Each job has its own JobReport in JobAgent FIX: (#7447) JobAgent exits when all the jobs have been processed FIX: (#7446) StatesAccountingAgent: skip the first iteration in order to avoid double commit after a restart CHANGE: (#7439) SandboxStore: remove external SE feature FIX: (#7436) JobDB: fix mismatch of string and integer jobIDs. In some cases API calls would fail because JObDB.getJobParameters and JobDB.getJobsAttributes return dictionaries with integer keys, while the function was called with a string jobID. This fixes for example the StalledJobAgent being unable to reschedule matched jobs. NEW: (#7425) TornadoPilotLoggingHandler modify the handler to accept VO name sent by a pilot. Required in a case where the VO cannot be guessed from a proxy. This change requires https://github.com/DIRACGrid/Pilot/pull/230 NEW: (#7421) SandboxDB: add VO field CHANGE: (#7414) move the content of SubmissionPolicy in SiteDirector

*FrameworkSystem

FIX: (#7443) correctly set the duration of tokens in cache

*RequestManagementSystem

FIX: (#7441) make sure OwnerDN is defined before trying to access its value

*MonitoringSystem

CHANGE: (#7432) moved to weekly indices for agent and service monitoring

- Python
Published by github-actions[bot] about 2 years ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a21

[integration]

*Test

CHANGE: (#7417) write the diracx CsSync config in the CS instead of taking the yaml file from diracx repo

*FrameworkSystem

CHANGE: (#7413) removed Proxies persistency flag

*Core

FIX: (#7412) Adding VOMS extensions without having environment variables set NEW: (#7412) Add DIRACDISABLEGCONFIG_REFRESH environment variable to prevent gConfig being accidentally used FIX: (#7409) Use proxy lifetime for tokens from legacy proxy exchange (https://github.com/DIRACGrid/diracx/issues/130)

*WorkloadManagement

FIX: (#7409) Add DiracX to payload proxies used by compute elements (#7402) FIX: (#7406) SiteDirector should not interact with CEs if there is 0 pilot to submit

*WorkloadManagementSystem

CHANGE: (#7407) JobDB simplifications CHANGE: (#7405) Removed Private Pilot functionality

- Python
Published by github-actions[bot] about 2 years ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.36

[rel-v8r0]

*WorkloadManagement

FIX: (#7397) JobAgent rescheduling wrong jobs

*WorkloadManagementSystem

FIX: (#7394) Correctly collect platform list from queues

- Python
Published by github-actions[bot] about 2 years ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a20

[integration]

*WorkloadManagement

FIX: (#7399) JobAgent rescheduling wrong jobs FIX: (#7387) JobAgent interaction with JobMonitoringClient

*WorkloadManagementSystem

CHANGE: (#7396) PilotAgentsDB: move from OwnerGroup to VO FIX: (#7379) The callback for the Stager was failing, because of a type mismatch in the jobID used to retrieve the status. Jobs never came out of Staging. NEW: (#7375) pilotWrapper: using CVMFS_locations for discovering the pilot files

*FrameworkSystem

FIX: (#7391) send notifications for expiring proxies

Documentation on how to deploy a third party tool (fluent-bit) to grab, format and send Dirac current logs to ElasticSearch and/or splitted logs files

*Resources

FIX: (#7376) AREXCE should break when a valid delegation ID is found

*Core

FIX: (#7374) Converting p12 files with filenames containing special characters

- Python
Published by github-actions[bot] about 2 years ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.35

[rel-v8r0]

*WorkloadManagement

FIX: (#7380) JobAgent interaction with JobMonitoringClient

*WorkloadManagementSystem

FIX: (#7378) The callback for the Stager was failing, because of a type mismatch in the jobID used to retrieve the status. Jobs never came out of Staging.

- Python
Published by github-actions[bot] about 2 years ago

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.34

[rel-v8r0]

*Core

FIX: (#7372) Converting p12 files with filenames containing special characters

*WorkloadManagementSystem

NEW: (#7371) pilotWrapper: using CVMFS_locations for discovering the pilot files

*Resources

FIX: (#7367) AREXCE should break when a valid delegation ID is found

- Python
Published by github-actions[bot] about 2 years ago

Recent Releases of dirac

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a68

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a67

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a66

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a65

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a64

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a63

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a62

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.76

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a61

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a60

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a59

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.75

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a58

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a57

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a56

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.74

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a55

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a54

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.73

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a53

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.72

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a52

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a51

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a50

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a49

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a48

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a47

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a46

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a45

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a44

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.71

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.70

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a43

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.69

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a42

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.68

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a41

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.67

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.66

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.65

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.64

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.63

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.62

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a40

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a39

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a38

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.61

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a37

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.60

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.59

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a36

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a35

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a34

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a33

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.58

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.57

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a32

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.56

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.55

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.54

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a31

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a30

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.53

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.52

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.51

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.50

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.49

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.48

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a29

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.47

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.46

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.45

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.44

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.43

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.42

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a28

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.41

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v8.0.40

dirac - https://github.com/DIRACGrid/DIRAC/releases/tag/v9.0.0a27