Data

Tools

Indexes

Applications

Experiments

Open Source Science

Recent Releases of acdh-oeaw/acdh-repo

acdh-oeaw/acdh-repo - Link against the arche-docs in the README

- PHP
Published by zozlak 6 months ago

acdh-oeaw/acdh-repo - Optimized database-embedded search functions

get_relatives(bigint[], ...) and get_relatives_metadata(bigint[], ...) are now available.

SELECT (get_relatives(array_agg(id), ...)).* FROM table_returning_thousands_of_resource_ids is few orders of magnitude faster than SELECT (get_relatives(id, ...)).* FROM table_returning_thousands_of_resource_ids. Same goes for the get_relatives_metadata(bigint[], ...).

- PHP
Published by zozlak 8 months ago

acdh-oeaw/acdh-repo - Playing with the Zenodo integration

- PHP
Published by zozlak 8 months ago

acdh-oeaw/acdh-repo - Check Zenodo integration

- PHP
Published by zozlak 8 months ago

acdh-oeaw/acdh-repo - Bot indexing tuning

- PHP
Published by zozlak 11 months ago

acdh-oeaw/acdh-repo - New metadata read mode implemented

See the https://acdh-oeaw.github.io/arche-docs/aux/metadataapifor_programmers.html#readmode-in-details

- PHP
Published by zozlak 11 months ago

acdh-oeaw/acdh-repo - Database indexing tuning

- PHP
Published by zozlak 11 months ago

acdh-oeaw/acdh-repo - Bugfixes

Fix removal of temporary binary files

- PHP
Published by zozlak 12 months ago

acdh-oeaw/acdh-repo - Spatial indexing tuning

- PHP
Published by zozlak 12 months ago

acdh-oeaw/acdh-repo - Bugfixes

/search endpoint called with metadata read mode none still returns technical metadata triples - the count of resources matching the search and the match triples.

- PHP
Published by zozlak 12 months ago

acdh-oeaw/acdh-repo - Resource creation failure reporting tuning

- PHP
Published by zozlak about 1 year ago

acdh-oeaw/acdh-repo - /download endpoint auth checks tuned

- PHP
Published by zozlak about 1 year ago

acdh-oeaw/acdh-repo - backup.sh script tuned

- PHP
Published by zozlak about 1 year ago

acdh-oeaw/acdh-repo - /download batch binary data download endpoint added

See https://app.swaggerhub.com/apis/zozlak/arche/5.5#/default/get_download

- PHP
Published by zozlak over 1 year ago

acdh-oeaw/acdh-repo - zozlak/auth bumped to ^3

- PHP
Published by zozlak over 1 year ago

acdh-oeaw/acdh-repo - /user/logout endpoint implemented

The /user/logout REST API endpoint allows user to invalidate the HTTP Basic authentication in a browser.

Due to the very nature of the HTTP Basic auth this is still unintuitive because the user has to cancel the login dialog shown by the browser.

The endpoint supports the redirect parameter just as the /user endpoint.

- PHP
Published by zozlak over 1 year ago

acdh-oeaw/acdh-repo - Fix metadata removal of tombstone resources

- PHP
Published by zozlak over 1 year ago

acdh-oeaw/acdh-repo - URI FTS fixes

- PHP
Published by zozlak over 1 year ago

acdh-oeaw/acdh-repo - Fix identifiers indexing

tr_full_text_search_maintain3 database-embedded function adjusted so it index identifier URI path parts as individual tokens. This makes full text search work for just a numeric resource id or other URIs unique part (e.g. the last part of the geonames or GND URIs).

- PHP
Published by zozlak over 1 year ago

acdh-oeaw/acdh-repo - PHP 8.4 deprecation fixes

- PHP
Published by zozlak over 1 year ago

acdh-oeaw/acdh-repo - Optimize full_text_search table indices

- PHP
Published by zozlak over 1 year ago

acdh-oeaw/acdh-repo - Large files FTS indexing fix

- PHP
Published by zozlak over 1 year ago

acdh-oeaw/acdh-repo - Various fixes

  • Add support for skipContentDisposition query parameter for the GET /{resId} endpoint. If the parameter is present, the response doesn't contain the Content-Disposition header making a browser opening the returned resource content in-line instead of downloading to the disk (of course only if the resource format is supported by a browser).
  • Resources automatically created because they were referred to from metadata triple's object position now contain all system metadata (creation date and user, ACL metadata, etc.)
  • Better TransactionController logging (include transaction id in all messages)
  • TransactionController immediately releases resources it rolled back instead of doing that at the very end of the rollback.
  • GET /transaction endpoint reports number of resources locked by a given transaction.

- PHP
Published by zozlak over 1 year ago

acdh-oeaw/acdh-repo - Trap database deadlocks on final commit as HTTP 409

- PHP
Published by zozlak over 1 year ago

acdh-oeaw/acdh-repo - Authorization enhancements

  • The /user/{userId} and /user endpoints take an optional redirect query parameter which causes and HTTP 303 redirect to a given location after a successful login. Allowed redirect locations are controlled by the rest.userEndpointAllowedRedirectRegex configuration property.
  • The /user endpoint can be used by non-admin users although they see only it shows only their own account to them.
  • If accessControl.cookie config object is set, the authorization module sets up a cookie storing coma-separated list of all role of the user who performed a last successful authentication. Unsuccessful authentication attempt clear the cookie. This can be used to expose the current authentication status to other services in the same domain.
  • All authenticated users are now assigned accessControl.loggedInRole role (group). This allows setting up authorization rules affecting any authorized user. This option is particularly useful when combined with SSO users.

- PHP
Published by zozlak over 1 year ago

acdh-oeaw/acdh-repo - Bugfixes

Fix exceptions handling so that long exception messages do not force response status code to 200 because of the output buffer overflow

- PHP
Published by zozlak over 1 year ago

acdh-oeaw/acdh-repo - Detect image dimensions and store them in metadata

- PHP
Published by zozlak almost 2 years ago

acdh-oeaw/acdh-repo - Spatial indexing fixes

When a binary data is spatially indexed, its bounding box is stored in the index. That way precision is lost but it assures the search can be performed quickly even if the original geometry consists of thousands of non-intersecting elements.

- PHP
Published by zozlak almost 2 years ago

acdh-oeaw/acdh-repo - Auth fixes

- PHP
Published by zozlak almost 2 years ago

acdh-oeaw/acdh-repo - Ported from EasyRdf to RdfInterface

- PHP
Published by zozlak about 2 years ago

acdh-oeaw/acdh-repo - Ported from EasyRdf to RdfInterface

- PHP
Published by zozlak about 2 years ago

acdh-oeaw/acdh-repo - Fix Access-Control-Allow-Headers CORS header value

- PHP
Published by zozlak over 2 years ago

acdh-oeaw/acdh-repo - AMQP handlers API reworked and FTS search tuning

Backward-incompatible changes

  • /search REST API endpoint - full text search highlighting parameter reworked:
    • All fts* parameters are now arrays (e.g. ftsQuery[], ftsStartSel[], etc.). This allows to specify multiple full text search highlighting queries with their own configuration.
      • The ftsProperty[] parameter can be double-nested just like the property[] and value[] once.
      • It's worth noting that because of the way PHP supports array request parameters and the internal implementation of the arche-lib, passing non-array parameters (e.g. ftsQuery, ftsProperty, etc.) still works.
    • Each highlighted result is now described by three technical triples in the RDF output ({N} goes from 1 to the number of higlighted results): <{resourceId}> <{repoCfg}$.schema.searchFts{N}> "higlighted result" <{resourceId}> <{repoCfg}$.schema.searchFtsProperty{N}> <RDFpropertyOfTheCorrespondingHihglightedResult> <{resourceId}> <{repoCfg}$.schema.searchFtsQuery{N}> <FTShighlightQueryOfTheCorrespondingHihglightedResult>
    • As a result two new mappings have to be defined in the config.yaml schema section: the searchFtsProperty and the searchFtsQuery
  • The AMQP handlers can now indicate error and error message (like PHP ones already could) but this required changing the format of the AMQP message returned by the handler. Now it should be a JSON object with at least status property. status: 0 indicates successful handler run while any other status is interpreted as an error. In such a case the status property value is used as the HTTP response status code and an optional message property is put as the response body (if not provided, a generic error message is used). Is status equals 0 and the event was triggered by a repository resource (and not a transaction), then the JSON object returned by the handler should also contain the metadata property with resource final metadata serialized as application/n-triples

Other changes

  • If the number of triples in the metadata output fits the metadata output cache size (the rest.outputTriplesCache configuration property), the triples are sorted by subject and predicate to produce a nicer output (particularly important for the text/turtle serialization).

- PHP
Published by zozlak over 2 years ago

acdh-oeaw/acdh-repo - 3.8.0

  • Metadata output rewritten to the quick-rdf-io library providing much better performance and lower memory usage for RDF/XML and JSON-LD formats (see the test results here).
  • Fixed the REST API response when a non-existing transaction id is supplied by a request.

- PHP
Published by zozlak over 2 years ago

acdh-oeaw/acdh-repo - Minor enhancements

The acdhOeaw\arche\core\MetadataGui generates HTML output with triple subjects being links.

- PHP
Published by zozlak almost 3 years ago

acdh-oeaw/acdh-repo - Bugfixes

acdhOeaw\arche\lib\util\Spatial::fromGeojson() honors the CRS specified in the geoJSON file.

- PHP
Published by zozlak almost 3 years ago

acdh-oeaw/acdh-repo - Bugfixes

- PHP
Published by zozlak almost 3 years ago

acdh-oeaw/acdh-repo - BinaryPayload bugfixes

- PHP
Published by zozlak almost 3 years ago

acdh-oeaw/acdh-repo - Bugfixes

Important bugfixes in transaction rollback handling

Until now PUT {apiBase}/{resourceId} requests were not rolled back on a transaction rollback. This is fixed now.

- PHP
Published by zozlak about 3 years ago

acdh-oeaw/acdh-repo - Bugfixes

/merge endpoint fixes:

  • drop the internal identifier of the merged resource instead of passing it to the merged-into resource
  • fix the way merged resource is deleted so it can be restored on the transaction rollback

- PHP
Published by zozlak about 3 years ago

acdh-oeaw/acdh-repo - Harden spatial indexing against data with BOM

- PHP
Published by zozlak about 3 years ago

acdh-oeaw/acdh-repo - Tuning

Search::post() - accept any property[]/value[]/language[]/type[] keys (instead of enforcing continues integer, starttin-with-0 keys).

- PHP
Published by zozlak about 3 years ago

acdh-oeaw/acdh-repo - Add support for collations in search ordering

  • The /search REST API endpoint allows to specify a desired ordering collation with the orderByCollation parameter.
  • The /describe REST API endpoint reports the default and available collations.

- PHP
Published by zozlak about 3 years ago

acdh-oeaw/acdh-repo - Bugfixes

PUT /transaction rolls it back on error.

- PHP
Published by zozlak over 3 years ago

acdh-oeaw/acdh-repo - get and getMetadata handlers introduced

- PHP
Published by zozlak over 3 years ago

acdh-oeaw/acdh-repo - Bugfixes

Fix handling or resourceProperties and relativesProperties:

  • honor these parameters no matter if they are passed trough request query, body or headers
  • honor them by all endpoints returning metadata and not only the /search one

- PHP
Published by zozlak over 3 years ago

acdh-oeaw/acdh-repo - Bugfixes

Do not expose the GEOM internal literal type in the generated RDF metadata. Use xsd:string instead.

- PHP
Published by zozlak over 3 years ago

acdh-oeaw/acdh-repo - Various improvements

REST API

  • GET /{resourceId} endpoint:
  • GET /{resourceId}/metadata:
    • Supports new resourceProperties[] and relativesProperties[] request parameters allowing to limit the set of returned RDF properties (leading to shorter response times and smaller response size).
    • Supports new generic metadata read mode {childrenDepth}_{parentsDepth}_{fromResource}_{toResource}.
  • GET /search and POST /search):
    • Supports new resourceProperties[] and relativesProperties[] request parameters allowing to limit the set of returned RDF properties (leading to shorter response times and smaller response size).
    • Supports new generic metadata read mode {childrenDepth}_{parentsDepth}_{fromResource}_{toResource}.
    • Honors metadata property value type while ordering search results (e.g. 10^^xsd:int is now greater then 3^^xsd:int).
    • Reports metadata values used for ordering in dedicated artificially created triples.
  • GET /describe endpoint emits the ETag and the Last-Modified headers

Database-embedded functions:

  • Declare get_relatives_metadata(), get_neighbors_metadata() and get_relatives() as STABLE PARALLEL SAFE allowing their in-lining in the execution plan and parallel execution plans usage.

Other changes:

  • TransactionDaemon orchestration in tests tuned to avoid hanging CI runs.

- PHP
Published by zozlak over 3 years ago

acdh-oeaw/acdh-repo - Bugfixes

Honor requested RDF output format in search

- PHP
Published by zozlak about 4 years ago

acdh-oeaw/acdh-repo - Bugfixes

Redirection of non-binary resource GET/HEAD to its metadata GET/HEAD fixed.

- PHP
Published by zozlak about 4 years ago

acdh-oeaw/acdh-repo - Bugfixes

Fixed an issue with an error being thrown after the response has been already send back to the client.

- PHP
Published by zozlak about 4 years ago

acdh-oeaw/acdh-repo - Indicate matching resources order in search responses

If the ordered search is requested, the response contains <resource> $.schema.searchOrder orderNumber triples allowing to reliably sort search results on the client side.

This release requires a new yaml config property $.schema.searchOrder to be defined.

- PHP
Published by zozlak about 4 years ago

acdh-oeaw/acdh-repo - Include identifiers in the full text search

- PHP
Published by zozlak about 4 years ago

acdh-oeaw/acdh-repo - Maintenance release

arche-lib version fixed at >=4 <4.1 to avoid problems with database changes in arche-core 3.3 coupled with arche-lib 4.1

- PHP
Published by zozlak about 4 years ago

acdh-oeaw/acdh-repo - Bugfixes

Required PHP version constraint fixed in composer.json

- PHP
Published by zozlak about 4 years ago

acdh-oeaw/acdh-repo - Better conflict handling

Map lock timeout on final acdhOeaw\arche\core\RestController::$pdo->commit() in acdhOeaw\arche\core\RestController::handleRequest() to HTTP 409 error (instead of reporting HTTP 500).

- PHP
Published by zozlak about 4 years ago

acdh-oeaw/acdh-repo - Allow create handlers to modify identifiers

Never skip id updates in acdhOeaw\arche\core\Metadata::save() as handlers can modify identifiers.

- PHP
Published by zozlak about 4 years ago

acdh-oeaw/acdh-repo - 3.2.0

TODO

- PHP
Published by zozlak about 4 years ago

acdh-oeaw/acdh-repo - API fixes

  • A user can only update his/her password trough the /user REST API to avoid privilege escalation.
  • /resource API can now return HTTP 409 when a conflict with another request being served in parallel is detected (instead of causing a deadlock).
    • This change introduced two new configuration properties $.transactionController.lockTimeout and $.transactionController->statementTimeout (see the config-sample.yaml) but the repository will work with the old config as well because default values are provided (1 second and 60 seconds, respectively).

- PHP
Published by zozlak over 4 years ago

acdh-oeaw/acdh-repo - Bugfixes

Fix a bug in handling DELETE /{resource} causing identifiers of deleted resources not to be returned if no delete handler was registered.

- PHP
Published by zozlak over 4 years ago

acdh-oeaw/acdh-repo - Bugfixes

DELETE /{resourceId} fixes

- PHP
Published by zozlak over 4 years ago

acdh-oeaw/acdh-repo - 3.0.0

REST API changes:

- PHP
Published by zozlak over 4 years ago

acdh-oeaw/acdh-repo - Better logging

- PHP
Published by zozlak over 4 years ago

acdh-oeaw/acdh-repo - Bugfixes

Make logging compliant with psr/log 3.0 and log only strings.

A recent psr/log release introduced PHP 8 type hints. As a consequence if a psr/log interface is used, only strings can be logged (even if particular psr/log implementation accepts broader set of log message types).

- PHP
Published by zozlak over 4 years ago

acdh-oeaw/acdh-repo - Bugfixes

Made compatible both with guzzle/psr7 v1 and v2

- PHP
Published by zozlak over 4 years ago

acdh-oeaw/acdh-repo - 2.0.0

REST API changes:

  • Users management API added.
  • Arche-lib bumped to 2.0.0 introducing new metadata read modes (relativesOnly, relativesReverse, parentsOnly and parentsReverse)

New features:

  • Spatial data indexing and search added.
  • New metadata read modes (relativesOnly, relativesReverse, parentsOnly and parentsReverse)

Backward-incompatible code API changes:

  • Base namespace changed from acdhOeaw\acdhRepo to acdhOeaw\arche\core
  • PHP 8 required.

Internal changes:

  • Lots of fixes to satisfy phpstan level 8 checks (hopefully fixing a few undiscovered bugs).

- PHP
Published by zozlak over 4 years ago

acdh-oeaw/acdh-repo - Mark automatically created resources with the transaction id

When the acdhOeaw\acdhRepo\Metadata::autoAddId() method creates a resource, it now marks it as belonging to the current transaction.

- PHP
Published by zozlak almost 5 years ago

acdh-oeaw/acdh-repo - full_text_search table structure changed

  • mid foreign key to the metadata table introduced
  • property and lang columns dropped
  • not null constraint on id column dropped - now only binary content FTS has it's value while metadata-bord FTS rows have the mid column value.

- PHP
Published by zozlak almost 5 years ago

acdh-oeaw/acdh-repo - ACL tuned for HEAD/GET binary on metadata-only resources

- PHP
Published by zozlak almost 5 years ago

acdh-oeaw/acdh-repo - Bugfixes

  • An embarrassing error in n-triples serialization fixed.
  • The MetadataReadOnly turtle serializer creates prefixes only if they are used more than once now.

- PHP
Published by zozlak almost 5 years ago

acdh-oeaw/acdh-repo - Minor improvements

HEAD on {resId}/metadata endpoint avoids fetching data which makes it a lot faster.

- PHP
Published by zozlak almost 5 years ago

acdh-oeaw/acdh-repo - Metadata serialization enhancement

Metadata serialization in application/n-triples and text/turtle format has been optimized.

Detailed results can be found on https://acdh-oeaw.github.io/arche-docs/aux/metadataapiperformance.html

- PHP
Published by zozlak almost 5 years ago

acdh-oeaw/acdh-repo - Full Text Search fixes and enhancements

  • FTS data derived from metadata are now automatically synced using database triggers. This assures FTS data are always in sync with the metadata.
  • FTS data now store also the language (of course only for FTS data coming from metadata as binary payloads don't have language).

Be aware this release requires a database schema update. Restore privileges on table full_text_search when needed after the migration.

- PHP
Published by zozlak about 5 years ago

acdh-oeaw/acdh-repo - API enhancement

  • readMode, writeMode, parentProperty and transactionId query parameters introduced which can be used instead of corresponding HTTP headers specifying metadata read/write mode, parent property (for metadata fetch) and transaction identifier. This simplifies API usage in environments where it's difficult to specify request's HTTP headers (e.g. in a browser, or in XLST).
  • relatives metadata read mode now includes also resources referenced by the requested resource and all its relatives.

- PHP
Published by zozlak about 5 years ago

acdh-oeaw/acdh-repo - Bugfixes

A fix in db_schema.sql fixing the delete_collection() procedure creation.

- PHP
Published by zozlak about 5 years ago

acdh-oeaw/acdh-repo - CORS support and utility scripts

  • Added support for emitting CORS headers.
  • Added checkHashes.php and cleanupStorage.php utility scripts.

- PHP
Published by zozlak over 5 years ago

acdh-oeaw/acdh-repo - bugfixes

Resolves #11

- PHP
Published by zozlak over 5 years ago

acdh-oeaw/acdh-repo - Bugfixes

Bump easyrdf library version to 0.14.3 to avoid problems with BC dates.

- PHP
Published by zozlak over 5 years ago

acdh-oeaw/acdh-repo - bugfixes

backup.php fixed (sql dump is now dumped where tar expects it and tar failures are properly recognized)

- PHP
Published by zozlak over 5 years ago

acdh-oeaw/acdh-repo - bugfixes

Search fixed for dates before 4713 BC

- PHP
Published by zozlak over 5 years ago

acdh-oeaw/acdh-repo - bugfixes

Basic fix for dates before 4713 BC fixed.

As they can't be stored as the Postgresql's timestamp the value_t field is filled with null.

To allow correct comparisons during search, all dates now fill the value_n field with year extracted from the date.

TODO - search for dates should make use of the value_n when value_t is null.

- PHP
Published by zozlak over 5 years ago

acdh-oeaw/acdh-repo - Bugfixes

An important bugfix

There was a serious bug in the transaction handling making it unable to rollback transactions with loops in the metadata.

The reason was relations.target_id foreign key not being deferrable combined with TransactionController::rollbackTransaction() not using transactions on the current database state handler.

This release addresses this issue.

- PHP
Published by zozlak over 5 years ago

acdh-oeaw/acdh-repo - Minor API improvements

  • Resource creation response contains created resource metadata (instead of an empty HTTP 201 Created).
  • GET on a resource without a binary payload redirects to resource's metadata endpoint (instead of returning HTTP 204 No Content).

- PHP
Published by zozlak over 5 years ago

acdh-oeaw/acdh-repo - Bugfixes

Backup.php bugfixes

- PHP
Published by zozlak over 5 years ago

acdh-oeaw/acdh-repo - CI and deployment fixes

  • More flexible tests config.yaml provided
  • Docker runtime environment for running tests added

- PHP
Published by zozlak over 5 years ago

acdh-oeaw/acdh-repo - /describe endpoint added

A new REST API endpoint added providing the repository configuration YAML to be used by clients (e.g. by the arche-lib)

- PHP
Published by zozlak over 5 years ago

acdh-oeaw/acdh-repo - Do not cast RDF string literals to numeric and date data types

Until now the repository automatically casted RDF string literals being proper numbers to xsd:decimal and ones being proper dates/date times to xsd:dateTime. As it turns out it can be undesired behaviour, this feature has been removed.

- PHP
Published by zozlak almost 6 years ago

acdh-oeaw/acdh-repo - Bugfixes

HEAD on a resource endpoint should return 204 if the GET on the same endpoint would return 204

- PHP
Published by zozlak almost 6 years ago

acdh-oeaw/acdh-repo - Fixes around BC dates

- PHP
Published by zozlak almost 6 years ago

acdh-oeaw/acdh-repo - Minor changes in logging

- PHP
Published by zozlak almost 6 years ago

acdh-oeaw/acdh-repo - Support for BC dates in ISO 8601 added.

- PHP
Published by zozlak almost 6 years ago

acdh-oeaw/acdh-repo - Bugfixes

Fix get_* embedded database functions to return active resources only

- PHP
Published by zozlak almost 6 years ago

acdh-oeaw/acdh-repo - Bugfixes

- PHP
Published by zozlak almost 6 years ago

acdh-oeaw/acdh-repo - GET metadata allows to specify the return format using the query parameter

The change is aimed at clients which don't allow user to set the HTTP Accept header easily, e.g. web browsers.

- PHP
Published by zozlak almost 6 years ago

acdh-oeaw/acdh-repo - Bugfixes

Always emit HTTP 404 for DELETE requests on deleted resources.

- PHP
Published by zozlak almost 6 years ago

acdh-oeaw/acdh-repo - Backup script made self-contained

The backup.php script doesn't depend on any external libraries now, therefore can be safely called from any location without providing the --vendorDir parameter (which has been removed as it's no longer needed).

- PHP
Published by zozlak almost 6 years ago

acdh-oeaw/acdh-repo - Backup script added

backup.php script added for making atomic repository dumps.

- PHP
Published by zozlak almost 6 years ago