LinkedPipes DCAT-AP Viewer

This is a native DCAT-AP 2.0.1 catalog viewer. It is originally developed for OpenData.cz projects, however, it is completely open-source and resuable. It addresses the most painful disadvantages of CKAN when it comes to representing DCAT compatible data such as - native DCAT-AP 2.0.1 input - handles large numbers of distributions of a single dataset - handles licenses attached to distributions - supports EU Vocabularies codelists

Requirements

• Node.js & npm
• OpenJDK for Apache Solr, if not dockerized
• Apache Solr 8.7 (Docker)
• Apache CouchDB 2.3 and up (Docker)

Requirements for data preparation

• LinkedPipes ETL, the preparation pipeline and the codelist pipeline

Installation

Install Solr or run in it Docker. It will contain the search index. For installation, you may proceed like this: - Download Apache Solr - e.g. solr-8.7.0.tgz - Extract the service installer tar xzf solr-8.7.0.tgz solr-8.7.0/bin/install_solr_service.sh --strip-components=2 - Run the service installer sudo bash ./install_solr_service.sh solr-8.7.0.tgz

Next, configure Solr like this: - Create Solr core sudo -u solr /opt/solr/bin/solr create -c dcat-ap-viewer

• Add following to dcat-ap-viewer/conf/solrconfig.xml file (adjust the version numbers as necessary) and restart Solr. <lib path="${solr.install.dir:../../../..}/dist/solr-analysis-extras-8.7.0.jar" /> <lib path="${solr.install.dir:../../../..}/contrib/analysis-extras/lib/icu4j-62.1.jar" /> <lib path="${solr.install.dir:../../../..}/contrib/analysis-extras/lucene-libs/lucene-analyzers-icu-8.7.0.jar" />

• Prepare Solr schema (change locale to your language, the example bellow is for cs and en): ``` curl http://localhost:8983/solr/dcat-ap-viewer/schema -X POST -H 'Content-type:application/json' --data-binary '{ "add-field-type" : { "name" : "asciistring", "class" : "solr.TextField", "positionIncrementGap" : "100", "analyzer" : { "tokenizer" : { "class" : "solr.WhitespaceTokenizerFactory" }, "filters" : [ { "class" : "solr.LowerCaseFilterFactory" }, { "class" : "solr.ASCIIFoldingFilterFactory", "preserveOriginal" : "false" } ] } }, "add-field-type" : { "name" : "stringicucs", "class" : "org.apache.solr.schema.ICUCollationField", "locale": "cs", "strength" : "primary" },
  "add-field-type" : { "name" : "stringicuen", "class" : "org.apache.solr.schema.ICUCollationField", "locale": "en", "strength" : "primary" }, "add-field" : { "name" : "iri", "type" : "string" , "indexed" : false, "multiValued" : false }, "add-field" : { "name" : "modified", "type" : "pdate", "multiValued" : false}, "add-field" : { "name" : "issued", "type" : "pdate", "multiValued" : false }, "add-field" : { "name" : "accrualperiodicity", "type" : "string", "multiValued" : false }, "add-field" : { "name" : "publisher", "type" : "string" , "indexed" : true, "multiValued" : false }, "add-field" : { "name" : "filetype", "type" : "strings", "indexed" : true }, "add-field" : { "name" : "license", "type" : "strings", "indexed" : true }, "add-field" : { "name" : "theme", "type" : "strings" }, "add-field" : { "name" : "temporalstart", "type" : "pdate", "docValues" : true, "multiValued" : false }, "add-field" : { "name" : "temporalend", "type" : "pdate", "docValues" : true, "multiValued" : false }, "add-field" : { "name" : "spatial", "type" : "strings" }, "add-field" : { "name" : "ispartof", "type" : "string" , "indexed" : true, "multiValued" : false }, "add-field" : { "name" : "dataservice_type", "type" : "strings", "indexed" : true },

  "add-field" : { "name" : "descriptioncs", "type" : "string", "multiValued" : false }, "add-field" : { "name" : "keywordcs", "type" : "strings" }, "add-field" : { "name" : "titlecs", "type" : "string", "multiValued" : false }, "add-field" : { "name" : "titlecssort", "type" : "stringicucs", "docValues" : true, "multiValued" : false }, "add-field" : { "name" : "titlecsquery", "type" : "asciistring", "multiValued" : false }, "add-copy-field" : { "source" : "titlecssort", "dest" : "titlecsquery" },

  "add-field" : { "name" : "descriptionen", "type" : "string", "multiValued" : false }, "add-field" : { "name" : "keyworden", "type" : "strings" }, "add-field" : { "name" : "titleen", "type" : "string", "multiValued" : false }, "add-field" : { "name" : "titleensort", "type" : "stringicuen", "docValues" : true, "multiValued" : false }, "add-field" : { "name" : "titleenquery", "type" : "asciistring", "multiValued" : false }, "add-copy-field" : { "source" : "titleensort", "dest" : "titleenquery" },

  "add-field" : { "name" : "vdfcodelist", "type" : "boolean", "multiValued" : false }, "add-field" : { "name" : "vdfpublic_data", "type" : "boolean", "multiValued" : false },

  "replace-field" : { "name": "text", "type" : "asciistring", "multiValued" : true, "indexed" : true, "stored" : false }, "add-copy-field" : { "source" : "titlecs", "dest" : "text" }, "add-copy-field" : { "source" : "titleen", "dest" : "text" }, "add-copy-field" : { "source" : "descriptioncs", "dest" : "text" }, "add-copy-field" : { "source" : "descriptionen", "dest" : "text" }, "add-copy-field" : { "source" : "keywordcs", "dest" : "text" }, "add-copy-field" : { "source" : "keyworden", "dest" : "text_" }
  }' ```

• And then: curl http://localhost:8983/solr/dcat-ap-viewer/config -H 'Content-type:application/json' -d'{ "set-property" : {"requestDispatcher.requestParsers.enableRemoteStreaming":true}, "set-property" : {"requestDispatcher.requestParsers.enableStreamBody":true} }'

Install Apache CouchDB, e.g. via a package manager, or run it in Docker. It will contain the datasets, distributions and, optionally, the code list labels.

Install LinkedPipes DCAT-AP viewer - Clone the repository - Install dependencies: npm install - Copy and edit the configuration file: cp configuration-example.yaml configuration.yaml Please refer to the profile of your choice, client/profile-lkod or client/profile-nkod , for more information about the configuration.

Load the data - Install LinkedPipes ETL and import, configure and run the preparation pipeline. It has the DCAT-AP catalog (its RDF representation) on the input, this is where you provide your DCAT-AP dump. The pipeline assumes that Solr is running on localhost:8983 and CouchDB is running on localhost:5984. - To enable support for the European Metadata Registry Named Authority Lists (EU MDR NALs) linked from DCAT-AP records, configure and run the codelist pipeline. It will populate the Apache CouchDB instance with the codelists.

Running DCAT-AP Viewer

Before the first run the DCAT-AP Viewer client javascript needs to be compiled using command: npm run build After the compilation is done the following command can be used to start the server: npm run start