GeoRDFBench Framework: Geospatial Semantic Benchmarking Simplified

GeoRDFBench can be used by researchers and practitioners interested in Semantic Benchmarking with SPARQL language. The framework however provides unique features for the geospatial aspects of RDF stores that support the GeoSPARQL standard. Documentation for the project is maintained at our GeoRDFBench web site.

For a quick acquaintance with the platform, please run the dockerized example at dockerized example which exhibits running a GeoSPARQL experiment with the RDF4J RDF store against the Scalability 10K dataset of the Geographica 2 benchmark.

The GeoRDFBench Framework Samples is a maven project using GeoRDFBench's runtime as dependence and showcases the use of the JSON Generator's API for constructing the workload specification for the LUBM(1, 0) benchmark.

The GeoRDFBench web site also provides the Multiple stores against LUBM(1, 0) Workload dockerized example showcasing the use of GeoRDFBench to run experiments with RDF4J, GraphDB and JenaGeoSPARQL against LUBM(1, 0) workload.

News

There is a new JSON Specification Library Endpoint module. Its main features are:

• It's a JavaScript module running on Node.js JS runtime environment and takes advantage of the Express web framework.
• It allows the deployment of the JSON Specification Library to a remote server, different from the experiments server, therefore promoting a more distributed approach to the GeoRDFBench Framework's operation.
• The user can choose the JSON Specification Library location which holds the json_defs structure, the endpoint's listening port and configure which CRUD operations are enabled for each specification category. Currently the implemented operations are: (C)reate, (R)ead and (D)elete.
• The experiment JSON specification arguments can now also be REST API URLs from the endpoint instead of file paths. An example can be found in the RunRDF4JExperimentRemoteSpecsTest test class of the RDF4JSUT module.
• The user can find details for the API in the Swagger (OpenAPI Specification) documentation
• The endpoint implements a REST API which currently allows GET operations, such as:
  • Get a list of available Specification categories
  • Get the Endpoint's configuration
  • Get a list of available Datasets
  • Get the JSON spec for dataset LUBM10DSoriginal
  • Get a list of available Querysets
  • Get the JSON spec for queryset scalabilityFuncQSoriginal
  • Get a list of available Execution Specs
  • Get the JSON spec for execution specification scalabilityESoriginal
  • ...

Next Steps

• New capabilities:

  • a user interface module to be used as a front-end with the new JSON Specs Endpoint which will further assist the interactive generation and modification of JSON specifications,
  • implement the remaining Update endpoint module's operation on JSON specification libraries,
  • support for the Hadoop file system,
  • a fourth Spark-based framework API, so that Spark-based distributed GeoSPARQL solutions, such as Strabo 2 can be tested, and
  • inference support for RDF modules

• New RDF Modules for SPARQL/GeoSPARQL:

  • Anzo Graph by Cambridge Semantics (now Altair),
  • Neo4J with neosemantics (n10s) plugin

• New Semantic Languages:

  • Graph Query Language (GQL) for running benchmarks on native LPGs

Sustainability plan and limitations

A full list of issues can be found here. Their status can be followed up in a GitHub Project.

GeoRDFBench's runtime was created to support:

• RDF store integration: fast integration of new or updated RDF stores by incorporating the common logic of working with RDF repositories through any of the well-known RDF Java Frameworks: OpenRDF Sesame, Eclipse RDF4J, Apache Jena.
• Benchmark specification Ser/Der: easy construction of GeoSPARQL/SPARQL benchmark specifications and even easier serialization/deserialization of these JSON specification files, increasing the reusability of these specifications and equally important improving the repeatability of experiments and confidence on experiment results.

Current limitations include the non automatic serialization/deserialization of the benchmark environment (host, os, RDF store components) specifications and the lack of support for distributed RDF stores.