Data

Tools

Indexes

Applications

Experiments

Open Source Science

indexing-service

Service for indexing metadata via RabbitMQ

https://github.com/kit-data-manager/indexing-service

Science Score: 62.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ✓
    CITATION.cff file
    Found CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ✓
    Committers with academic emails
    4 of 11 committers (36.4%) from academic institutions
  • ✓
    Institutional organization owner
    Organization kit-data-manager has institutional domain (www.scc.kit.edu)
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (12.5%) to scientific vocabulary

Keywords

restful-api

Keywords from Contributors

interpretability hack meshing standardization robust bridges gym-environments parallel optim dynamics
Last synced: 6 months ago · JSON representation ·

Repository

Service for indexing metadata via RabbitMQ

Basic Info
  • Host: GitHub
  • Owner: kit-data-manager
  • License: apache-2.0
  • Language: Java
  • Default Branch: main
  • Homepage:
  • Size: 940 KB
Statistics
  • Stars: 0
  • Watchers: 2
  • Forks: 1
  • Open Issues: 12
  • Releases: 9
Topics
restful-api
Created over 5 years ago · Last pushed 6 months ago
Metadata Files
Readme Changelog License Citation

README.md

Indexing-service

Build Status codecov CodeQL License ![docker] ![currentVersion] size

[docker]: [currentVersion]:

:information_source: The service should not be directly accessible from the outside.

:warning: For mapping documents only Gemma is available currently!

A standalone service which receives messages about changes in metadata documents maps them to a common format and ingests them into elasticsearch.

Visualization of use case structure.

How to build

Dependencies that are needed to build and are not being downloaded via gradle:

  • OpenJDK 17 or higher $ bash build.sh PATH/TO/INSTALLATION/DIR ## How to start $ PATH/TO/INSTALLATION/DIR/run.sh

Prerequisites

You might want to take a look at testbed4inf, which should make it easy to satisfy those.

  • Python3
    • PIP
    • xmltodict
    • wget
  • a running RabbitMQ instance
  • a running elasticsearch instance

Setup using Docker

Install Python for Gemma

sudo apt-get install --assume-yes python3 python3-pip pip3 install xmltodict wget

Install and Start Network

docker network create network4datamanager

Install and Start Elasticsearch

docker pull elasticsearch:7.9.3 docker run -d --name elasticsearch4metastore -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" elasticsearch:7.9.3

Install and Start RabbitMQ

docker run -d --hostname rabbitmq --net network4datamanager --name rabbitmq4docker -p 5672:5672 -p 15672:15672 rabbitmq:3-management

Managing Services

To start/stop a single service just type docker start/stop name_of_container e.g.: docker stop elasticsearch4metastore

More information

License

See LICENSE file in this repository.

Owner

  • Name: KIT Data Manager
  • Login: kit-data-manager
  • Kind: organization
  • Email: webmaster@datamanger.kit.edu
  • Location: Karlsruhe, Germany

Generic software and recommendations for FAIR research data management by DEM.

Citation (CITATION.cff) 

cff-version: 1.2.0
message: "If you use this software, please cite it as below."
authors:
- family-names: "Hartmann"
  given-names: "Volker"
  affiliation: Karlsruhe Institute of Technology
  email: volker.hartmann@kit.edu
  orcid: "https://orcid.org/0000-0001-6383-5214"
- family-names: "Jejkal"
  given-names: "Thomas"
  affiliation: Karlsruhe Institute of Technology
  email: thomas.jejkal@kit.edu
  orcid: "https://orcid.org/0000-0003-2804-688X"
- family-names: "Pfeil"
  given-names: "Andreas"
  affiliation: Karlsruhe Institute of Technology
  email: andreas.pfeil@kit.edu
  orcid: "https://orcid.org/0000-0001-6575-1022"
title: "Indexing and Mapping Service"
type: software
abstract: Indexing and Mapping Service is a software for (optional) mapping and indexing metadata documents for (elastic)search.
version: 1.0.3
date-released: 2024-11-25
url: "https://github.com/kit-data-manager/indexing-service"
repository-code:  "https://github.com/kit-data-manager/indexing-service"
license: Apache-2.0

GitHub Events

Total
  • Create event: 33
  • Release event: 2
  • Issues event: 2
  • Delete event: 38
  • Issue comment event: 9
  • Push event: 97
  • Pull request event: 66
Last Year
  • Create event: 33
  • Release event: 2
  • Issues event: 2
  • Delete event: 38
  • Issue comment event: 9
  • Push event: 97
  • Pull request event: 66

Committers

Last synced: 10 months ago

All Time
  • Total Commits: 470
  • Total Committers: 11
  • Avg Commits per committer: 42.727
  • Development Distribution Score (DDS): 0.468
Past Year
  • Commits: 72
  • Committers: 5
  • Avg Commits per committer: 14.4
  • Development Distribution Score (DDS): 0.444
Top Committers
Name Email Commits
Volker Hartmann v****n@k****u 250
dependabot[bot] 4****] 160
renovate[bot] 2****] 16
Andreas Pfeil p****l@A****x 14
Andreas Pfeil v****t@A****x 11
Andreas Pfeil p****l@d****u 10
andreas.pfeil a****l@k****u 2
Thomas Jejkal t****l@k****u 2
Andreas Pfeil v****t@A****l 2
Andreas Pfeil p****l@A****l 2
github-actions[bot] 4****] 1
Committer Domains (Top 20 + Academic)
kit.edu: 3 dyn-dhcp-141-52-74-39.scc.kit.edu: 1 andreass-air.fritz.box: 1 andreass-imac.fritz.box: 1

Issues and Pull Requests

Last synced: 10 months ago

All Time
  • Total issues: 11
  • Total pull requests: 192
  • Average time to close issues: 29 days
  • Average time to close pull requests: about 2 months
  • Total issue authors: 2
  • Total pull request authors: 5
  • Average comments per issue: 1.09
  • Average comments per pull request: 1.63
  • Merged pull requests: 146
  • Bot issues: 0
  • Bot pull requests: 157
Past Year
  • Issues: 0
  • Pull requests: 9
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 1
  • Average comments per issue: 0
  • Average comments per pull request: 0.0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 9
View more stats
Top Authors
Issue Authors
  • VolkerHartmann (5)
  • renovate[bot] (4)
  • ghost (1)
Pull Request Authors
  • dependabot[bot] (142)
  • renovate[bot] (34)
  • VolkerHartmann (28)
  • github-actions[bot] (5)
  • ThomasJejkal (1)
  • Pfeil (1)
Top Labels
Issue Labels
dependencies (4) enhancement (2) bug (2)
Pull Request Labels
dependencies (176) java (78) github_actions (10) bug (3) enhancement (2)

Dependencies

build.gradle maven
  • com.github.jknack:handlebars 4.2.0 implementation
  • com.h2database:h2 1.4.200 implementation
  • edu.kit.datamanager:generic-message-consumer 0.2.0 implementation
  • edu.kit.datamanager:service-base 0.2.0 implementation
  • org.apache.tika:tika-core 1.18 implementation
  • org.javers:javers-core 5.3.2 implementation
  • org.javers:javers-spring-boot-starter-sql 3.9.0 implementation
  • org.json:json 20200518 implementation
  • org.postgresql:postgresql 42.2.1 implementation
  • org.springdoc:springdoc-openapi-data-rest 1.4.8 implementation
  • org.springdoc:springdoc-openapi-ui 1.4.8 implementation
  • org.springframework.boot:spring-boot-starter-data-rest * implementation
  • org.springframework.boot:spring-boot-starter-mail * implementation
  • org.apache.httpcomponents:httpclient 4.5.6 runtimeOnly
  • junit:junit 4.13.1 testImplementation
  • org.asciidoctor:asciidoctor-gradle-plugin 1.5.9.2 testImplementation
  • org.mockito:mockito-core 2.18.0 testImplementation
  • org.powermock:powermock-api-mockito2 2.0.0-beta.5 testImplementation
  • org.powermock:powermock-module-junit4 2.0.0-beta.5 testImplementation
  • org.springframework.boot:spring-boot-starter-test * testImplementation
  • org.springframework.restdocs:spring-restdocs-mockmvc * testImplementation
  • org.springframework.restdocs:spring-restdocs-mockmvc 2.0.2.RELEASE testImplementation
  • org.springframework.security:spring-security-test * testImplementation
  • org.springframework:spring-test * testImplementation
.github/workflows/branch_on_issue.yml actions
  • robvanderleek/create-issue-branch main composite
.github/workflows/codeql-analysis.yml actions
  • actions/checkout v2 composite
  • github/codeql-action/analyze v1 composite
  • github/codeql-action/autobuild v1 composite
  • github/codeql-action/init v1 composite
.github/workflows/docker-publish.yml actions
  • actions/checkout v2 composite
  • actions/setup-java v2 composite
  • crazy-max/ghaction-docker-meta v1 composite
  • docker/build-push-action v2 composite
  • docker/login-action v1 composite
  • docker/setup-buildx-action v1 composite
  • docker/setup-qemu-action v1 composite
.github/workflows/gradle.yml actions
  • actions/checkout v2 composite
  • actions/setup-java v2 composite
  • codecov/codecov-action v1 composite
Dockerfile docker
  • build-env-java latest build
  • python 3.10 build