Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/dissco/dissco-core-deployment

DiSSCo core infrastructure. Contains files for deploying and managing the core infrastructure components

https://github.com/dissco/dissco-core-deployment

Science Score: 26.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (8.5%) to scientific vocabulary
Last synced: 4 months ago · JSON representation

Repository

DiSSCo core infrastructure. Contains files for deploying and managing the core infrastructure components

Basic Info
  • Host: GitHub
  • Owner: DiSSCo
  • License: apache-2.0
  • Language: Python
  • Default Branch: main
  • Size: 666 KB
Statistics
  • Stars: 2
  • Watchers: 1
  • Forks: 0
  • Open Issues: 0
  • Releases: 0
Created almost 3 years ago · Last pushed 5 months ago
Metadata Files
Readme License

README.md

dissco-core-deployment

DiSSCo core deployment contains all files necessary for deploying the DiSSCo core applications. This consists of two parts: - Deployment of helm charts - Deployment of DiSSCo resources

The repo is divided into the different environments (development/acceptance/production).

Deployment of helm charts

The deployment of the helm charts in the helm folder (for example acceptance-argocd) is done by the infrastructure pipeline. These helm charts contain operators and application which are needed to run the DiSSCo core infrastructure. The principle of App-of-Apps is used. We bundled the deployment of the ArgoCD deployments into a single helm chart. This helm chart is deployed on the cluster and manages all the underlying helm charts. Any changes can be made in this repo and pushed. When it is merged in the main branch, argoCD will trigger and update the cluster. The helm charts contain: - Elastic stack operator, adds CRDs for the deployment of elastic resources - KEDA, adds CRDs and applications for the management of KEDA resources - Metrics servers, deploys Kubernetes native metrics service, required for things such as kubectl top node - Secret Driver AWS, deploys application for the secret driver for AWS - Secret Manager, deploys application for the management of secrets (needs AWS driver for AWS Secret Store) - RabbitMQ, adds CRDs for rabbitMQ

Deployment of DiSSCo resources

The deployment of the DiSSCo resources happens by deploying the argocd-deployment.yaml. This deployment needs to be done manually. This will trigger ArgoCD to deploy all resource (recursive) in a particular folder. This deployment will include: - Traefik, this deploys the CRD and Ingress, including all security settings. No good Helm chart exists for how we use it, so deployment of resources is necessary - Elasticsearch + Kibana, deploys a HA cluster (3) of Elasticsearch and a Kibana instance - Kafka, deploys a HA kafka cluster (3) together with the required ZooKeepers (3) - Secret Manager, resources needed to inject the AWS Secret Store secrets into Kubernetes - DiSSCo applications

DiSSCo applications

The DiSSCo applications are structured so that they are self-containing packages. Each folder includes everything needed to run the service, this may contain: - Deployment file - Service, for internal connectivity - KEDA files, for automated scaling - IngressRoute, for routing - Kafka topic, for messages

Change management

After all applications are deployed, all changes should be made by creating PR's on the repo and merge them to main. ArgoCD monitors the main branch of the GitHub repo. When new changes are made it will automatically recognise that kubernetes is no longer in sync with the files in the repo. It can then automatically synchronise the cluster with the changed kubernetes files. There is also an option to leave the actual syncrhonisation a manual action.

Kube Green

The installation of kube-green is still manual. Follow the steps on this website: https://kube-green.dev/docs/install/

Owner

  • Name: DiSSCo
  • Login: DiSSCo
  • Kind: organization
  • Email: info@dissco.eu
  • Location: Europe

Distributed System of Scientific Collections - pan-European Research Infrastructure. Updates on DiSSCo and natural science collections

GitHub Events

Total
  • Delete event: 33
  • Push event: 168
  • Pull request review event: 88
  • Pull request review comment event: 55
  • Pull request event: 178
  • Create event: 78
Last Year
  • Delete event: 33
  • Push event: 168
  • Pull request review event: 88
  • Pull request review comment event: 55
  • Pull request event: 178
  • Create event: 78

Issues and Pull Requests

Last synced: 5 months ago

All Time
  • Total issues: 0
  • Total pull requests: 138
  • Average time to close issues: N/A
  • Average time to close pull requests: 4 days
  • Total issue authors: 0
  • Total pull request authors: 4
  • Average comments per issue: 0
  • Average comments per pull request: 0.0
  • Merged pull requests: 106
  • Bot issues: 0
  • Bot pull requests: 2
Past Year
  • Issues: 0
  • Pull requests: 96
  • Average time to close issues: N/A
  • Average time to close pull requests: 2 days
  • Issue authors: 0
  • Pull request authors: 3
  • Average comments per issue: 0
  • Average comments per pull request: 0.0
  • Merged pull requests: 73
  • Bot issues: 0
  • Bot pull requests: 2
View more stats
Top Authors
Issue Authors
Pull Request Authors
  • southeo (94)
  • samleeflang (40)
  • dependabot[bot] (2)
  • TomDijkema (2)
Top Labels
Issue Labels
Pull Request Labels
dependencies (2) python (2)