https://github.com/grycap/oscar-worker

An alternative to OpenFaaS nats-queue-worker for long-running functions

https://github.com/grycap/oscar-worker

Science Score: 26.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • CITATION.cff file
  • codemeta.json file
    Found codemeta.json file
  • .zenodo.json file
    Found .zenodo.json file
  • DOI references
  • Academic publication links
  • Academic email domains
  • Institutional organization owner
  • JOSS paper metadata
  • Scientific vocabulary similarity
    Low similarity (10.0%) to scientific vocabulary

Keywords

kubernetes nats openfaas
Last synced: 4 months ago · JSON representation

Repository

An alternative to OpenFaaS nats-queue-worker for long-running functions

Basic Info
  • Host: GitHub
  • Owner: grycap
  • License: apache-2.0
  • Language: Python
  • Default Branch: master
  • Homepage:
  • Size: 73.2 KB
Statistics
  • Stars: 11
  • Watchers: 7
  • Forks: 5
  • Open Issues: 4
  • Releases: 7
Topics
kubernetes nats openfaas
Created over 7 years ago · Last pushed about 3 years ago
Metadata Files
Readme License

README.md

OSCAR Worker

Docker Build Status License

OSCAR Worker is the default queue worker for OpenFaaS in the OSCAR framework. It enables launching long-running functions as Kubernetes Jobs when /async-function/ path is used to make requests.

The goal is to ensure that each invocation has the specified resources and, furthermore, that functions can be executed in parallel depending on the resources available in the cluster.

openfaas async worker

Configuration

You can configure the worker through environment variables. To modify the default values you can edit the oscar-worker-dep.yaml file:

yaml ... env: # Token to access the k8s API server (if not set reads the content of '/var/run/secrets/kubernetes.io/serviceaccount/token') # - name: KUBE_TOKEN # value: "xxx" - name: KUBERNETES_SERVICE_HOST value: "kubernetes.default" - name: KUBERNETES_SERVICE_PORT value: "443" - name: NATS_ADDRESS value: "nats.openfaas" - name: NATS_PORT value: "4222" - name: JOB_TTL_SECONDS_AFTER_FINISHED value: 60 - name: JOB_BACKOFF_LIMIT value: 3 ...

Deployment

In order to deploy the OSCAR Worker you need to have already installed OpenFaaS in the Kubernetes cluster. Then, delete the nats-queue-worker deployment:

bash kubectl delete deploy queue-worker -n openfaas

And create the required namespaces, RBAC and deployment:

bash kubectl apply -f yaml/oscar-worker-namespaces.yaml kubectl apply -f yaml/oscar-worker-rbac.yaml kubectl apply -f yaml/oscar-worker-dep.yaml

Secrets

If your OpenFaaS function have secrets defined, you must duplicate them to the oscar-fn namespace for granting access to jobs:

bash kubectl get secret <SECRET_NAME> -n openfaas-fn -o yaml \ | sed s/"namespace: openfaas-fn"/"namespace: oscar-fn"/\ | kubectl apply -n oscar-fn -f -

Logs

If you want to inspect worker's logs run:

bash kubectl logs deploy/oscar-worker -n oscar

To see specific function invocation logs, first get all pods of the oscar-fn namespace and then query the one you want:

bash kubectl get pods -n oscar-fn kubectl logs POD_NAME -n oscar-fn

Clear completed Jobs

Completed Jobs can be automatically deleted after finishing by enabling the TTLAfterFinished feature gate of Kubernetes versions >= v1.12. TTL Seconds to clean up Jobs can be configured through the JOB_TTL_SECONDS_AFTER_FINISHED environment variable of the worker.

To delete completed jobs manually, execute:

bash kubectl delete job $(kubectl get job -o=jsonpath='{.items[?(@.status.succeeded==1)].metadata.name}' -n oscar-fn) -n oscar-fn

Owner

  • Name: GRyCAP
  • Login: grycap
  • Kind: organization

Grid y Computación de Altas Prestaciones

GitHub Events

Total
Last Year

Issues and Pull Requests

Last synced: 10 months ago

All Time
  • Total issues: 6
  • Total pull requests: 14
  • Average time to close issues: about 2 hours
  • Average time to close pull requests: about 2 hours
  • Total issue authors: 4
  • Total pull request authors: 3
  • Average comments per issue: 2.5
  • Average comments per pull request: 0.0
  • Merged pull requests: 13
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 0
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
Top Authors
Issue Authors
  • alexellis (2)
  • gmolto (1)
  • srisco (1)
  • seb-835 (1)
Pull Request Authors
  • srisco (12)
  • alpegon (1)
  • TrellixVulnTeam (1)
Top Labels
Issue Labels
bug (1)
Pull Request Labels

Dependencies

Dockerfile docker
  • bitnami/minideb stretch build
requirements.txt pypi
  • asyncio-nats-streaming *
  • packaging *
  • requests *