Data

Tools

Indexes

Applications

Experiments

Open Source Science

fluxoperator

Deploy a Flux MiniCluster to Kubernetes with the operator

https://github.com/flux-framework/flux-operator

Science Score: 67.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ✓
    CITATION.cff file
    Found CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ✓
    DOI references
    Found 4 DOI reference(s) in README
  • ✓
    Academic publication links
    Links to: zenodo.org
  • ○
    Committers with academic emails
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (14.1%) to scientific vocabulary

Keywords

flux flux-framework kubernetes operator
Last synced: 6 months ago · JSON representation ·

Repository

Deploy a Flux MiniCluster to Kubernetes with the operator

Basic Info
Statistics
  • Stars: 35
  • Watchers: 8
  • Forks: 8
  • Open Issues: 23
  • Releases: 4
Topics
flux flux-framework kubernetes operator
Created over 3 years ago · Last pushed 6 months ago
Metadata Files
Readme Changelog License Citation Authors Notice Copyright

README.md

Flux Operator

docs/development/the-operator.jpg DOI

The Flux Operator is a Kubernetes Cluster Operator that you can install to your cluster to create and control a Flux Framework "MiniCluster" to launch jobs to.

Read more, including user and developer guides, and project background in our 💛 Documentation 💛

Resources

Organization

The basic idea is that we present the idea of a MiniCluster that is a custom resource definition (CRD) that defines a job container (that does not need to have Flux) that (when submit) will create a set of config maps, secrets, and the final Indexed Job that has the pod containers running with Flux. Since this is a batchv1.Job, it will have states that we can track.

And you can find the following here:

  • Flux Controllers are under controllers/flux for the MiniCluster
  • API Spec are under api/v1alpha2/ also for MiniCluster
  • Packages include supporting packages for job conditions (state), if we eventually want that.
  • Config includes mostly automatically generated yaml configuration files needed by Kubernetes

And the following external resources might be useful:

  • Flux Framework
  • Flux RESTful API: provides the interface for submitting jobs, if no command provided to the operator.
  • Python SDK: for deploying MiniClusters and port forwarding.
  • Flux HPC Examples containers and CRD for the operator to run Flux with HPC workloads (under development)
  • Flux Cloud: automation of experiments using the Flux Operator

Note we welcome contributions to code or to suggest features or identify bugs!

Citation

You can follow the CITATION.cff (right sidebar in GitHub) to cite, or view the paper directly here A direct (copy paste) citation is the following:

Sochat V, Culquicondor A, Ojea A and Milroy D. The Flux Operator (version 1). F1000Research 2024, 13:203 (https://doi.org/10.12688/f1000research.147989.1)

License

HPCIC DevTools is distributed under the terms of the MIT license. All new contributions must be made under this license.

See LICENSE, COPYRIGHT, and NOTICE for details.

SPDX-License-Identifier: (MIT)

LLNL-CODE- 842614

Owner

  • Name: flux-framework
  • Login: flux-framework
  • Kind: organization

Citation (CITATION.cff) 

cff-version: 1.2.0
message: "If you use this software, please cite it as below."
authors:
  - family-names: Sochat
    given-names: Vanessa
    orcid: https://orcid.org/0000-0002-4387-3819
  - family-names: Culquicondor
    given-names: Aldo
  - family-names: Ojea
    given-names: Antonio
  - family-names: Milroy
    given-names: Daniel
    orcid: https://orcid.org/0000-0001-6500-3227
title: "The Flux Operator"
version: 0.2.1
identifiers:
  - type: doi
    value: 10.12688/f1000research.147989.1
date-released: 2024-03-21

GitHub Events

Total
  • Issues event: 2
  • Watch event: 5
  • Delete event: 11
  • Push event: 34
  • Pull request event: 19
  • Create event: 10
Last Year
  • Issues event: 2
  • Watch event: 5
  • Delete event: 11
  • Push event: 34
  • Pull request event: 19
  • Create event: 10

Committers

Last synced: 9 months ago

All Time
  • Total Commits: 262
  • Total Committers: 4
  • Avg Commits per committer: 65.5
  • Development Distribution Score (DDS): 0.019
Past Year
  • Commits: 15
  • Committers: 1
  • Avg Commits per committer: 15.0
  • Development Distribution Score (DDS): 0.0
Top Committers
Name Email Commits
vsoch v****h 257
Zeke Morton 4****n 2
Antonio Ojea a****a@g****m 2
Jason Kincl j****l@r****m 1
Committer Domains (Top 20 + Academic)
redhat.com: 1 google.com: 1

Issues and Pull Requests

Last synced: 6 months ago

All Time
  • Total issues: 36
  • Total pull requests: 159
  • Average time to close issues: about 2 months
  • Average time to close pull requests: 3 days
  • Total issue authors: 2
  • Total pull request authors: 2
  • Average comments per issue: 2.14
  • Average comments per pull request: 0.12
  • Merged pull requests: 132
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 2
  • Pull requests: 15
  • Average time to close issues: about 2 hours
  • Average time to close pull requests: about 4 hours
  • Issue authors: 1
  • Pull request authors: 1
  • Average comments per issue: 0.0
  • Average comments per pull request: 0.0
  • Merged pull requests: 11
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • vsoch (35)
  • kannon92 (1)
Pull Request Authors
  • vsoch (157)
  • aojea (2)
Top Labels
Issue Labels
bug (1) help wanted (1)
Pull Request Labels

Packages

  • Total packages: 2
  • Total downloads:
    • pypi 36 last-month
  • Total dependent packages: 1
    (may contain duplicates)
  • Total dependent repositories: 0
    (may contain duplicates)
  • Total versions: 27
  • Total maintainers: 1
proxy.golang.org: github.com/flux-framework/flux-operator
  • Versions: 2
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Forks count: 7.8%
Stargazers count: 8.1%
Average: 8.9%
Dependent packages count: 8.9%
Dependent repos count: 10.6%
Last synced: 7 months ago
pypi.org: fluxoperator

Python SDK for the Flux Operator

  • Versions: 25
  • Dependent Packages: 1
  • Dependent Repositories: 0
  • Downloads: 36 Last month
Rankings
Dependent packages count: 6.6%
Downloads: 8.1%
Average: 15.7%
Stargazers count: 16.1%
Forks count: 17.3%
Dependent repos count: 30.6%
Maintainers (1)
vsochat
Last synced: 6 months ago

Dependencies

go.mod go
  • cloud.google.com/go v0.97.0
  • github.com/Azure/go-autorest v14.2.0+incompatible
  • github.com/Azure/go-autorest/autorest v0.11.27
  • github.com/Azure/go-autorest/autorest/adal v0.9.20
  • github.com/Azure/go-autorest/autorest/date v0.3.0
  • github.com/Azure/go-autorest/logger v0.2.1
  • github.com/Azure/go-autorest/tracing v0.6.0
  • github.com/PuerkitoBio/purell v1.1.1
  • github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578
  • github.com/beorn7/perks v1.0.1
  • github.com/cespare/xxhash/v2 v2.1.2
  • github.com/davecgh/go-spew v1.1.1
  • github.com/emicklei/go-restful/v3 v3.8.0
  • github.com/evanphx/json-patch v4.12.0+incompatible
  • github.com/fsnotify/fsnotify v1.5.1
  • github.com/go-logr/logr v1.2.3
  • github.com/go-logr/zapr v1.2.0
  • github.com/go-openapi/jsonpointer v0.19.5
  • github.com/go-openapi/jsonreference v0.19.5
  • github.com/go-openapi/swag v0.19.14
  • github.com/gogo/protobuf v1.3.2
  • github.com/golang-jwt/jwt/v4 v4.2.0
  • github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da
  • github.com/golang/protobuf v1.5.2
  • github.com/google/gnostic v0.5.7-v3refs
  • github.com/google/go-cmp v0.5.6
  • github.com/google/gofuzz v1.1.0
  • github.com/google/uuid v1.1.2
  • github.com/imdario/mergo v0.3.12
  • github.com/josharian/intern v1.0.0
  • github.com/json-iterator/go v1.1.12
  • github.com/mailru/easyjson v0.7.6
  • github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369
  • github.com/mitchellh/hashstructure/v2 v2.0.2
  • github.com/moby/spdystream v0.2.0
  • github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
  • github.com/modern-go/reflect2 v1.0.2
  • github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822
  • github.com/nxadm/tail v1.4.8
  • github.com/onsi/ginkgo v1.16.5
  • github.com/onsi/gomega v1.19.0
  • github.com/pkg/errors v0.9.1
  • github.com/prometheus/client_golang v1.12.1
  • github.com/prometheus/client_model v0.2.0
  • github.com/prometheus/common v0.32.1
  • github.com/prometheus/procfs v0.7.3
  • github.com/spf13/pflag v1.0.5
  • go.uber.org/atomic v1.7.0
  • go.uber.org/multierr v1.6.0
  • go.uber.org/zap v1.19.1
  • golang.org/x/crypto v0.0.0-20220315160706-3147a52a75dd
  • golang.org/x/net v0.0.0-20220722155237-a158d28d115b
  • golang.org/x/oauth2 v0.0.0-20211104180415-d3ed0bb246c8
  • golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f
  • golang.org/x/term v0.0.0-20210927222741-03fcf44c2211
  • golang.org/x/text v0.3.7
  • golang.org/x/time v0.0.0-20220210224613-90d013bbcef8
  • gomodules.xyz/jsonpatch/v2 v2.2.0
  • google.golang.org/appengine v1.6.7
  • google.golang.org/protobuf v1.28.0
  • gopkg.in/inf.v0 v0.9.1
  • gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7
  • gopkg.in/yaml.v2 v2.4.0
  • gopkg.in/yaml.v3 v3.0.1
  • k8s.io/api v0.25.0
  • k8s.io/apiextensions-apiserver v0.24.0
  • k8s.io/apimachinery v0.25.0
  • k8s.io/client-go v0.25.0
  • k8s.io/component-base v0.24.0
  • k8s.io/klog/v2 v2.70.1
  • k8s.io/kube-openapi v0.0.0-20220803162953-67bda5d908f1
  • k8s.io/utils v0.0.0-20220728103510-ee6ede2d64ed
  • sigs.k8s.io/controller-runtime v0.12.1
  • sigs.k8s.io/json v0.0.0-20220713155537-f223a00ba0e2
  • sigs.k8s.io/structured-merge-diff/v4 v4.2.3
  • sigs.k8s.io/yaml v1.3.0
go.sum go
  • 962 dependencies
docs/requirements.txt pypi
  • ipykernel *
  • jupyter_client *
  • markupsafe ==2.0.1
  • matplotlib *
  • myst-parser *
  • nbconvert *
  • nbsphinx *
  • notebook *
  • numpy >=1.16
  • pandas *
  • readthedocs-sphinx-search *
  • recommonmark *
  • sphinx-copybutton *
  • sphinx_markdown_tables *
  • sphinx_material *
.github/workflows/build-deploy.yaml actions
  • actions/checkout v3 composite
  • actions/setup-go v3 composite
  • docker/login-action v2 composite
Dockerfile docker
  • gcr.io/distroless/static nonroot build
  • golang 1.18 build
.github/workflows/docs.yaml actions
  • JamesIves/github-pages-deploy-action ba1486788b0490a235422264426c45848eac35c6 composite
  • actions/checkout v3 composite
.github/workflows/helm.yaml actions
  • actions/checkout v3 composite
  • actions/setup-go v3 composite
  • docker/login-action v2 composite
.github/workflows/main.yaml actions
  • actions/checkout v3 composite
  • actions/setup-go v3 composite
  • crate-ci/typos 7ad296c72fa8265059cc03d1eda562fbdfcd6df2 composite
  • medyagh/setup-minikube 697f2b7aaed5f70bf2a94ee21a4ec3dde7b12f92 composite
.github/workflows/release.yaml actions
  • actions/checkout v3 composite
  • actions/setup-go v3 composite
  • docker/login-action v2 composite
  • softprops/action-gh-release v1 composite
.github/workflows/test-python.yaml actions
  • actions/checkout v3 composite
  • actions/setup-go v3 composite
  • medyagh/setup-minikube 697f2b7aaed5f70bf2a94ee21a4ec3dde7b12f92 composite
docs/_static/versions.json meteor
examples/machine-learning/tensorflow/setup.py pypi
sdk/python/v1alpha1/requirements.txt pypi
  • future *
  • kubernetes *
  • python_dateutil >=2.5.3
  • requests *
  • setuptools >=21.0.0
  • six >=1.10
  • urllib3 >=1.25.3
sdk/python/v1alpha1/setup.py pypi
  • kubernetes *
sdk/python/v1alpha1/test-requirements.txt pypi
  • pytest * test
  • pytest-cov >=2.8.1 test
  • pytest-randomly ==1.2.3 test