Data

Tools

Indexes

Applications

Experiments

Open Source Science

supersonic

Server infrastructure for GPU inference-as-a-service in large scientific experiments

https://github.com/fastmachinelearning/supersonic

Science Score: 67.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ✓
    CITATION.cff file
    Found CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ✓
    DOI references
    Found 6 DOI reference(s) in README
  • ✓
    Academic publication links
    Links to: zenodo.org
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (14.9%) to scientific vocabulary
Last synced: 9 months ago · JSON representation ·

Repository

Server infrastructure for GPU inference-as-a-service in large scientific experiments

Basic Info
Statistics
  • Stars: 7
  • Watchers: 15
  • Forks: 5
  • Open Issues: 1
  • Releases: 6
Created over 1 year ago · Last pushed 11 months ago
Metadata Files
Readme License Citation

README.md

Version DOI Artifact Hub Downloads License

logo logo SuperSONIC

The SuperSONIC project implements server infrastructure for inference-as-a-service applications in large high energy physics (HEP) and multi-messenger astrophysics (MMA) experiments. The server infrastructure is designed for deployment at Kubernetes clusters equipped with GPUs.

Currently, SuperSONIC supports the following functionality: - GPU inference-as-a-service via Nvidia Triton Inference Server - Load balancing across many GPUs via Envoy Proxy - Load-based autoscaling via KEDA - Monitoring via Prometheus, Grafana, and OpenTelemetry - Rate limiting - Token-based authentication

Installation

Pre-requisites: - a Kubernetes cluster with access to GPUs - a Prometheus instance installed on the cluster, or Prometheus CRDs to deploy your own instance - KEDA CRDs installed on the cluster (only if using autoscaling)

Install the latest released version from the Helm repository ``` helm repo add fastml https://fastmachinelearning.org/SuperSONIC helm repo update helm install fastml/supersonic -n -f ```
Install directly from a GitHub branch/tag/commit ``` git clone https://github.com/fastmachinelearning/SuperSONIC.git cd SuperSONIC git checkout helm dependency build helm/supersonic helm install helm/supersonic -n -f ```

To construct the values.yaml file for your application, follow Configuration guide.

The full list of configuration parameters is available in the Configuration reference.

Server diagram

diagram diagram-dark

Status of deployment

| | CMS | ATLAS | IceCube | |:---|:---:|:---:|:---:| | Purdue Geddes | ✅ | - | - | | Purdue Anvil | ✅ | - | - | | NRP Nautilus | ✅ | ✅ | ✅ | | UChicago | - | ✅ | - |

Publications

Dmitry Kondratyev, Benedikt Riedel, Yuan-Tang Chou, Miles Cochran-Branson, Noah Paladino, David Schultz, Mia Liu, Javier Duarte, Philip Harris, and Shih-Chieh Hsu
SuperSONIC: Cloud-Native Infrastructure for ML Inferencing
In Practice and Experience in Advanced Research Computing 2025: The Power of Collaboration (PEARC '25)
Association for Computing Machinery, New York, NY, USA. Article 29, 1–5. 2025.
https://doi.org/10.1145/3708035.3736049

Owner

  • Name: Fast Machine Learning Lab
  • Login: fastmachinelearning
  • Kind: organization
  • Email: fml@fastmachinelearning.org

Real-time and accelerated ML for fundamental sciences

Citation (CITATION.cff) 

cff-version: 1.2.0
message: "If you use this software, please cite it as below."
authors:
- family-names: "Kondratyev"
  given-names: "Dmitry"
  affiliation: Purdue University
  orcid: "https://orcid.org/0000-0002-7874-2480"
- family-names: "Chou"
  given-names: "Yuan-Tang"
  affiliation: University of Washington
  orcid: "https://orcid.org/0000-0002-2204-5731"
- family-names: "Paladino"
  given-names: "Noah"
  affiliation: MIT
  orcid: "https://orcid.org/0000-0003-1225-537X"
- family-names: "Riedel"
  given-names: "Benedikt"
  affiliation: University of Wisconsin-Madison
  orcid: "https://orcid.org/0000-0002-9524-8943"
- family-names: "Cochran-Branson"
  given-names: "Miles"
  affiliation: University of Washington
  orcid: "https://orcid.org/0000-0003-1020-1108"
title: "SuperSONIC"
version: 0.1.2
doi: 10.5281/zenodo.14816533
date-released: 2025-02-05
url: "https://github.com/fastmachinelearning/SuperSONIC"
abstract: >+
  Server infrastructure for inference-as-a-service in large
  scientific experiments.
keywords:
  - Kubernetes
  - NVIDIA Triton Inference Server
  - inference as a service
  - GPU
  - machine learning

GitHub Events

Total
  • Release event: 16
  • Watch event: 7
  • Delete event: 49
  • Issue comment event: 5
  • Member event: 1
  • Push event: 771
  • Pull request event: 133
  • Fork event: 6
  • Create event: 68
Last Year
  • Release event: 16
  • Watch event: 7
  • Delete event: 49
  • Issue comment event: 5
  • Member event: 1
  • Push event: 771
  • Pull request event: 133
  • Fork event: 6
  • Create event: 68

Issues and Pull Requests

Last synced: 10 months ago

All Time
  • Total issues: 0
  • Total pull requests: 114
  • Average time to close issues: N/A
  • Average time to close pull requests: about 15 hours
  • Total issue authors: 0
  • Total pull request authors: 6
  • Average comments per issue: 0
  • Average comments per pull request: 0.12
  • Merged pull requests: 96
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 0
  • Pull requests: 114
  • Average time to close issues: N/A
  • Average time to close pull requests: about 15 hours
  • Issue authors: 0
  • Pull request authors: 6
  • Average comments per issue: 0
  • Average comments per pull request: 0.12
  • Merged pull requests: 96
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
Pull Request Authors
  • kondratyevd (97)
  • ngpaladi (12)
  • ytchoutw (10)
  • milescb (4)
  • jmduarte (2)
  • briedel (2)
Top Labels
Issue Labels
Pull Request Labels