Data

Tools

Indexes

Applications

Experiments

Open Source Science

systemds

An open source ML system for the end-to-end data science lifecycle

https://github.com/apache/systemds

Science Score: 36.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ✓
    Committers with academic emails
    9 of 221 committers (4.1%) from academic institutions
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (12.8%) to scientific vocabulary

Keywords

dml java python systemds

Keywords from Contributors

distributed deep-neural-networks interactive cryptocurrency transformer mesh interpretability sequences generic projection
Last synced: 6 months ago · JSON representation

Repository

An open source ML system for the end-to-end data science lifecycle

Basic Info
Statistics
  • Stars: 1,061
  • Watchers: 84
  • Forks: 500
  • Open Issues: 58
  • Releases: 10
Topics
dml java python systemds
Created over 10 years ago · Last pushed 6 months ago
Metadata Files
Readme Contributing License Citation

README.md

Apache SystemDS

Overview: Apache SystemDS is an open-source machine learning (ML) system for the end-to-end data science lifecycle from data preparation and cleaning, over efficient ML model training, to debugging and serving. ML algorithms or pipelines are specified in a high-level language with R-like syntax or related Python and Java APIs (with many builtin primitives), and the system automatically generates hybrid runtime plans of local, in-memory operations and distributed operations on Apache Spark. Additional backends exist for GPUs and federated learning.

Resource | Links ---------|------ Quick Start | Install, Quick Start and Hello World Documentation: | SystemDS Documentation Python Documentation | Python SystemDS Documentation Issue Tracker | Jira Dashboard

Status and Build: SystemDS is renamed from SystemML which is an Apache Top Level Project. To build from source visit SystemDS Install from source

Build Documentation LicenseCheck Java Tests codecov Python Test Total PyPI downloads Monthly PyPI downloads

Owner

  • Name: The Apache Software Foundation
  • Login: apache
  • Kind: organization

GitHub Events

Total
  • Create event: 15
  • Commit comment event: 10
  • Release event: 1
  • Watch event: 35
  • Delete event: 14
  • Issue comment event: 289
  • Push event: 214
  • Pull request review comment event: 98
  • Pull request review event: 66
  • Pull request event: 316
  • Fork event: 38
Last Year
  • Create event: 15
  • Commit comment event: 10
  • Release event: 1
  • Watch event: 35
  • Delete event: 14
  • Issue comment event: 289
  • Push event: 214
  • Pull request review comment event: 98
  • Pull request review event: 66
  • Pull request event: 316
  • Fork event: 38

Committers

Last synced: 8 months ago

All Time
  • Total Commits: 8,935
  • Total Committers: 221
  • Avg Commits per committer: 40.43
  • Development Distribution Score (DDS): 0.563
Past Year
  • Commits: 377
  • Committers: 36
  • Avg Commits per committer: 10.472
  • Development Distribution Score (DDS): 0.631
Top Committers
Name Email Commits
MATTHIAS BOEHM m****m@u****m 3,902
baunsgaard b****d@t****t 810
Niketan Pansare n****r@u****m 456
Deron Eriksson d****n@u****m 325
SHIRISH TATIKONDA s****o@u****m 269
arnabp a****i@t****t 219
Janardhan Pulivarthi j****3@p****m 172
PRITHVIRAJ SEN s****p@u****m 170
Tian Zhang t****g@u****m 168
DOUGLAS R. BURDICK d****c@u****m 144
Mike Dusenberry m****b@u****m 130
Shafaq Siddiqi s****i@t****t 129
JIHONG MA j****a@u****m 123
Arvind Surve a****s@y****m 111
Mark Dokter m****k@d****c 105
HAOJUN WANG h****w@u****m 102
Glenn Weidner g****r@u****m 93
Berthold Reinwald r****d@u****m 87
Luciano Resende l****e@a****g 84
Faraz Makari Manshadi f****i@u****m 79
YUANYUAN TIAN y****n@u****m 77
Nakul Jindal n****2@g****m 76
FREDERICK R. REISS f****s@u****m 67
sebwrede s****e@k****t 64
Kevin Innerebner k****r@y****m 53
e-strauss l****r@g****e 48
Olga o****a@g****m 45
ywcb00 y****0@y****g 45
dependabot[bot] 4****] 43
Alexandre V. Evfimievski e****i@u****m 42
and 191 more...
Committer Domains (Top 20 + Academic)
us.ibm.com: 40 student.tugraz.at: 34 tugraz.at: 3 apache.org: 3 gmx.net: 2 qq.com: 2 campus.tu-berlin.de: 2 ibm.com: 2 know-center.at: 2 dokter.cc: 1 ibm.us.com: 1 campus.tu-berlin.com: 1 botzner.com: 1 gmx.de: 1 nyu.edu: 1 nitkkr.ac.in: 1 almaden.ibm.com: 1 capitalone.com: 1 ywcb.org: 1 atos.net: 1 eng.ucsd.edu: 1 cs.washington.edu: 1 tu-berlin.de: 1 ece.ubc.ca: 1

Issues and Pull Requests

Last synced: 6 months ago

All Time
  • Total issues: 2
  • Total pull requests: 868
  • Average time to close issues: N/A
  • Average time to close pull requests: 22 days
  • Total issue authors: 2
  • Total pull request authors: 88
  • Average comments per issue: 0.0
  • Average comments per pull request: 1.62
  • Merged pull requests: 80
  • Bot issues: 0
  • Bot pull requests: 55
Past Year
  • Issues: 1
  • Pull requests: 389
  • Average time to close issues: N/A
  • Average time to close pull requests: 11 days
  • Issue authors: 1
  • Pull request authors: 45
  • Average comments per issue: 0.0
  • Average comments per pull request: 1.55
  • Merged pull requests: 31
  • Bot issues: 0
  • Bot pull requests: 33
View more stats
Top Authors
Issue Authors
  • christinadionysio (1)
  • theidkamp (1)
Pull Request Authors
  • Baunsgaard (247)
  • e-strauss (93)
  • dependabot[bot] (55)
  • phaniarnab (38)
  • christinadionysio (35)
  • ReneEnjilian (30)
  • sebwrede (30)
  • aarnaty (25)
  • MaximilianSchreff (23)
  • min-guk (23)
  • Frxms (20)
  • j143 (19)
  • fzoepffel (18)
  • Jaybit0 (12)
  • gghsu (10)
Top Labels
Issue Labels
Pull Request Labels
dependencies (55) github_actions (46) java (9)

Packages

  • Total packages: 2
  • Total downloads:
    • pypi 85 last-month
  • Total docker downloads: 95,397
  • Total dependent packages: 0
    (may contain duplicates)
  • Total dependent repositories: 2
    (may contain duplicates)
  • Total versions: 19
  • Total maintainers: 3
  • Total advisories: 1
pypi.org: systemds

Apache SystemDS - An open source ML system for the end-to-end data science lifecycle

  • Versions: 11
  • Dependent Packages: 0
  • Dependent Repositories: 1
  • Downloads: 85 Last month
  • Docker Downloads: 0
Rankings
Docker downloads count: 1.1%
Stargazers count: 2.0%
Forks count: 2.5%
Average: 8.2%
Dependent packages count: 10.1%
Downloads: 11.9%
Dependent repos count: 21.5%
Maintainers (3)
DAMSLab j143 systemds
Advisories (1)
Last synced: 6 months ago
repo1.maven.org: org.apache.systemds:systemds

An open source ML system for the end-to-end data science lifecycle

  • Versions: 8
  • Dependent Packages: 0
  • Dependent Repositories: 1
  • Docker Downloads: 95,397
Rankings
Docker downloads count: 2.7%
Forks count: 9.2%
Stargazers count: 12.0%
Average: 19.0%
Dependent repos count: 20.8%
Dependent packages count: 50.2%
Advisories (1)
Last synced: 6 months ago

Dependencies

.github/action/action.yml actions
  • Dockerfile * docker
.github/workflows/build-cron.yml actions
  • actions/checkout v3 composite
  • actions/setup-java v3 composite
.github/workflows/build.yml actions
  • actions/checkout v3 composite
  • actions/setup-java v3 composite
.github/workflows/docker-cd.yml actions
  • actions/checkout v3 composite
  • docker/build-push-action v3 composite
  • docker/login-action v2 composite
  • docker/metadata-action v4 composite
  • docker/setup-buildx-action v2 composite
.github/workflows/docker-release.yml actions
  • actions/checkout v3 composite
  • docker/build-push-action v3 composite
  • docker/login-action v2 composite
  • docker/metadata-action v4 composite
  • docker/setup-buildx-action v2 composite
.github/workflows/docker-testImage.yml actions
  • actions/checkout v3 composite
  • docker/build-push-action v3 composite
  • docker/login-action v2 composite
  • docker/metadata-action v4 composite
  • docker/setup-buildx-action v2 composite
.github/workflows/documentation.yml actions
  • actions/cache v3 composite
  • actions/checkout v3 composite
  • actions/setup-java v3 composite
  • actions/setup-python v4 composite
.github/workflows/license.yml actions
  • actions/checkout v3 composite
  • actions/setup-java v3 composite
.github/workflows/monitoringUITests.yml actions
  • actions/checkout v3 composite
  • actions/setup-node v3 composite
.github/workflows/python.yml actions
  • actions/cache v3 composite
  • actions/checkout v3 composite
  • actions/setup-java v3 composite
  • actions/setup-python v4 composite
.github/workflows/release-scripts.yml actions
  • actions/cache v2 composite
  • actions/checkout v2 composite
  • actions/setup-java v2 composite
.github/action/Dockerfile docker
  • apache/systemds testing-latest build
pom.xml maven
  • io.netty:netty-all 4.1.68.Final provided
  • junit:junit 4.13.1 provided
  • org.antlr:antlr4 4.8 provided
  • org.apache.derby:derby 10.14.2.0 provided
  • org.codehaus.janino:janino 3.0.16 provided
  • org.jcuda:jcublas 10.2.0 provided
  • org.jcuda:jcublas-natives 10.2.0 provided
  • org.jcuda:jcuda 10.2.0 provided
  • org.jcuda:jcuda-natives 10.2.0 provided
  • org.jcuda:jcudnn 10.2.0 provided
  • org.jcuda:jcudnn-natives 10.2.0 provided
  • org.jcuda:jcusolver 10.2.0 provided
  • org.jcuda:jcusolver-natives 10.2.0 provided
  • org.jcuda:jcusparse 10.2.0 provided
  • org.jcuda:jcusparse-natives 10.2.0 provided
  • com.fasterxml.jackson.core:jackson-databind 2.13.4.1
  • com.google.protobuf:protobuf-java 3.20.3
  • com.google.protobuf:protobuf-java-util 3.20.3
  • commons-logging:commons-logging 1.1.3
  • net.sf.py4j:py4j 0.10.9
  • org.antlr:antlr4-runtime 4.8
  • org.apache.commons:commons-math3 3.4.1
  • org.apache.hadoop:hadoop-client 3.3.4
  • org.apache.hadoop:hadoop-common 3.3.4
  • org.apache.hadoop:hadoop-hdfs 3.3.4
  • org.apache.logging.log4j:log4j-api 2.17.2
  • org.apache.logging.log4j:log4j-core 2.17.2
  • org.apache.maven.plugins:maven-assembly-plugin 3.3.0
  • org.apache.maven.plugins:maven-gpg-plugin 1.6
  • org.apache.maven.plugins:maven-javadoc-plugin 3.2.0
  • org.apache.spark:spark-core_2.12 3.3.1
  • org.apache.spark:spark-mllib_2.12 3.3.1
  • org.apache.spark:spark-sql_2.12 3.3.1
  • org.apache.wink:wink-json4j 1.4
  • org.slf4j:jcl-over-slf4j 1.7.36
  • org.slf4j:jul-to-slf4j 1.7.36
  • org.slf4j:slf4j-api 1.7.36
  • org.slf4j:slf4j-reload4j 1.7.36
  • com.github.stephenc.jcip:jcip-annotations 1.0-1 test
  • org.openjdk.jol:jol-core 0.10 test
scripts/monitoring/package.json npm
  • @angular-devkit/build-angular ~13.2.6 development
  • @angular/cli ~13.2.6 development
  • @angular/compiler-cli ~13.2.0 development
  • @types/jasmine ~3.10.0 development
  • @types/jest ^28.1.1 development
  • @types/node ^12.11.1 development
  • jasmine-core ~4.0.0 development
  • karma ~6.3.0 development
  • karma-chrome-launcher ~3.1.0 development
  • karma-coverage ~2.1.0 development
  • karma-firefox-launcher ~2.1.2 development
  • karma-jasmine ~4.0.0 development
  • karma-jasmine-html-reporter ~1.7.0 development
  • typescript ~4.5.2 development
  • @angular/animations ~13.2.0
  • @angular/cdk ^13.2.6
  • @angular/common ~13.2.0
  • @angular/compiler ~13.2.0
  • @angular/core ~13.2.0
  • @angular/forms ~13.2.0
  • @angular/material ^13.2.6
  • @angular/platform-browser ~13.2.0
  • @angular/platform-browser-dynamic ~13.2.0
  • @angular/router ~13.2.0
  • chart.js ~3.8.2
  • chartjs-adapter-moment ^1.0.0
  • jsplumb ~2.15.6
  • moment ^2.29.4
  • rxjs ~7.5.0
  • tslib ^2.3.0
  • zone.js ~0.11.4
src/main/python/docs/requires-docs.txt pypi
  • numpy *
  • py4j *
  • requests *
  • scipy *
  • sphinx *
  • sphinx_rtd_theme *
docs/Gemfile rubygems
  • jekyll >= 0
  • rouge >= 0
.github/workflows/cleanup-transient-artifacts.yml actions
  • actions/checkout v3 composite
.github/workflows/javaTests.yml actions
  • ./.github/action/ * composite
  • actions/cache v3 composite
  • actions/checkout v3 composite
  • actions/download-artifact v3 composite
  • actions/upload-artifact v3 composite
src/main/python/setup.py pypi
scripts/resource/requirements.txt pypi
  • boto3 *
  • pandas *
scripts/staging/cuda-counter-based-prng/pom.xml maven
  • org.jcuda:jcuda 10.2.0