superbenchmark

A validation and profiling tool for AI infrastructure

https://github.com/microsoft/superbenchmark

Science Score: 44.0%

This score indicates how likely this project is to be science-related based on various indicators:

✓
CITATION.cff file
Found CITATION.cff file
✓
codemeta.json file
Found codemeta.json file
✓
.zenodo.json file
Found .zenodo.json file
○
DOI references
○
Academic publication links
○
Committers with academic emails
○
Institutional organization owner
○
JOSS paper metadata
○
Scientific vocabulary similarity
Low similarity (6.3%) to scientific vocabulary

Keywords

ai-system azure benchmark hacktoberfest superbench

Keywords from Contributors

transformer sequences optim large-language-model mesh interpretability cryptocurrencies energy-system generic projection

Last synced: 6 months ago · JSON representation ·

Repository

A validation and profiling tool for AI infrastructure

Basic Info

Host: GitHub
Owner: microsoft
License: mit
Language: Python
Default Branch: main
Homepage: https://aka.ms/superbench
Size: 9.98 MB

Statistics

Stars: 330
Watchers: 15
Forks: 71
Open Issues: 12
Releases: 14

Topics

ai-system azure benchmark hacktoberfest superbench

Created about 5 years ago · Last pushed 6 months ago

Metadata Files

Readme License Code of conduct Citation Codeowners Security Support

SuperBench

| Azure Pipelines | Build Status | |--------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | cpu-unit-test | | | cuda-unit-test | | | ansible-integration-test | |

SuperBench is a validation and profiling tool for AI infrastructure.

📢 v0.12.0 has been released!

Check aka.ms/superbench for more details.

Citations

To cite SuperBench in your publications:

bib @inproceedings {superbench, author = {Yifan Xiong and Yuting Jiang and Ziyue Yang and Lei Qu and Guoshuai Zhao and Shuguang Liu and Dong Zhong and Boris Pinzur and Jie Zhang and Yang Wang and Jithin Jose and Hossein Pourreza and Jeff Baxter and Kushal Datta and Prabhat Ram and Luke Melton and Joe Chau and Peng Cheng and Yongqiang Xiong and Lidong Zhou}, title = {{SuperBench}: Improving Cloud {AI} Infrastructure Reliability with Proactive Validation}, booktitle = {2024 USENIX Annual Technical Conference (USENIX ATC 24)}, year = {2024}, isbn = {978-1-939133-41-0}, address = {Santa Clara, CA}, pages = {835--850}, url = {https://www.usenix.org/conference/atc24/presentation/xiong}, publisher = {USENIX Association}, month = jul }

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.

Owner

Name: Microsoft
Login: microsoft
Kind: organization
Email: opensource@microsoft.com
Location: Redmond, WA

Website: https://opensource.microsoft.com
Twitter: OpenAtMicrosoft
Repositories: 7,257
Profile: https://github.com/microsoft

Open source projects and samples from Microsoft

Citation (CITATION.bib)

@inproceedings {superbench,
	author = {Yifan Xiong and Yuting Jiang and Ziyue Yang and Lei Qu and Guoshuai Zhao and Shuguang Liu and Dong Zhong and Boris Pinzur and Jie Zhang and Yang Wang and Jithin Jose and Hossein Pourreza and Jeff Baxter and Kushal Datta and Prabhat Ram and Luke Melton and Joe Chau and Peng Cheng and Yongqiang Xiong and Lidong Zhou},
	title = {{SuperBench}: Improving Cloud {AI} Infrastructure Reliability with Proactive Validation},
	booktitle = {2024 USENIX Annual Technical Conference (USENIX ATC 24)},
	year = {2024},
	isbn = {978-1-939133-41-0},
	address = {Santa Clara, CA},
	pages = {835--850},
	url = {https://www.usenix.org/conference/atc24/presentation/xiong},
	publisher = {USENIX Association},
	month = jul
}

GitHub Events

Total

Create event: 43
Release event: 1
Issues event: 15
Watch event: 52
Delete event: 33
Issue comment event: 127
Push event: 167
Pull request review event: 240
Pull request review comment event: 202
Pull request event: 111
Fork event: 15

Last Year

Create event: 43
Release event: 1
Issues event: 15
Watch event: 52
Delete event: 33
Issue comment event: 127
Push event: 167
Pull request review event: 240
Pull request review comment event: 202
Pull request event: 111
Fork event: 15

Committers

Last synced: 10 months ago

All Time

Total Commits: 464
Total Committers: 26
Avg Commits per committer: 17.846
Development Distribution Score (DDS): 0.75

Past Year

Commits: 45
Committers: 10
Avg Commits per committer: 4.5
Development Distribution Score (DDS): 0.756

Top Committers

Name	Email	Commits
Yifan Xiong	y**g@m**m	116
guoshzhao	g**o@m**m	96
Yuting Jiang	y**g@m**m	77
Yuting Jiang	v**g@m**m	46
dependabot[bot]	4****]	34
Ziyue Yang	z**g@m**m	29
Yang Wang	y**1@m**m	14
Hongtao Zhang	g**t@g**m	11
TobeyQin	3****n	11
pdr	d**i@g**m	7
Microsoft Open Source	m****e	5
rafsalas19	7****9	3
pnunna93	1****3	2
F̷N̷	k**t@g**m	1
Hossein Pourreza	p**a@y**m	1
Jared Bowden	9****D	1
Jeff Daily	j**y@a**m	1
Jie Zhang	1****0	1
Jorge Esguerra	1****g	1
Kaiyu Xie	s**y@g**m	1
Kirill Prosvirov	p**k@g**m	1
Lei Qu	5****9	1
Maxim Evtush	1****h	1
omahs	7****s	1
yangpanMS	3****S	1
Peng Cheng	c**5@o**m	1

Committer Domains (Top 20 + Academic)

microsoft.com: 6 amd.com: 1

Issues and Pull Requests

Last synced: 6 months ago

All Time

Total issues: 23
Total pull requests: 226
Average time to close issues: 2 months
Average time to close pull requests: 10 days
Total issue authors: 17
Total pull request authors: 22
Average comments per issue: 0.48
Average comments per pull request: 1.38
Merged pull requests: 167
Bot issues: 0
Bot pull requests: 31

Past Year

Issues: 14
Pull requests: 99
Average time to close issues: 5 days
Average time to close pull requests: 11 days
Issue authors: 10
Pull request authors: 13
Average comments per issue: 0.29
Average comments per pull request: 1.46
Merged pull requests: 53
Bot issues: 0
Bot pull requests: 19

View more stats

Top Authors

Issue Authors

hcoona (3)
polarG (2)
TobeyQin (2)
cp5555 (2)
netllama (2)
homily707 (1)
fno2010 (1)
avnf (1)
Tiger1218 (1)
ffflyZzz (1)
guoshzhao (1)
LiliDeng (1)
DK-DARKmatter (1)
quge009 (1)
yukirora (1)

Pull Request Authors

guoshzhao (56)
abuccts (42)
dependabot[bot] (31)
polarG (26)
yukirora (23)
dpower4 (17)
RyoYang (4)
TobeyQin (4)
Akado2009 (4)
WenqingLan1 (2)
Aishwarya-Tonpe (2)
jorgeesg (2)
maximevtush (2)
Tiger1218 (2)
hbabak (2)

Top Labels

Issue Labels

iteration plan (4) test (1) benchmarks (1) micro-benchmarks (1)

Pull Request Labels

benchmarks (52) dependencies (35) micro-benchmarks (29) CI/CD (20) bug (14) test (13) documentation (13) containers (11) model-benchmarks (11) enhancement (8) runner (5) CLI (4) executor (4) release (3) configuration (2) utils (2) tool (1)

Packages

Total packages: 1
Total downloads: unknown

Total dependent packages: 0
Total dependent repositories: 0
Total versions: 14

proxy.golang.org: github.com/microsoft/superbenchmark

Documentation: https://pkg.go.dev/github.com/microsoft/superbenchmark#section-documentation
License: mit
Latest release: v0.12.0
published 7 months ago

Versions: 14
Dependent Packages: 0
Dependent Repositories: 0

Rankings

Dependent packages count: 6.4%

Average: 6.6%

Dependent repos count: 6.8%

Last synced: 6 months ago

Dependencies

.github/workflows/build-image.yml actions

actions/checkout v2 composite
docker/build-push-action v2 composite
docker/login-action v1 composite
docker/setup-buildx-action v1 composite
docker/setup-qemu-action v1 composite

.github/workflows/codeql-analysis.yml actions

actions/checkout v3 composite
github/codeql-action/analyze v2 composite
github/codeql-action/autobuild v2 composite
github/codeql-action/init v2 composite

.github/workflows/gh-pages.yml actions

actions/checkout v2 composite
actions/setup-node v2 composite
webfactory/ssh-agent v0.5.0 composite

.github/workflows/lint.yml actions

actions/checkout v2 composite

website/package-lock.json npm

1241 dependencies

website/package.json npm

@docusaurus/core 2.0.0-beta.1
@docusaurus/preset-classic 2.0.0-beta.1
@docusaurus/theme-search-algolia ^2.0.0-beta.4
@mdx-js/react ^1.6.21
@svgr/webpack ^5.5.0
clsx ^1.1.1
file-loader ^6.2.0
prism-react-renderer ^1.2.1
react ^17.0.1
react-dom ^17.0.1
url-loader ^4.1.1

setup.py pypi

ansible_base >=2.10.9
ansible_runner >=2.0.0rc1
colorlog >=6.7.0
importlib_metadata *
jinja2 >=2.10.1
joblib >=1.0.1
jsonlines >=2.0.0
knack >=0.7.2
markdown >=3.3.0
matplotlib >=3.0.0
natsort >=7.1.1
networkx >=1.11
numpy >=1.19.2
omegaconf ==2.0.6
openpyxl >=3.0.7
pandas ==1.1.5
pssh *
pyyaml >=5.3
requests >=2.27.1
seaborn >=0.11.2
tcping >=0.1.1rc1
urllib3 >=1.26.9
xlrd >=2.0.1
xlsxwriter >=1.3.8
xmltodict >=0.12.0

superbenchmark

Science Score: 44.0%

Keywords

Keywords from Contributors

Repository

Basic Info

Statistics

Topics

Metadata Files

README.md

SuperBench

Check aka.ms/superbench for more details.

Citations

Trademarks

Owner

Citation (CITATION.bib)

GitHub Events

Total

Last Year

Committers

All Time

Past Year

Top Committers

Committer Domains (Top 20 + Academic)

Issues and Pull Requests

All Time

Past Year

Top Authors

Issue Authors

Pull Request Authors

Top Labels

Issue Labels

Pull Request Labels

Packages

proxy.golang.org: github.com/microsoft/superbenchmark

Rankings

Dependencies