https://github.com/cedrickchee/dawnbench-analysis
DAWNBench analysis of CIFAR-10 time-to-accuracy.
Science Score: 13.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
✓codemeta.json file
Found codemeta.json file -
○.zenodo.json file
-
○DOI references
-
○Academic publication links
-
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (8.1%) to scientific vocabulary
Keywords
Repository
DAWNBench analysis of CIFAR-10 time-to-accuracy.
Basic Info
- Host: GitHub
- Owner: cedrickchee
- License: mit
- Language: Jupyter Notebook
- Default Branch: master
- Size: 441 KB
Statistics
- Stars: 2
- Watchers: 2
- Forks: 0
- Open Issues: 0
- Releases: 0
Topics
Metadata Files
README.md
DAWNBench Analysis
Introduction
DAWNBench is an end-to-end deep learning training and inference benchmark. Computation time and cost are critical resources in building deep models, yet many existing benchmarks focus solely on model accuracy.
This repository contains the analysis of DAWNBench results. We will analyze two common deep learning workloads, CIFAR-10 and ImageNet time-to-accuracy and training cost across different optimization strategies, model architectures, software frameworks, clouds, and hardware.
For the start, we will analyze only CIFAR-10 workload across two model architectures:
- Custom Wide ResNet
- ResNet18 (TODO)
|Rank | Time to 94% Accuracy | Model | Framework | Hardware |
| --- | --- | --- | --- | --- |
| 1 | 00:52:11 | Custom Wide ResNet + AdamW + modified 1 cycle policy
Cedric Chee
source | fastai / PyTorch 0.4.0 | 1 K80 (AWS p2.xlarge) |
| 2 | 00:55:51 | Custom Wide ResNet + modified 1 cycle policy
Cedric Chee
source | fastai / PyTorch 0.4.0 | 1 K80 (AWS p2.xlarge) |
| 3 | 0:59:38 | Custom DarkNet
Cedric Chee
source | PyTorch 0.3.1 | 1 K80 (AWS p2.xlarge) |
| 4 | 1:18:22 | Custom Wide ResNet
fast.ai students team
source | fastai / PyTorch 0.4.0 | 1 K80 (AWS p2.xlarge) |
Analysis
1. Custom Wide ResNet fast.ai students team DAWNBench submission (Baseline)
2. My Custom Wide ResNet + AdamW + modified 1 cycle policy hyper-parameters
3. My Custom Wide ResNet + modified 1 cycle policy hyper-parameters
Code to replicate all analyses
Instructions for building and running the container.
You will need to have Docker and nvidia-docker installed in order to run this.
- cd into cloned repo.
docker build -t cifar10 ../run_container.sh cifar10
Once the Docker container started, all you have to do is access Jupyter Notebook through this url, https://localhost:8888 in your web browser. Then, enter jupyter as password. Open the notebook and click 'run all cells'.
Owner
- Name: Cedric Chee
- Login: cedrickchee
- Kind: user
- Location: PID 1
- Company: InvictusByte
- Website: https://cedricchee.com
- Twitter: cedric_chee
- Repositories: 227
- Profile: https://github.com/cedrickchee
Lead Software Engineer | LLMs | full stack Go/JS dev, backend | product dev @ startups | 🧑🎓 CompSci | alumni: fast.ai, Antler.co
GitHub Events
Total
Last Year
Issues and Pull Requests
Last synced: 12 months ago
All Time
- Total issues: 0
- Total pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Total issue authors: 0
- Total pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Past Year
- Issues: 0
- Pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Issue authors: 0
- Pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0