Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/dptech-corp/uni-core

an efficient distributed PyTorch framework

https://github.com/dptech-corp/uni-core

Science Score: 26.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (12.4%) to scientific vocabulary

Keywords

distributed pytorch
Last synced: 4 months ago · JSON representation

Repository

an efficient distributed PyTorch framework

Basic Info
  • Host: GitHub
  • Owner: dptech-corp
  • License: mit
  • Language: Python
  • Default Branch: main
  • Homepage:
  • Size: 417 KB
Statistics
  • Stars: 136
  • Watchers: 6
  • Forks: 37
  • Open Issues: 21
  • Releases: 3
Topics
distributed pytorch
Created over 3 years ago · Last pushed 10 months ago
Metadata Files
Readme License

README.md

Uni-Core, an efficient distributed PyTorch framework

Uni-Core is built for rapidly creating PyTorch models with high performance, especially for Transfromer-based models. It supports the following features: - Distributed training over multi-GPUs and multi-nodes - Mixed-precision training with fp16 and bf16 - High-performance fused CUDA kernels - model checkpoint management - Friendly logging - Buffered (GPU-CPU overlapping) data loader - Gradient accumulation - Commonly used optimizers and LR schedulers - Easy to create new models

Installation

Build from source

You can use python setup.py install or pip install . to build Uni-Core from source. The CUDA version in the build environment should be the same as the one in PyTorch.

You can also use python setup.py install --disable-cuda-ext to disalbe the cuda extension operator when cuda is not available.

Use pre-compiled python wheels

We also pre-compiled wheels by GitHub Actions. You can download them from the Release. And you should check the pyhon version, PyTorch version and CUDA version. For example, for PyToch 1.12.1, python 3.7, and CUDA 11.3, you can install unicore-0.0.1+cu113torch1.12.1-cp37-cp37m-linuxx8664.whl.

Docker image

We also provide the docker image. you can pull it by docker pull dptechnology/unicore:0.0.1-pytorch1.11.0-cuda11.3. To use GPUs within docker, you need to install nvidia-docker-2 first.

Example

To build a model, you can refer to example/bert.

Related projects

Acknowledgement

The main framework is from facebookresearch/fairseq.

The fused kernels are from guolinke/fused_ops.

Dockerfile is from guolinke/pytorch-docker.

License

This project is licensed under the terms of the MIT license. See LICENSE for additional details.

Owner

  • Name: DP Technology
  • Login: dptech-corp
  • Kind: organization
  • Location: China

GitHub Events

Total
  • Watch event: 25
  • Delete event: 16
  • Issue comment event: 3
  • Push event: 55
  • Pull request event: 15
  • Fork event: 6
  • Create event: 13
  • Commit comment event: 1
Last Year
  • Watch event: 25
  • Delete event: 16
  • Issue comment event: 3
  • Push event: 55
  • Pull request event: 15
  • Fork event: 6
  • Create event: 13
  • Commit comment event: 1

Issues and Pull Requests

Last synced: 6 months ago

All Time
  • Total issues: 0
  • Total pull requests: 7
  • Average time to close issues: N/A
  • Average time to close pull requests: 19 minutes
  • Total issue authors: 0
  • Total pull request authors: 1
  • Average comments per issue: 0
  • Average comments per pull request: 0.0
  • Merged pull requests: 7
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 0
  • Pull requests: 7
  • Average time to close issues: N/A
  • Average time to close pull requests: 19 minutes
  • Issue authors: 0
  • Pull request authors: 1
  • Average comments per issue: 0
  • Average comments per pull request: 0.0
  • Merged pull requests: 7
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • lucifer1004 (2)
  • wangjx22 (2)
  • PKUfjh (1)
  • dgg95223 (1)
  • orgw (1)
  • lindsey98 (1)
  • CLG68 (1)
Pull Request Authors
  • guolinke (11)
  • leasunhy (5)
  • robotcator (4)
Top Labels
Issue Labels
Pull Request Labels

Dependencies

requirements.txt pypi
  • iopath *
  • lmdb *
  • ml_collections *
  • numpy *
  • scipy *
  • tensorboardX *
  • tokenizers *
  • tqdm *
setup.py pypi
  • lmdb *
  • ml_collections *
  • numpy *
  • numpy <1.20.0
  • scipy *
  • tensorboardX *
  • tokenizers *
  • torch >=1.11.0
  • tqdm *
.github/workflows/docker_rdma.yml actions
  • actions/checkout v3 composite
  • docker/build-push-action v3 composite
  • docker/login-action v2 composite
  • docker/setup-buildx-action v2 composite
  • docker/setup-qemu-action v2 composite
.github/workflows/docker_rdma_latest.yml actions
  • actions/checkout v3 composite
  • docker/build-push-action v3 composite
  • docker/login-action v2 composite
  • docker/setup-buildx-action v2 composite
  • docker/setup-qemu-action v2 composite
.github/workflows/publish.yml actions
  • actions/checkout v3 composite
  • actions/create-release v1 composite
  • actions/setup-python v3 composite
  • actions/upload-release-asset v1 composite
  • joutvhu/get-release v1 composite
docker/rdma/Dockerfile docker
  • nvcr.io/nvidia/pytorch 22.04-py3 build