Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/beomi/easy-lm-trainer

πŸ€— μ΅œμ†Œν•œμ˜ μ„ΈνŒ…μœΌλ‘œ LM을 ν•™μŠ΅ν•˜κΈ° μœ„ν•œ μƒ˜ν”Œμ½”λ“œ

https://github.com/beomi/easy-lm-trainer

Science Score: 26.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • β—‹
    CITATION.cff file
  • βœ“
    codemeta.json file
    Found codemeta.json file
  • βœ“
    .zenodo.json file
    Found .zenodo.json file
  • β—‹
    DOI references
  • β—‹
    Academic publication links
  • β—‹
    Committers with academic emails
  • β—‹
    Institutional organization owner
  • β—‹
    JOSS paper metadata
  • β—‹
    Scientific vocabulary similarity
    Low similarity (1.9%) to scientific vocabulary

Keywords

boilerplate huggingface language-model transformers

Keywords from Contributors

interactive projection generic sequences archival genomics observability autograding hacking shellcodes
Last synced: 5 months ago · JSON representation

Repository

πŸ€— μ΅œμ†Œν•œμ˜ μ„ΈνŒ…μœΌλ‘œ LM을 ν•™μŠ΅ν•˜κΈ° μœ„ν•œ μƒ˜ν”Œμ½”λ“œ

Basic Info
  • Host: GitHub
  • Owner: Beomi
  • Language: Python
  • Default Branch: main
  • Homepage:
  • Size: 5.55 MB
Statistics
  • Stars: 58
  • Watchers: 2
  • Forks: 8
  • Open Issues: 2
  • Releases: 0
Topics
boilerplate huggingface language-model transformers
Created over 3 years ago · Last pushed over 2 years ago
Metadata Files
Readme

README.md

Easy LM Trainer

Huggingface Transformersλ₯Ό μ‚¬μš©ν•΄ LM을 ν•™μŠ΅ν•  λ•Œ, λ‹¨μˆœ LM(CLM) ν•™μŠ΅ μž‘μ—…μ„ 보닀 μ‰½κ²Œ μ‹œμž‘ν•˜κΈ° μœ„ν•œ Boilerplate ν”„λ‘œμ νŠΈ.

ν™˜κ²½

  • CPython 3.10+
  • PyTorchλŠ” CUDA ν™˜κ²½μ— 맞게 μ„€μΉ˜ν•˜κΈ° (1.12.1 이상)

bash pip install -r requirements.txt pip install -U deepspeed # Deepspeed branch ν•œμ •

μ‹€ν–‰

bash ./train.sh

  • μ˜ˆμ‹œ ν™˜κ²½μ€ RAM 1TB, GPU A100 40GB x4μž₯ ν™˜κ²½μ—μ„œ μ‹€ν—˜
  • CUDA 11.6/11.7
  • PyTorch 2.0
  • KoAlpaca Dataset을 ν•™μŠ΅ λ°μ΄ν„°λ‘œ μ‚¬μš©ν•¨
  • DeepSpeed ZeRO3, Optimizer와 Parameter λͺ¨λ‘ CPU Offload
  • Seq len 1024
  • Max batch size 1 (per GPU)
  • GPUλ‹Ή μ•½ 27GB vram μ‚¬μš© (= V100 32Gμ—μ„œλ„ μ‚¬μš© κ°€λŠ₯ν•  κ²ƒμœΌλ‘œ μ˜ˆμƒ)

Owner

  • Name: Junbum Lee
  • Login: Beomi
  • Kind: user
  • Location: Seoul, South Korea

AI/ML GDE @ml-gde. Korean AI/NLP Researcher and creator of multiple Korean PLMs. Focused on advancing Open LLMs.

GitHub Events

Total
  • Watch event: 1
Last Year
  • Watch event: 1

Committers

Last synced: about 1 year ago

All Time
  • Total Commits: 23
  • Total Committers: 2
  • Avg Commits per committer: 11.5
  • Development Distribution Score (DDS): 0.043
Past Year
  • Commits: 0
  • Committers: 0
  • Avg Commits per committer: 0.0
  • Development Distribution Score (DDS): 0.0
Top Committers
Name Email Commits
Junbum Lee j****n@b****t 22
dependabot[bot] 4****] 1
Committer Domains (Top 20 + Academic)
beomi.net: 1

Issues and Pull Requests

Last synced: 10 months ago

All Time
  • Total issues: 3
  • Total pull requests: 2
  • Average time to close issues: 4 days
  • Average time to close pull requests: 1 minute
  • Total issue authors: 2
  • Total pull request authors: 2
  • Average comments per issue: 2.0
  • Average comments per pull request: 0.0
  • Merged pull requests: 2
  • Bot issues: 0
  • Bot pull requests: 1
Past Year
  • Issues: 0
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • park1200656 (2)
  • sjhyeon2 (1)
Pull Request Authors
  • dependabot[bot] (1)
  • Beomi (1)
Top Labels
Issue Labels
Pull Request Labels
dependencies (1)