Data

Tools

Indexes

Applications

Experiments

Open Source Science

LLM-LNS-Quantization

https://github.com/pouya-haghi/LLM-LNS-Quantization

Science Score: 44.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ✓
    CITATION.cff file
    Found CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (1.9%) to scientific vocabulary
Last synced: 6 months ago · JSON representation ·

Repository

Basic Info
  • Host: GitHub
  • Owner: pouya-haghi
  • License: mit
  • Language: Python
  • Default Branch: main
  • Size: 3.34 MB
Statistics
  • Stars: 1
  • Watchers: 1
  • Forks: 0
  • Open Issues: 0
  • Releases: 0
Created over 2 years ago · Last pushed over 1 year ago
Metadata Files
Readme License Citation Codeowners

README.md

LLM-LNS-Quantization

This repository updates LMEvalHarness to add new quantization methods to it.

The new formats are: - LNS8, LNS4 - Dynamic LNS with optimizations (per-block quantization) in 8 and 4 bits - FP8, FP4 - MX block floating-point - ZeroQuant - VSQuant - INT8 (W8A8) - SmoothQuant - LLM.int8()

The file is under lm_eval/models/huggingface.py.

Owner

  • Login: pouya-haghi
  • Kind: user

Citation (CITATION.bib) 

@software{eval-harness,
  author       = {Gao, Leo and
                  Tow, Jonathan and
                  Biderman, Stella and
                  Black, Sid and
                  DiPofi, Anthony and
                  Foster, Charles and
                  Golding, Laurence and
                  Hsu, Jeffrey and
                  McDonell, Kyle and
                  Muennighoff, Niklas and
                  Phang, Jason and
                  Reynolds, Laria and
                  Tang, Eric and
                  Thite, Anish and
                  Wang, Ben and
                  Wang, Kevin and
                  Zou, Andy},
  title        = {A framework for few-shot language model evaluation},
  month        = sep,
  year         = 2021,
  publisher    = {Zenodo},
  version      = {v0.0.1},
  doi          = {10.5281/zenodo.5371628},
  url          = {https://doi.org/10.5281/zenodo.5371628}
}

GitHub Events

Total
  • Watch event: 1
Last Year
  • Watch event: 1

Dependencies

Dockerfile docker
  • nvidia/cuda 11.2.0-cudnn8-runtime-ubuntu20.04 build
tests/Dockerfile docker
  • nvidia/cuda 11.2.0-cudnn8-runtime-ubuntu20.04 build
requirements.txt pypi
setup.py pypi
  • accelerate >=0.17.1
  • datasets >=2.0.0
  • einops *
  • jsonlines *
  • numexpr *
  • omegaconf >=2.2
  • openai >=0.6.4
  • peft >=0.2.0
  • pybind11 >=2.6.2
  • pycountry *
  • pytablewriter *
  • rouge-score >=0.0.4
  • sacrebleu ==1.5.0
  • scikit-learn >=0.24.1
  • sqlitedict *
  • torch >=1.7
  • tqdm-multiprocess *
  • transformers >=4.1
  • zstandard *
tests/requirements.txt pypi
tests/setup.py pypi
  • accelerate >=0.17.1
  • datasets >=2.0.0
  • einops *
  • jsonlines *
  • numexpr *
  • omegaconf >=2.2
  • openai >=0.6.4
  • peft >=0.2.0
  • pybind11 >=2.6.2
  • pycountry *
  • pytablewriter *
  • rouge-score >=0.0.4
  • sacrebleu ==1.5.0
  • scikit-learn >=0.24.1
  • sqlitedict *
  • torch >=1.7
  • tqdm-multiprocess *
  • transformers >=4.1
  • zstandard *