Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/amazon-science/peft-design-spaces

Official implementation for "Parameter-Efficient Fine-Tuning Design Spaces"

https://github.com/amazon-science/peft-design-spaces

Science Score: 26.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (3.6%) to scientific vocabulary
Last synced: 9 months ago · JSON representation

Repository

Official implementation for "Parameter-Efficient Fine-Tuning Design Spaces"

Basic Info
  • Host: GitHub
  • Owner: amazon-science
  • License: apache-2.0
  • Language: Python
  • Default Branch: main
  • Homepage:
  • Size: 3.61 MB
Statistics
  • Stars: 26
  • Watchers: 2
  • Forks: 0
  • Open Issues: 2
  • Releases: 0
Created over 3 years ago · Last pushed over 3 years ago
Metadata Files
Readme Contributing License Code of conduct

README.md

Parameter-Efficient Fine-Tuning Design Spaces

This is the official implementation of the Parameter-Efficient Fine-Tuning Design Spaces.

Usage

One needs to setup the enrironment before running the experiments.

Setup

cd models pip install -e .

Experiments

Evaluating the S4-model (PEFT is the alias) with the RoBERTa backbone on GLUE

``` export TASK_NAME=sst-2

python runglue.py \ --modelnameorpath roberta-base-uncased \ --taskname $TASKNAME \ --dotrain \ --doeval \ --maxseqlength 128 \ --perdevicetrainbatchsize 16 \ --learningrate 5e-5 \ --numtrainepochs 10.0 \ --outputdir /tmp/$TASKNAME \ --overwriteoutputdir \ --trainadapter \ --adapter_config PEFT ```

Acknowledgement

Part of our codes are adapted from adapter-transformers.

License

This project is licensed under the Apache-2.0 License.

Owner

  • Name: Amazon Science
  • Login: amazon-science
  • Kind: organization

GitHub Events

Total
  • Watch event: 2
Last Year
  • Watch event: 2

Issues and Pull Requests

Last synced: over 1 year ago

All Time
  • Total issues: 2
  • Total pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Total issue authors: 2
  • Total pull request authors: 0
  • Average comments per issue: 0.0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 0
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • KlaineWei (1)
  • Chan-1996 (1)
Pull Request Authors
Top Labels
Issue Labels
Pull Request Labels