Data

Tools

Indexes

Applications

Experiments

Open Source Science

data

DATA: Decomposed Attention-based Task Adaptation for Rehearsal-Free Continual Learning

https://github.com/xnhyacinth/data

Science Score: 26.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (8.6%) to scientific vocabulary
Last synced: 10 months ago · JSON representation

Repository

DATA: Decomposed Attention-based Task Adaptation for Rehearsal-Free Continual Learning

Basic Info
Statistics
  • Stars: 6
  • Watchers: 1
  • Forks: 0
  • Open Issues: 2
  • Releases: 0
Created over 1 year ago · Last pushed 10 months ago
Metadata Files
Readme License Citation

README.md

DATA

Requirements

Install LLaMA-Factory following LLaMA-Factory.

bash cd DATA pip install -e ".[torch,metrics]"

Data

  • data folder has 15 tasks of Long Sequence Benchmark.

  • Download the datasets from official websites.

  • From Google drive: (we unified the formats of the above datasets). Link

Run

shell bash config/run.sh ${num_gpus} ${gpus} ${model} ${tuning_method} ${bs} ${lr_type} ${lr} ${filter} ${mode} ${select} ${r} ${deepspeed} ${data_rank1} ${data_rank2} ${restore} ${scale} ${adaprompt} ${reinit} ${ortho_mu} ${gap_layers} ${bakebone} ${nomlp} ${project} ${replay}

LLaMA2-7B

LoRA

shell bash config/run.sh 2 0,1 llama2-7b lora 16 constant 1e-4 0 all 0 8 -1 0 0 0 0 0 0 0 0 0 0 0 0

LoRAReplay

shell bash config/run.sh 2 0,1 llama2-7b lora 16 constant 1e-4 0 all 0 8 -1 0 0 0 0 0 0 0 0 0 0 0 1

DATA

shell bash config/run.sh 2 0,1 llama2-7b data 1 constant 1e-4 0 all 0 8 -1 2 8 0 0 8 1 1 4 0 0 0 0

Details

Coming Soon!

Referencing and Citing

If you find our work useful in your research and would like to cite our project, please use the following citation: found this work useful, please consider giving this repository a star and citing our paper as follows: bibtex @article{liao2025data, title={DATA: Decomposed Attention-based Task Adaptation for Rehearsal-Free Continual Learning}, author={Liao, Huanxuan and He, Shizhu and Hao, Yupu and Zhao, Jun and Liu, Kang }, journal={arXiv preprint arXiv:2502.11482}, year={2025} }

Owner

  • Login: Xnhyacinth
  • Kind: user

GitHub Events

Total
  • Issues event: 10
  • Watch event: 4
  • Issue comment event: 4
  • Push event: 10
  • Create event: 2
Last Year
  • Issues event: 10
  • Watch event: 4
  • Issue comment event: 4
  • Push event: 10
  • Create event: 2

Issues and Pull Requests

Last synced: over 1 year ago

All Time
  • Total issues: 0
  • Total pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Total issue authors: 0
  • Total pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 0
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • ShristiDasBiswas (7)
Pull Request Authors
Top Labels
Issue Labels
Pull Request Labels

Dependencies

docker/docker-cuda/Dockerfile docker
  • nvcr.io/nvidia/pytorch 24.02-py3 build
docker/docker-cuda/docker-compose.yml docker
docker/docker-npu/Dockerfile docker
  • ascendai/cann 8.0.rc1-910b-ubuntu22.04-py3.8 build
docker/docker-npu/docker-compose.yml docker
docker/docker-rocm/Dockerfile docker
  • hardandheavy/transformers-rocm 2.2.0 build
docker/docker-rocm/docker-compose.yml docker
pyproject.toml pypi
requirements.txt pypi
  • accelerate >=0.30.1,<=0.34.2
  • av *
  • datasets >=2.16.0,<=2.21.0
  • einops *
  • fastapi *
  • fire *
  • fuzzywuzzy *
  • gradio >=4.0.0,<5.0.0
  • matplotlib >=3.7.0
  • numpy <2.0.0
  • packaging *
  • pandas >=2.0.0
  • peft >=0.11.1,<=0.12.0
  • protobuf *
  • pydantic *
  • pyyaml *
  • sacrebleu *
  • sacremoses *
  • scipy *
  • sentencepiece *
  • sse-starlette *
  • thop *
  • tiktoken *
  • transformers >=4.41.2,<=4.45.2
  • trl >=0.8.6,<=0.9.6
  • uvicorn *
setup.py pypi
src/llamafactory/train/cl/rouge/requirements.txt pypi
  • absl-py *
  • nltk *
  • numpy *
  • six >=1.14
src/llamafactory/train/cl/rouge/setup.py pypi
  • absl-py *
  • nltk *
  • numpy *
  • six >=1.14.0
O-LoRA/src/rouge/requirements.txt pypi
  • absl-py *
  • nltk *
  • numpy *
  • six >=1.14
O-LoRA/src/rouge/setup.py pypi
  • absl-py *
  • nltk *
  • numpy *
  • six >=1.14.0