d2po

The official PyTorch implementation of the paper "Earlier Tokens Contribute More: Learning Direct Preference Optimization From Temporal Decay Perspective".

https://github.com/lotusrc/d2po

Science Score: 54.0%

This score indicates how likely this project is to be science-related based on various indicators:

✓
CITATION.cff file
Found CITATION.cff file
✓
codemeta.json file
Found codemeta.json file
✓
.zenodo.json file
Found .zenodo.json file
○
DOI references
✓
Academic publication links
Links to: arxiv.org
○
Academic email domains
○
Institutional organization owner
○
JOSS paper metadata
○
Scientific vocabulary similarity
Low similarity (2.9%) to scientific vocabulary

Last synced: 7 months ago · JSON representation ·

Repository

The official PyTorch implementation of the paper "Earlier Tokens Contribute More: Learning Direct Preference Optimization From Temporal Decay Perspective".

Basic Info

Host: GitHub
Owner: LotuSrc
License: apache-2.0
Language: Python
Default Branch: main
Size: 46.4 MB

Statistics

Stars: 1
Watchers: 1
Forks: 0
Open Issues: 1
Releases: 0

Created about 1 year ago · Last pushed 11 months ago

Metadata Files

Readme Contributing License Code of conduct Citation Security

README.md

D^2PO

This repository contains the code in both PyTorch for our paper.

Earlier Tokens Contribute More: Learning Direct Preference Optimization From Temporal Decay Perspective

Quick Start

The config template is in examples/exp. When gamma is set smaller than 1.0, the experiment runs in D^2PO mode. bash llamafactory-cli train examples/exp/llama3_full_d2po.yaml

Owner

Login: LotuSrc
Kind: user
Location: Beijing China
Company: Meituan

Repositories: 2
Profile: https://github.com/LotuSrc

Citation (CITATION.cff)

cff-version: 1.2.0
date-released: 2024-03
message: "If you use this software, please cite it as below."
authors:
- family-names: "Zheng"
  given-names: "Yaowei"
- family-names: "Zhang"
  given-names: "Richong"
- family-names: "Zhang"
  given-names: "Junhao"
- family-names: "Ye"
  given-names: "Yanhan"
- family-names: "Luo"
  given-names: "Zheyan"
- family-names: "Feng"
  given-names: "Zhangchi"
- family-names: "Ma"
  given-names: "Yongqiang"
title: "LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models"
url: "https://arxiv.org/abs/2403.13372"
preferred-citation:
  type: conference-paper
  conference:
    name: "Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations)"
  authors:
    - family-names: "Zheng"
      given-names: "Yaowei"
    - family-names: "Zhang"
      given-names: "Richong"
    - family-names: "Zhang"
      given-names: "Junhao"
    - family-names: "Ye"
      given-names: "Yanhan"
    - family-names: "Luo"
      given-names: "Zheyan"
    - family-names: "Feng"
      given-names: "Zhangchi"
    - family-names: "Ma"
      given-names: "Yongqiang"
  title: "LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models"
  url: "https://arxiv.org/abs/2403.13372"
  year: 2024
  publisher: "Association for Computational Linguistics"
  address: "Bangkok, Thailand"

GitHub Events

Total

Issues event: 1
Watch event: 3
Push event: 1
Create event: 2

Last Year

Issues event: 1
Watch event: 3
Push event: 1
Create event: 2

Dependencies

.github/workflows/label_issue.yml actions

.github/workflows/publish.yml actions

actions/checkout v4 composite
actions/setup-python v5 composite
pypa/gh-action-pypi-publish release/v1 composite

.github/workflows/tests.yml actions

actions/cache v4 composite
actions/checkout v4 composite
actions/setup-python v5 composite

docker/docker-cuda/Dockerfile docker

${BASE_IMAGE} latest build

docker/docker-cuda/docker-compose.yml docker

docker/docker-npu/Dockerfile docker

ascendai/cann 8.0.0-910b-ubuntu22.04-py3.10 build

docker/docker-npu/docker-compose.yml docker

docker/docker-rocm/Dockerfile docker

hardandheavy/transformers-rocm 2.2.0 build

docker/docker-rocm/docker-compose.yml docker

pyproject.toml pypi

requirements.txt pypi

accelerate >=0.34.0,<=1.6.0
av *
datasets >=2.16.0,<=3.5.0
einops *
fastapi *
fire *
gradio >=4.38.0,<=5.25.0
librosa *
matplotlib >=3.7.0
numpy <2.0.0
omegaconf *
packaging *
pandas >=2.0.0
peft >=0.14.0,<=0.15.1
protobuf *
pydantic <=2.10.6
pyyaml *
scipy *
sentencepiece *
sse-starlette *
tiktoken *
tokenizers >=0.19.0,<=0.21.1
transformers >=4.45.0,<=4.51.3,
trl >=0.8.6,<=0.9.6
tyro <0.9.0
uvicorn *

setup.py pypi

ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Open Source Science