Data

Tools

Indexes

Applications

Experiments

Open Source Science

llm-based-mt-via-model-editing

Mitigating errors and improving translation quality in LLM-based Machine Translation via Model Editing

https://github.com/weichuanw/llm-based-mt-via-model-editing

Science Score: 54.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ✓
    CITATION.cff file
    Found CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ✓
    Academic publication links
    Links to: arxiv.org
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (6.4%) to scientific vocabulary
Last synced: 6 months ago · JSON representation ·

Repository

Mitigating errors and improving translation quality in LLM-based Machine Translation via Model Editing

Basic Info
  • Host: GitHub
  • Owner: weichuanW
  • Language: Python
  • Default Branch: main
  • Size: 2.2 MB
Statistics
  • Stars: 5
  • Watchers: 1
  • Forks: 0
  • Open Issues: 0
  • Releases: 0
Created over 1 year ago · Last pushed over 1 year ago
Metadata Files
Readme Citation Support

README.md

Mitigating the Language Mismatch and Repetition Issues in LLM-based Machine Translation via Model Editing

Brief Instruction

Here is a brief instruction for the implementation of the paper Mitigating the Language Mismatch and Repetition Issues in LLM-based Machine Translation via Model Editing. We will update more details soon.

Data

You can download the data used in the paper at Google Drive

Direct Reproduction

You can refer to the Runner folder to check the bash for direct reproduction with you own environment and 

pip install -r requirements.txt

Other Usage

You can download the data from Google Drive and use the intermediate data to do quick adaptation. If you have further questions or cooperation willings, please create new issues or email me

Owner

  • Name: Wang weichuan
  • Login: weichuanW
  • Kind: user

Citation (CITATION.cff) 

# This CITATION.cff file was generated with cffinit.
# Visit https://bit.ly/cffinit to generate yours today!

cff-version: 1.2.0
title: >-
  Mitigating the Language Mismatch and Repetition Issues in
  LLM-based Machine Translation via Model Editing
message: >-
  If you use this software, please cite it using the
  metadata from this file.
type: software
authors:
  - given-names: Weichuan
    family-names: Wang
    email: weicwang2-c@my.cityu.edu.hk
    name-particle: Weichuan
    affiliation: City University of Hong Kong
    orcid: 'https://orcid.org/0000-0001-5714-488X'
identifiers:
  - type: url
    value: 'https://arxiv.org/pdf/2410.07054'
repository-code: >-
  https://github.com/weichuanW/llm-based-mt-via-model-editing

GitHub Events

Total
  • Watch event: 7
  • Push event: 12
Last Year
  • Watch event: 7
  • Push event: 12

Dependencies

requirements.txt pypi
  • DataProperty ==1.0.1
  • GitPython ==3.1.40
  • Jinja2 ==3.1.2
  • Levenshtein ==0.23.0
  • MarkupSafe ==2.1.3
  • Morfessor ==2.0.6
  • Pillow ==9.3.0
  • PyICU ==2.13
  • PyYAML ==6.0.1
  • Pygments ==2.17.2
  • absl-py ==2.1.0
  • accelerate ==0.31.0
  • aeidon ==1.13
  • aiofiles ==23.2.1
  • aiohttp ==3.9.0
  • aiosignal ==1.3.1
  • altair ==5.2.0
  • annotated-types ==0.6.0
  • anyio ==3.7.1
  • appdirs ==1.4.4
  • async-timeout ==4.0.3
  • attrs ==23.1.0
  • beautifulsoup4 ==4.12.2
  • bitsandbytes ==0.41.2.post2
  • blis ==0.7.11
  • catalogue ==2.0.10
  • certifi ==2022.12.7
  • chardet ==5.2.0
  • charset-normalizer ==2.1.1
  • cheroot ==10.0.0
  • chinese-converter ==1.1.1
  • click ==8.1.7
  • cloudpathlib ==0.16.0
  • colorama ==0.4.6
  • concatenate ==0.1.2
  • confection ==0.1.4
  • contourpy ==1.2.0
  • cvxopt ==1.3.2
  • cycler ==0.12.1
  • cymem ==2.0.8
  • datasets ==3.1.0
  • deepspeed ==0.12.3
  • dill ==0.3.7
  • docker-pycreds ==0.4.0
  • docstring-parser ==0.15
  • docx2txt ==0.8
  • einops ==0.7.0
  • entmax ==1.3
  • evaluate ==0.4.1
  • exceptiongroup ==1.2.0
  • fairseq2 ==0.2.1
  • fairseq2n ==0.2.1
  • fastapi ==0.104.1
  • fastbm25 ==0.0.2
  • ffmpy ==0.3.1
  • filelock ==3.9.0
  • fluent.syntax ==0.19.0
  • fonttools ==4.46.0
  • frozenlist ==1.4.0
  • fsspec ==2023.10.0
  • future ==1.0.0
  • gcld3 ==3.0.13
  • gitdb ==4.0.11
  • gradio ==4.8.0
  • gradio_client ==0.7.1
  • hjson ==3.1.0
  • hpack ==4.0.0
  • httpcore ==1.0.2
  • httpx ==0.25.2
  • huggingface-hub ==0.26.2
  • importlib-resources ==6.1.1
  • iniparse ==0.5
  • install ==1.3.5
  • jaraco.functools ==4.0.0
  • jiwer ==3.0.4
  • joblib ==1.3.2
  • jsonargparse ==3.13.1
  • jsonlines ==4.0.0
  • jsonschema ==4.20.0
  • jsonschema-specifications ==2023.11.2
  • kiwisolver ==1.4.5
  • langcodes ==3.3.0
  • langdetect ==1.0.9
  • lightning-utilities ==0.11.2
  • lxml ==4.9.3
  • markdown-it-py ==3.0.0
  • matplotlib ==3.8.2
  • mbstrdecoder ==1.1.3
  • mdurl ==0.1.2
  • mistletoe ==1.2.1
  • more-itertools ==10.1.0
  • mpmath ==1.3.0
  • multidict ==6.0.4
  • multiprocess ==0.70.15
  • murmurhash ==1.0.10
  • networkx ==3.0
  • ninja ==1.11.1.1
  • nltk ==3.9.1
  • numexpr ==2.10.1
  • numpy ==1.24.1
  • nvidia-cublas-cu12 ==12.1.3.1
  • nvidia-cuda-cupti-cu12 ==12.1.105
  • nvidia-cuda-nvrtc-cu12 ==12.1.105
  • nvidia-cuda-runtime-cu12 ==12.1.105
  • nvidia-cudnn-cu12 ==8.9.2.26
  • nvidia-cufft-cu12 ==11.0.2.54
  • nvidia-curand-cu12 ==10.3.2.106
  • nvidia-cusolver-cu12 ==11.4.5.107
  • nvidia-cusparse-cu12 ==12.1.0.106
  • nvidia-nccl-cu12 ==2.19.3
  • nvidia-nvjitlink-cu12 ==12.6.77
  • nvidia-nvtx-cu12 ==12.1.105
  • orjson ==3.9.10
  • packaging ==23.2
  • pandas ==2.1.3
  • pathvalidate ==3.2.1
  • peft ==0.6.2
  • phply ==1.2.6
  • ply ==3.11
  • polyglot ==16.7.4
  • portalocker ==2.8.2
  • preshed ==3.0.9
  • prl ==0.94b0
  • protobuf ==4.25.1
  • psutil ==5.9.6
  • py-cpuinfo ==9.0.0
  • pyarrow ==18.0.0
  • pyarrow-hotfix ==0.6
  • pybind11 ==2.12.0
  • pycld2 ==0.41
  • pydantic ==2.5.2
  • pydantic_core ==2.14.5
  • pydub ==0.25.1
  • pyenchant ==3.2.2
  • pynvml ==11.5.0
  • pyparsing ==3.1.1
  • pytablewriter ==1.2.0
  • python-Levenshtein ==0.23.0
  • python-dateutil ==2.8.2
  • python-multipart ==0.0.6
  • pytorch-lightning ==2.2.2
  • pytz ==2023.3.post1
  • rank-bm25 ==0.2.2
  • rapidfuzz ==3.5.2
  • referencing ==0.32.0
  • regex ==2023.10.3
  • registrate ==0.1.6
  • responses ==0.18.0
  • rich ==13.7.0
  • rouge-score ==0.1.2
  • rpds-py ==0.13.2
  • ruamel.yaml ==0.18.5
  • ruamel.yaml.clib ==0.2.8
  • sacrebleu ==2.4.2
  • safetensors ==0.4.5
  • scikit-learn ==1.3.2
  • scipy ==1.11.4
  • seaborn ==0.13.1
  • semantic-version ==2.10.0
  • sentencepiece ==0.1.99
  • sentry-sdk ==1.37.0
  • setproctitle ==1.3.3
  • shellingham ==1.5.4
  • shtab ==1.6.4
  • sklearn ==0.0.post11
  • smart-open ==6.4.0
  • smmap ==5.0.1
  • sniffio ==1.3.0
  • sonar-space ==0.2.1
  • soundfile ==0.12.1
  • sox ==1.5.0
  • spacy ==3.7.4
  • spacy-language-detection ==0.2.1
  • spacy-legacy ==3.0.12
  • spacy-loggers ==1.0.5
  • sqlitedict ==2.1.0
  • srsly ==2.4.8
  • starlette ==0.27.0
  • sympy ==1.12
  • tabledata ==1.3.3
  • tabulate ==0.9.0
  • tbb ==2021.13.1
  • tcolorpy ==0.1.6
  • tensor-parallel ==2.0.0
  • thinc ==8.2.3
  • threadpoolctl ==3.2.0
  • tokenizers ==0.15.0
  • tomlkit ==0.12.0
  • toolz ==0.12.0
  • torch ==2.2.2
  • torchaudio ==2.2.2
  • torcheval ==0.0.7
  • torchmetrics ==0.10.3
  • torchvision ==0.16.0
  • tqdm ==4.66.6
  • tqdm-multiprocess ==0.0.11
  • transformers ==4.35.2
  • translate-toolkit ==3.11.1
  • triton ==2.2.0
  • trl ==0.7.4
  • typepy ==1.3.2
  • typer ==0.9.0
  • tyro ==0.5.17
  • tzdata ==2023.3
  • unbabel-comet ==2.2.2
  • urllib3 ==1.26.13
  • uvicorn ==0.24.0.post1
  • vobject ==0.9.6.1
  • wandb ==0.16.0
  • wasabi ==1.1.2
  • weasel ==0.3.4
  • websockets ==11.0.3
  • word2number ==1.1
  • xlrd ==2.0.1
  • xxhash ==3.4.1
  • yarl ==1.9.3
  • zhon ==2.0.2
  • zstandard ==0.23.0