Data

Tools

Indexes

Applications

Experiments

Open Source Science

memit_swh

https://github.com/eric8932/memit_swh

Science Score: 44.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ✓
    CITATION.cff file
    Found CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (9.4%) to scientific vocabulary
Last synced: 6 months ago · JSON representation ·

Repository

Basic Info
  • Host: GitHub
  • Owner: Eric8932
  • License: mit
  • Language: Python
  • Default Branch: main
  • Size: 28.6 MB
Statistics
  • Stars: 0
  • Watchers: 1
  • Forks: 0
  • Open Issues: 0
  • Releases: 0
Created almost 3 years ago · Last pushed over 2 years ago
Metadata Files
Readme License Citation

README.md

MEMIT: Mass-Editing Memory in a Transformer

Editing thousands of facts into a transformer memory at once.

Table of Contents

Installation

We recommend conda for managing Python, CUDA, and PyTorch; pip is for everything else. To get started, simply install conda and run: bash CONDA_HOME=$CONDA_HOME ./scripts/setup_conda.sh

$CONDA_HOME should be the path to your conda installation, e.g., ~/miniconda3.

MEMIT Algorithm Demo

notebooks/memit.ipynb demonstrates MEMIT. The API is simple; simply specify a requested rewrite of the following form:

python request = [ { "prompt": "{} plays the sport of", "subject": "LeBron James", "target_new": { "str": "football" } }, { "prompt": "{} plays the sport of", "subject": "Michael Jordan", "target_new": { "str": "baseball" } }, ]

Other similar example(s) are included in the notebook.

Running the Full Evaluation Suite

experiments/evaluate.py can be used to evaluate any method in baselines/.

For example: python3 -m experiments.evaluate \ --alg_name=MEMIT \ --model_name=EleutherAI/gpt-j-6B \ --hparams_fname=EleutherAI_gpt-j-6B.json \ --num_edits=10000 \ --use_cache Results from each run are stored at results/<method_name>/run_<run_id> in a specific format: bash results/ |__ MEMIT/ |__ run_<run_id>/ |__ params.json |__ case_0.json |__ case_1.json |__ ... |__ case_10000.json

To summarize the results, you can use experiments/summarize.py: bash python3 -m experiments.summarize --dir_name=MEMIT --runs=run_<run1>,run_<run2>

Running python3 -m experiments.evaluate -h or python3 -m experiments.summarize -h provides details about command-line flags.

How to Cite

bibtex @article{meng2022memit, title={Mass Editing Memory in a Transformer}, author={Kevin Meng and Sen Sharma, Arnab and Alex Andonian and Yonatan Belinkov and David Bau}, journal={arXiv preprint arXiv:2210.07229}, year={2022} }

Owner

  • Name: WenH
  • Login: Eric8932
  • Kind: user

Citation (CITATION.cff) 

cff-version: 1.2.0
message: "If you use this software, please cite it as below."
preferred-citation:
  type: article
  authors:
  - family-names: "Meng"
    given-names: "Kevin"
  - family-names: "Sen Sharma"
    given-names: "Arnab"
  - family-names: "Andonian"
    given-names: "Alex"
  - family-names: "Belinkov"
    given-names: "Yonatan"
  - family-names: "Bau"
    given-names: "David"
  journal: "arXiv preprint arXiv:2210.07229"
  title: "Mass-Editing Memory in a Transformer"
  year: 2022

GitHub Events

Total
Last Year

Dependencies

baselines/mend/requirements.txt pypi
  • allennlp *
  • click ==7.1.2
  • datasets *
  • hydra-core *
  • jsonlines *
  • numpy *
  • spacy *
  • torch *
  • wandb *