Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/aiot-mlsys-lab/meda

[NAACL 2025🔥] Official implementation of "MEDA: Dynamic KV Cache Allocation for Efficient Multimodal Long-Context Inference"

https://github.com/aiot-mlsys-lab/meda

Science Score: 23.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ✓
    Academic publication links
    Links to: arxiv.org
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (11.6%) to scientific vocabulary
Last synced: 9 months ago · JSON representation

Repository

[NAACL 2025🔥] Official implementation of "MEDA: Dynamic KV Cache Allocation for Efficient Multimodal Long-Context Inference"

Basic Info
  • Host: GitHub
  • Owner: AIoT-MLSys-Lab
  • License: mit
  • Language: Python
  • Default Branch: main
  • Homepage:
  • Size: 23.8 MB
Statistics
  • Stars: 3
  • Watchers: 1
  • Forks: 0
  • Open Issues: 0
  • Releases: 0
Created over 1 year ago · Last pushed over 1 year ago
Metadata Files
Readme License

Readme.md

MEDA: Dynamic KV Cache Allocation for Efficient Multimodal Long-Context Inference

The codebase for NAACL 2025 paper: MEDA: Dynamic KV Cache Allocation for Efficient Multimodal Long-Context Inference.

📃 [Paper] • 💻 [Github]

If you find our project helpful, please give us a star ⭐ on GitHub to stay updated.


Quick Step to Setup Environment

We recommend using Anaconda to create a new environment and install the required packages. You can create a new environment and install the required packages using the following commands: ```bash

conda create -n meda python=3.10 conda activate meda pip install -r requirements.txt pip install --upgrade pip
```

Quick Step to Run the Code

You can run the inference code using the following command to run the multimodal long-context sample: bash cd Dynamic-MLLMs bash download_dataset.sh bash ./scripts/meda_eval.sh

For tasks related to video tasks of lm-evaluation-harness GitHub Repository,
we recommend using the latest version by running:

bash git clone https://github.com/EvolvingLMMs-Lab/lmms-eval.git Then, follow the installation instructions provided in the repository.

Citation

bibtex @article{wan2025meda, title={MEDA: Dynamic KV Cache Allocation for Efficient Multimodal Long-Context Inference}, author={Wan, Zhongwei and Shen, Hui and Wang, Xin and Liu, Che and Mai, Zheda and Zhang, Mi}, journal={arXiv preprint arXiv:2502.17599}, year={2025} }

Owner

  • Name: OSU AIoT-MLSys Lab
  • Login: AIoT-MLSys-Lab
  • Kind: organization
  • Location: United States of America

GitHub Events

Total
  • Issues event: 3
  • Watch event: 9
  • Issue comment event: 2
  • Public event: 1
  • Push event: 6
  • Fork event: 2
Last Year
  • Issues event: 3
  • Watch event: 9
  • Issue comment event: 2
  • Public event: 1
  • Push event: 6
  • Fork event: 2

Issues and Pull Requests

Last synced: 10 months ago

All Time
  • Total issues: 1
  • Total pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Total issue authors: 1
  • Total pull request authors: 0
  • Average comments per issue: 0.0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 1
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 1
  • Pull request authors: 0
  • Average comments per issue: 0.0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • Mr-xiu (1)
  • Ethan-Yang02 (1)
  • GaoMengGladys (1)
Pull Request Authors
Top Labels
Issue Labels
Pull Request Labels

Dependencies

requirements.txt pypi
  • Babel ==2.13.1
  • GitPython ==3.1.40
  • Jinja2 ==3.1.2
  • Levenshtein ==0.24.0
  • MarkupSafe ==2.1.3
  • Pillow ==9.3.0
  • PyYAML ==6.0.1
  • QtPy ==2.4.1
  • Send2Trash ==1.8.2
  • accelerate ==0.24.0
  • aiofiles ==23.2.1
  • aiohttp ==3.9.1
  • aiosignal ==1.3.1
  • alpaca-eval ==0.3.6
  • altair ==5.1.2
  • annotated-types ==0.6.0
  • antlr4-python3-runtime ==4.9.3
  • anyio ==3.7.1
  • appdirs ==1.4.4
  • argon2-cffi ==23.1.0
  • argon2-cffi-bindings ==21.2.0
  • arrow ==1.3.0
  • async-lru ==2.0.4
  • async-timeout ==4.0.3
  • attrs ==23.1.0
  • beautifulsoup4 ==4.12.2
  • bitsandbytes ==0.41.1
  • bleach ==6.1.0
  • blessed ==1.20.0
  • cachetools ==5.3.2
  • certifi ==2023.11.17
  • cffi ==1.16.0
  • charset-normalizer ==3.3.2
  • click ==8.1.7
  • colorama ==0.4.6
  • contourpy ==1.2.0
  • cycler ==0.12.1
  • datasets ==2.15.0
  • deepspeed ==0.12.6
  • defusedxml ==0.7.1
  • diffusers ==0.24.0
  • dill ==0.3.7
  • distro ==1.8.0
  • docker-pycreds ==0.4.0
  • einops ==0.7.0
  • et-xmlfile ==1.1.0
  • fastapi ==0.104.1
  • fastjsonschema ==2.19.0
  • ffmpy ==0.3.1
  • filelock ==3.13.1
  • fire ==0.5.0
  • flash-attn ==2.3.4
  • fonttools ==4.45.0
  • fqdn ==1.5.1
  • frozenlist ==1.4.0
  • fsspec ==2023.10.0
  • gitdb ==4.0.11
  • gpustat ==1.1.1
  • gradio ==3.35.0
  • gradio_client ==0.7.0
  • h11 ==0.14.0
  • hjson ==3.1.0
  • httpcore ==1.0.2
  • httpx ==0.25.1
  • huggingface-hub ==0.20.3
  • icecream ==2.1.3
  • idna ==3.4
  • importlib-metadata ==7.0.0
  • importlib-resources ==6.1.1
  • iopath ==0.1.10
  • ipywidgets ==8.1.1
  • isoduration ==20.11.0
  • joblib ==1.3.2
  • json5 ==0.9.14
  • jsonlines ==4.0.0
  • jsonpointer ==2.4
  • jsonschema ==4.20.0
  • jsonschema-specifications ==2023.11.1
  • kiwisolver ==1.4.5
  • linkify-it-py ==2.0.3
  • markdown-it-py ==2.2.0
  • matplotlib ==3.8.2
  • mdit-py-plugins ==0.3.3
  • mdurl ==0.1.2
  • mistune ==3.0.2
  • mpmath ==1.3.0
  • multidict ==6.0.4
  • multiprocess ==0.70.15
  • nbclient ==0.9.0
  • nbconvert ==7.11.0
  • nbformat ==5.9.2
  • networkx ==3.0
  • ninja ==1.11.1.1
  • notebook ==7.0.6
  • notebook_shim ==0.2.3
  • numpy ==1.26.2
  • nvidia-ml-py ==12.535.133
  • nvitop ==1.3.1
  • omegaconf ==2.3.0
  • openai ==1.3.5
  • opencv-python ==4.9.0.80
  • openpyxl ==3.1.2
  • orjson ==3.9.10
  • overrides ==7.4.0
  • pandas ==2.1.3
  • pandas-compat ==0.1.1
  • pandocfilters ==1.5.0
  • peft ==0.4.0
  • plotly ==5.19.0
  • portalocker ==2.8.2
  • prometheus-client ==0.19.0
  • protobuf ==4.25.1
  • psutil *
  • py-cpuinfo ==9.0.0
  • pyarrow ==14.0.1
  • pyarrow-hotfix ==0.6
  • pycparser ==2.21
  • pydantic ==2.5.1
  • pydantic_core ==2.14.3
  • pydub ==0.25.1
  • pynvml ==11.5.0
  • pyparsing ==3.1.1
  • pypinyin ==0.50.0
  • python-Levenshtein ==0.24.0
  • python-dotenv ==1.0.0
  • python-json-logger ==2.0.7
  • python-multipart ==0.0.6
  • pytz ==2023.3.post1
  • qtconsole ==5.5.1
  • rapidfuzz ==3.6.1
  • referencing ==0.31.0
  • regex ==2023.10.3
  • requests ==2.31.0
  • retrying ==1.3.4
  • rfc3339-validator ==0.1.4
  • rfc3986-validator ==0.1.1
  • rich ==13.7.0
  • rouge ==1.0.1
  • rpds-py ==0.13.1
  • runipy ==0.1.5
  • safetensors ==0.4.2
  • scikit-learn ==1.3.2
  • scipy ==1.11.4
  • semantic-version ==2.10.0
  • sentencepiece ==0.1.99
  • sentry-sdk ==1.35.0
  • setproctitle ==1.3.3
  • shellingham ==1.5.4
  • shortuuid ==1.0.11
  • smmap ==5.0.1
  • sniffio ==1.3.0
  • soupsieve ==2.5
  • starlette ==0.27.0
  • sympy ==1.12
  • tenacity ==8.2.3
  • termcolor ==2.4.0
  • terminado ==0.18.0
  • threadpoolctl ==3.2.0
  • tiktoken ==0.5.1
  • timm ==0.9.12
  • tinycss2 ==1.2.1
  • tokenizers ==0.15.2
  • tomli ==2.0.1
  • tomlkit ==0.12.0
  • toolz ==0.12.0
  • torch ==2.1.1
  • torchaudio ==2.1.1
  • torchvision ==0.16.1
  • tornado ==6.4
  • tqdm ==4.66.1
  • transformers ==4.37.0
  • transformers-stream-generator ==0.0.4
  • triton ==2.1.0
  • typer ==0.9.0
  • types-python-dateutil ==2.8.19.14
  • tzdata ==2023.3
  • uc-micro-py ==1.0.3
  • uri-template ==1.3.0
  • urllib3 ==2.1.0
  • uvicorn ==0.24.0.post1
  • wandb ==0.16.0
  • webcolors ==1.13
  • webencodings ==0.5.1
  • websocket-client ==1.7.0
  • websockets ==11.0.3
  • widgetsnbextension ==4.0.9
  • word2number ==1.1
  • xformers ==0.0.23
  • xxhash ==3.4.1
  • yarl ==1.9.3
  • zipp ==3.17.0