Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/buaadreamer/mllm-finetuning-demo

使用LLaMA-Factory微调多模态大语言模型的示例代码 Demo of Finetuning Multimodal LLM with LLaMA-Factory

https://github.com/buaadreamer/mllm-finetuning-demo

Science Score: 13.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (1.5%) to scientific vocabulary

Keywords

finetune-llm huggingface-datasets llama-factory llava lora mllm paligemma pretraining supervised-finetuning transformers yi-vl
Last synced: 5 months ago · JSON representation

Repository

使用LLaMA-Factory微调多模态大语言模型的示例代码 Demo of Finetuning Multimodal LLM with LLaMA-Factory

Basic Info
  • Host: GitHub
  • Owner: BUAADreamer
  • Language: Python
  • Default Branch: master
  • Homepage:
  • Size: 61.5 KB
Statistics
  • Stars: 32
  • Watchers: 1
  • Forks: 2
  • Open Issues: 2
  • Releases: 0
Topics
finetune-llm huggingface-datasets llama-factory llava lora mllm paligemma pretraining supervised-finetuning transformers yi-vl
Created over 1 year ago · Last pushed over 1 year ago
Metadata Files
Readme

README.md

MLLM-Finetuning-Demo

安装LLaMA-Factory

shell git clone https://github.com/hiyouga/LLaMA-Factory.git cd LLaMA-Factory pip install -e .[torch,metrics] cd .. # 回到项目根目录

预训练

LLaVA中的特征对齐，冻结language_model和vision_tower，只微调multi_modal_projector。 shell CUDA_VISIBLE_DEVICES=0 llamafactory-cli train config/llava_pt.yaml

微调

shell CUDA_VISIBLE_DEVICES=0 llamafactory-cli train config/llava_lora_sft.yaml

网页聊天

shell CUDA_VISIBLE_DEVICES=0 llamafactory-cli webchat \ --model_name_or_path llava-hf/llava-1.5-7b-hf \ --adapter_name_or_path saves/llava1_5-7b/lora/sft \ --template llava

上传数据集到Huggingface

请在 upload_dataset.py 中替换您自己的key.

shell python3 upload_dataset.py

导出和上传模型到huggingface

请在 config/llava_lora_sft_export.yaml 中替换您自己的 export_hub_model_id 和 hf_hub_token.

shell CUDA_VISIBLE_DEVICES=0 llamafactory-cli export config/llava_lora_sft_export.yaml

Owner

  • Login: BUAADreamer
  • Kind: user
  • Location: Beijing
  • Company: Beihang University

GitHub Events

Total
  • Watch event: 30
Last Year
  • Watch event: 30

Issues and Pull Requests

Last synced: 11 months ago

All Time
  • Total issues: 2
  • Total pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Total issue authors: 1
  • Total pull request authors: 0
  • Average comments per issue: 0.0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 2
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 1
  • Pull request authors: 0
  • Average comments per issue: 0.0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • Shajiu (2)
Pull Request Authors
Top Labels
Issue Labels
Pull Request Labels