https://github.com/alexisxty/llama-chinese

Llama中文社区,最好的中文Llama大模型,完全开源可商用

https://github.com/alexisxty/llama-chinese

Science Score: 10.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • CITATION.cff file
  • codemeta.json file
  • .zenodo.json file
  • DOI references
  • Academic publication links
    Links to: arxiv.org
  • Academic email domains
  • Institutional organization owner
  • JOSS paper metadata
  • Scientific vocabulary similarity
    Low similarity (4.6%) to scientific vocabulary
Last synced: 10 months ago · JSON representation

Repository

Llama中文社区,最好的中文Llama大模型,完全开源可商用

Basic Info
  • Host: GitHub
  • Owner: Alexisxty
  • Language: Python
  • Default Branch: main
  • Homepage:
  • Size: 16.3 MB
Statistics
  • Stars: 0
  • Watchers: 0
  • Forks: 0
  • Open Issues: 0
  • Releases: 0
Fork of LlamaFamily/Llama-Chinese
Created over 2 years ago · Last pushed over 2 years ago

https://github.com/Alexisxty/Llama-Chinese/blob/main/

English

Llama-Chinese

Llama

Llama

llama.family

Llama2Atom-7B



## - [ Llama](#-llama) - [ ](#-) - [ Llama2](#-llama2) - [ Atom](#-atom) - [](#) - [](#) - [](#) - [ ](#-) - [ ](#-) - [](#) - [MetaLlama2](#metallama2) - [Llama2](#llama2) - [Llama2Atom](#llama2atom) - [](#) - [FastAPI](#fastapi) - [Gradio](#gradio) - [Docker](#docker) - [ ](#-) - [ ](#-) - [Step1: ](#step1-) - [Step2: ](#step2-) - [Step3: ](#step3-) - [LoRA](#lora) - [](#) - [Step4: ](#step4-) - [LoRA](#lora-1) - [](#-1) - [ ](#-) - [ ](#-) - [TensorRT-LLM](#TensorRT-LLM) - [vLLM](#vllm) - [JittorLLMs](#jittorllms) - [lmdeploy](#lmdeploy) - [ ](#-) - [ ](#-) - [LangChain](#langchain) - [ ](#-) - [ ](#-) - [MetaLlama2](#metallama2) - [Llama](#llama) - [Llama2](#llama2) - [ ](#-) - [ ](#-) ## Llama LlamaLlama **\*Llama2\*** LLM
### Llama2 ****NLP ****Llama2 **** **** ****NLP ### ****Llama2NLP ****Llama2 **** ****Llama2 **** **** ****Llama2 ****Llama2 ### ****Llama2Llama2Llama2NLP ****Llama2
## 20240308APIAtom-1B,7B,13B 3[API](https://llama.family/docs/chat-completion-v1) 2023108JittorLLMs[JittorLLMs](#jittorllms) 2023912[Atom-7B](https://huggingface.co/FlagAlpha/Atom-7B)[Atom-7B-Chat](https://huggingface.co/FlagAlpha/Atom-7B-Chat)100B token[llama.family](https://llama.family/) 202392[](#-)[](#-) 2023828Llama2[Atom-7B](https://huggingface.co/FlagAlpha/Atom-7B)[](https://mp.weixin.qq.com/s/Bdx0JTVh1kgPn5ydYxIkEw) 2023826[FastAPI](#fastapi) 2023826MetaHugging Face[](https://github.com/LlamaFamily/Llama-Chinese/blob/main/scripts/convert2hf/README.md) 2023826[Code Llama](#-)
- 2023815[PEFT](#) - 2023814[](https://llama.family) - 202383FasterTransformervLLMGPU[](#-) - 2023731Llama2[](https://mp.weixin.qq.com/s/lExUU7z_MvgJ7tzQPF8tUQ) - 2023728[Docker](#docker) - 2023727[LangChain](#langchain) - 2023726Llama2-13B[4bit](#-) - 2023725Llama - 2023724[FlagAlpha](https://huggingface.co/FlagAlpha)Llama2-13B - 2023724[llama.family](https://llama.family/)Llama2-70B - 2023723Llama2Hugging Face[FlagAlpha](https://huggingface.co/FlagAlpha) - 2023722Llama2[llama.family](https://llama.family/)Meta - 2023721MetaLlama2 Chat[](#-) - 2023721Llama2Hugging Face - 2023720[](https://chinesellama.feishu.cn/wiki/space/7257824476874768388?ccm_open_type=lark_wiki_spaceLink) - 2023720Llama2 - 2023719Llama2 - 2023719Llama2 - 2023719Llama2
## Llama2 Hugging FaceMetaHugging Facetransformers[](https://github.com/LlamaFamily/Llama-Chinese/blob/main/scripts/convert2hf/README.md)
- Llama2-7Bhttps://pan.xunlei.com/s/VN_kR2fwuJdG1F3CoF33rwpIA1?pwd=z9kf - Llama2-7B-Chathttps://pan.xunlei.com/s/VN_kQa1_HBvV-X9QVI6jV2kOA1?pwd=xmra - Llama2-13Bhttps://pan.xunlei.com/s/VN_izibaMDoptluWodzJw4cRA1?pwd=2qqb - Llama2-13B-Chathttps://pan.xunlei.com/s/VN_iyyponyapjIDLXJCNfqy7A1?pwd=t3xw - Llama2-7B Hugging Facehttps://pan.xunlei.com/s/VN_t0dUikZqOwt-5DZWHuMvqA1?pwd=66ep - Llama2-7B-Chat Hugging Facehttps://pan.xunlei.com/s/VN_oaV4BpKFgKLto4KgOhBcaA1?pwd=ufir - Llama2-13B Hugging Facehttps://pan.xunlei.com/s/VN_yT_9G8xNOz0SDWQ7Mb_GZA1?pwd=yvgf - Llama2-13B-Chat Hugging Facehttps://pan.xunlei.com/s/VN_yA-9G34NGL9B79b3OQZZGA1?pwd=xqrg - Llama2-70B-Chat Hugging Facehttps://pan.xunlei.com/s/VNa_vCGzCy3h3N7oeFXs2W1hA1?pwd=uhxh# - CodeLlama-7bhttps://pan.baidu.com/s/1cIPzdNywWLvQI7_2QanOEQ?pwd=zfwi - CodeLlama-7b-Pythonhttps://pan.baidu.com/s/1liY8klGoDagYbpw-g-oFag?pwd=i952 - CodeLlama-7b-Instructhttps://pan.baidu.com/s/108o9_DT2E_vfSGtOnDCQVw?pwd=zkt9 - CodeLlama-13bhttps://pan.baidu.com/s/1lLaeHv0XEBv0iiZzI1dpnw?pwd=qn99 - CodeLlama-13b-Pythonhttps://pan.baidu.com/s/1OLVfvZS_oqL3oqMKwsI87w?pwd=a78k - CodeLlama-13b-Instructhttps://pan.baidu.com/s/1HyxJl4w8wElgkZRh2ATrXQ?pwd=seg6 - CodeLlama-34bhttps://pan.baidu.com/s/1vEw0pFgIkctPUN4_5_6pIQ?pwd=q8eu
## Atom **Atom**LlamaC-Eval821

ceval

AtomAtom-7BAtom-13BLlama2Atom-7BAtom-7B-Chat[Hugging Face](https://huggingface.co/FlagAlpha)[Atom-7B](#llama2atom)Atom ### AtomLlama2[ ](#-) 1T token ### Llama2G65,000/350emoji ### Atom4KPINeural Tangent Kernel NTK32K ## Llama2: | | | | ---------------------------------------------------------- | ------------------------------------------------------------ | | | | | [Wikipedia](https://github.com/goldsmith/Wikipedia) | Wikipedia | | [](https://github.com/BAAI-WuDao/Model) | 200G | | [Clue](https://github.com/CLUEbenchmark/CLUEDatasetSearch) | Clue | | | 150 | | [MNBVC](https://github.com/esbatmop/MNBVC) | MNBVC | **!** ## MetaHugging Facehttps://huggingface.co/meta-llama Llamahttps://huggingface.co/FlagAlpha ### #### MetaLlama2 Llama27B13B70BLlama2-Chat | | | | | | ---------- | ---------- | ------------------------- | --------------------- | | | Llama2-7B | meta-llama/Llama-2-7b-hf | [HuggingFace](https://huggingface.co/meta-llama/Llama-2-7b-hf) \| [](https://pan.xunlei.com/s/VN_t0dUikZqOwt-5DZWHuMvqA1?pwd=66ep) | | | Llama2-13B | meta-llama/Llama-2-13b-hf | [HuggingFace](https://huggingface.co/meta-llama/Llama-2-13b-hf) \| [](https://pan.xunlei.com/s/VN_yT_9G8xNOz0SDWQ7Mb_GZA1?pwd=yvgf) | | | Llama2-70B | meta-llama/Llama-2-70b-hf | [HuggingFace](https://huggingface.co/meta-llama/Llama-2-70b-hf) | | Chat | Llama2-7B-Chat | meta-llama/Llama-2-7b-chat-hf | [HuggingFace](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf) \| [](https://pan.xunlei.com/s/VN_oaV4BpKFgKLto4KgOhBcaA1?pwd=ufir) | | Chat | Llama2-13B-Chat | meta-llama/Llama-2-13b-chat-hf | [HuggingFace](https://huggingface.co/meta-llama/Llama-2-13b-chat-hf) \| [](https://pan.xunlei.com/s/VN_yA-9G34NGL9B79b3OQZZGA1?pwd=xqrg) | | Chat | Llama2-70B-Chat | meta-llama/Llama-2-70b-chat-hf | [HuggingFace](https://huggingface.co/meta-llama/Llama-2-70b-chat-hf) \| [](https://pan.xunlei.com/s/VNa_vCGzCy3h3N7oeFXs2W1hA1?pwd=uhxh#) | #### Llama2 Llama2-ChatLlama2LoRA[Hugging Face](https://huggingface.co/FlagAlpha)7B13B | | | | | | | ---------- | ---------- | ------------- | ----------------- | ------------------- | | | Llama2-Chinese-7b-Chat | FlagAlpha/Llama2-Chinese-7b-Chat | meta-llama/Llama-2-7b-chat-hf |[HuggingFace](https://huggingface.co/FlagAlpha/Llama2-Chinese-7b-Chat) | | | Llama2-Chinese-13b-Chat | FlagAlpha/Llama2-Chinese-13b-Chat| meta-llama/Llama-2-13b-chat-hf |[HuggingFace](https://huggingface.co/FlagAlpha/Llama2-Chinese-13b-Chat) | | LoRA | Llama2-Chinese-7b-Chat-LoRA | FlagAlpha/Llama2-Chinese-7b-Chat-LoRA | meta-llama/Llama-2-7b-chat-hf |[HuggingFace](https://huggingface.co/FlagAlpha/Llama2-Chinese-7b-Chat-LoRA) | | LoRA | Llama2-Chinese-13b-Chat-LoRA | FlagAlpha/Llama2-Chinese-13b-Chat-LoRA | meta-llama/Llama-2-13b-chat-hf |[HuggingFace](https://huggingface.co/FlagAlpha/Llama2-Chinese-13b-Chat-LoRA) | #### Llama2Atom Atom-7BAtom-7B[llama.family](https://llama.family) | | | | | | --------------- | --------------- | ------------------------------ | ------------------------------------------------------------ | | | Atom-7B | FlagAlpha/Atom-7B | [HuggingFace](https://huggingface.co/FlagAlpha/Atom-7B) \| [ModelScope](https://modelscope.cn/models/FlagAlpha/Atom-7B) \| [WiseModel](https://wisemodel.cn/models/FlagAlpha/Atom-7B) | | Chat | Atom-7B-Chat | FlagAlpha/Atom-7B-Chat | [HuggingFace](https://huggingface.co/FlagAlpha/Atom-7B-Chat) \| [ModelScope](https://modelscope.cn/models/FlagAlpha/Atom-7B-Chat) \| [WiseModel](https://wisemodel.cn/models/FlagAlpha/Atom-7B-Chat)| ### ```python import torch from transformers import AutoTokenizer, AutoModelForCausalLM device_map = "cuda:0" if torch.cuda.is_available() else "auto" model = AutoModelForCausalLM.from_pretrained('FlagAlpha/Atom-7B-Chat',device_map=device_map,torch_dtype=torch.float16,load_in_8bit=True,trust_remote_code=True,use_flash_attention_2=True) model =model.eval() tokenizer = AutoTokenizer.from_pretrained('FlagAlpha/Atom-7B-Chat',use_fast=False) tokenizer.pad_token = tokenizer.eos_token input_ids = tokenizer(['Human: \nAssistant: '], return_tensors="pt",add_special_tokens=False).input_ids if torch.cuda.is_available(): input_ids = input_ids.to('cuda') generate_input = { "input_ids":input_ids, "max_new_tokens":512, "do_sample":True, "top_k":50, "top_p":0.95, "temperature":0.3, "repetition_penalty":1.3, "eos_token_id":tokenizer.eos_token_id, "bos_token_id":tokenizer.bos_token_id, "pad_token_id":tokenizer.pad_token_id } generate_ids = model.generate(**generate_input) text = tokenizer.decode(generate_ids[0]) print(text) ``` ### FastAPI API[FastAPI](https://github.com/tiangolo/fastapi)API[API ](https://github.com/LlamaFamily/Llama-Chinese/blob/main/scripts/api/README.md) ### Gradio gradioAtom-7B
``` python examples/chat_gradio.py --model_name_or_path FlagAlpha/Atom-7B-Chat ``` ### Docker [Docker](https://github.com/LlamaFamily/Llama-Chinese/blob/main/docs/chat_gradio_guide.md) dockerdocker[chat_gradio.py](../examples/chat_gradio.py) ```bash git clone https://github.com/LlamaFamily/Llama-Chinese.git cd Llama-Chinese docker build -f docker/Dockerfile -t flagalpha/llama2-chinese:gradio . ``` docker-composechat_gradio ```bash cd Llama-Chinese/docker docker-compose up -d --build ``` ## Llama2LLaMA0.13%Llama2 - LlamaLlama2 - Llama[](https://github.com/LlamaFamily/Llama-Chinese/tree/main/data)[](#-) - [train/pretrain/pretrain.sh](https://github.com/LlamaFamily/Llama-Chinese/blob/main/train/pretrain/pretrain.sh) - [train/pretrain/pretrain_clm.py](https://github.com/LlamaFamily/Llama-Chinese/blob/main/train/pretrain/pretrain_clm.py) - [DeepSpeed](https://github.com/microsoft/DeepSpeed) - ZeRO-2 [train/pretrain/ds_config_zero2.json](https://github.com/LlamaFamily/Llama-Chinese/blob/main/train/pretrain/ds_config_zero2.json) - ZeRO-3 [train/pretrain/ds_config_zero3.json](https://github.com/LlamaFamily/Llama-Chinese/blob/main/train/pretrain/ds_config_zero3.json) - [train/pretrain/accuracy.py](https://github.com/LlamaFamily/Llama-Chinese/blob/main/train/pretrain/accuracy.py) ## LoRALoRA[LoRA: Low-Rank Adaptation of Large Language Models](https://arxiv.org/abs/2106.09685)Github[LoRA](https://github.com/microsoft/LoRA) ### Step1: [requirements.txt](https://github.com/LlamaFamily/Llama-Chinese/blob/main/requirements.txt) ### Step2: datasft - [data/train_sft.csv](https://github.com/LlamaFamily/Llama-Chinese/blob/main/data/train_sft.csv) - [data/dev_sft.csv](https://github.com/LlamaFamily/Llama-Chinese/blob/main/data/dev_sft.csv) csvtext ``` "Human: "++"\nAssistant: "+ ``` ``` Human: Assistant: ``` ### Step3: #### LoRA LoRA[train/sft/finetune_lora.sh](https://github.com/LlamaFamily/Llama-Chinese/blob/main/train/sft/finetune_lora.sh)LoRA[train/sft/finetune_clm_lora.py](https://github.com/LlamaFamily/Llama-Chinese/blob/main/train/sft/finetune_clm_lora.py)`--include localhost:0` #### [train/sft/finetune.sh](https://github.com/LlamaFamily/Llama-Chinese/blob/main/train/sft/finetune.sh)[train/sft/finetune_clm.py](https://github.com/LlamaFamily/Llama-Chinese/blob/main/train/sft/finetune_clm.py) ### Step4: #### LoRA LoRA[Llama2](#llama2)LoRA [PEFT](https://github.com/huggingface/peft)base_model_name_or_pathfinetune_model_path ```python import torch from transformers import AutoTokenizer, AutoModelForCausalLM from peft import PeftModel,PeftConfig # : finetune_model_path='FlagAlpha/Llama2-Chinese-7b-Chat-LoRA' finetune_model_path='' config = PeftConfig.from_pretrained(finetune_model_path) # : base_model_name_or_path='meta-llama/Llama-2-7b-chat' tokenizer = AutoTokenizer.from_pretrained(config.base_model_name_or_path,use_fast=False) tokenizer.pad_token = tokenizer.eos_token device_map = "cuda:0" if torch.cuda.is_available() else "auto" model = AutoModelForCausalLM.from_pretrained(config.base_model_name_or_path,device_map=device_map,torch_dtype=torch.float16,load_in_8bit=True,trust_remote_code=True,use_flash_attention_2=True) model = PeftModel.from_pretrained(model, finetune_model_path, device_map={"": 0}) model =model.eval() input_ids = tokenizer(['Human: \nAssistant: '], return_tensors="pt",add_special_tokens=False).input_ids if torch.cuda.is_available(): input_ids = input_ids.to('cuda') generate_input = { "input_ids":input_ids, "max_new_tokens":512, "do_sample":True, "top_k":50, "top_p":0.95, "temperature":0.3, "repetition_penalty":1.3, "eos_token_id":tokenizer.eos_token_id, "bos_token_id":tokenizer.bos_token_id, "pad_token_id":tokenizer.pad_token_id } generate_ids = model.generate(**generate_input) text = tokenizer.decode(generate_ids[0]) print(text) ``` #### [](#) ## [Hugging Face](https://huggingface.co/FlagAlpha)13B[FlagAlpha/Llama2-Chinese-13b-Chat](https://huggingface.co/FlagAlpha/Llama2-Chinese-13b-Chat)4bit[FlagAlpha/Llama2-Chinese-13b-Chat-4bit](https://huggingface.co/FlagAlpha/Llama2-Chinese-13b-Chat-4bit) ``` pip install git+https://github.com/PanQiWei/AutoGPTQ.git ``` ```python from transformers import AutoTokenizer from auto_gptq import AutoGPTQForCausalLM model = AutoGPTQForCausalLM.from_quantized('FlagAlpha/Llama2-Chinese-13b-Chat-4bit', device="cuda:0") tokenizer = AutoTokenizer.from_pretrained('FlagAlpha/Llama2-Chinese-13b-Chat-4bit',use_fast=False) input_ids = tokenizer(['Human: \nAssistant: '], return_tensors="pt",add_special_tokens=False).input_ids.to('cuda') generate_input = { "input_ids":input_ids, "max_new_tokens":512, "do_sample":True, "top_k":50, "top_p":0.95, "temperature":0.3, "repetition_penalty":1.3, "eos_token_id":tokenizer.eos_token_id, "bos_token_id":tokenizer.bos_token_id, "pad_token_id":tokenizer.pad_token_id } generate_ids = model.generate(**generate_input) text = tokenizer.decode(generate_ids[0]) print(text) ``` ## lmdeployFasterTransformervLLMJittorLLMs ### TensorRT-LLM [TensorRT-LLM](https://github.com/NVIDIA/TensorRT-LLM/tree/main)NVIDIA [inference-speed/GPU/TensorRT-LLM_example](https://github.com/LlamaFamily/Llama-Chinese/tree/main/inference-speed/GPU/TensorRT-LLM_example) ### vLLM [vLLM](https://github.com/vllm-project/vllm)PageAttentionHuggingFace Transformers24FasterTrainsformervLLMfp16 [inference-speed/GPU/vllm_example](https://github.com/LlamaFamily/Llama-Chinese/blob/main/inference-speed/GPU/vllm_example/README.md) ### JittorLLMs [JittorLLMs](https://github.com/Jittor/JittorLLMs)swap80%,JittorPytorch40%20% [inference-speed/GPU/JittorLLMs](https://github.com/LlamaFamily/Llama-Chinese/blob/main/inference-speed/GPU/JittorLLMs_example/README.md) ### lmdeploy [lmdeploy](https://github.com/InternLM/lmdeploy/) C++/CUDA python/gRPC/http WebUI tensor parallel fp16/weight int4/kv cache int8 [inference-speed/GPU/lmdeploy_example](https://github.com/LlamaFamily/Llama-Chinese/tree/main/inference-speed/GPU/lmdeploy_example) ## Llama2Llama2MetaLlama2-7B-ChatLlama2-13B-Chat[AtomBulb](https://github.com/AtomEcho/AtomBulb)95 Prompt5 ``` [INST] <> You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature. The answer always been translate into Chinese language. If a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information. The answer always been translate into Chinese language. <> 5 [/INST] ``` Llama2-7B-Chat[meta_eval_7B.md](assets/meta_eval_7B.md)Llama2-13B-Chat[meta_eval_13B.md](assets/meta_eval_13B.md) MetaLlama2 ChatLlama2Llama2 ## WolframAlpha LangChainLlama2LangChain[LangChain](https://github.com/langchain-ai/langchain) ### LangChain LangChainLlama2 LLM[examples/llama2_for_langchain.py](https://github.com/LlamaFamily/Llama-Chinese/blob/main/examples/llama2_for_langchain.py) ```python from llama2_for_langchain import Llama2 # FlagAlpha/Atom-7B-Chat llm = Llama2(model_name_or_path='FlagAlpha/Atom-7B-Chat') while True: human_input = input("Human: ") response = llm(human_input) print(f"Llama2: {response}") ``` ## Meta2023824Code LlamaLlama2Code LlamaPythonCode Llama - PythonCode Llama - Instruct7B13B34B | | | | | | |-----------------------|------------------------|------|------|------| | Code Llama | CodeLlama-7b | | | | | | CodeLlama-13b | | | | | | CodeLlama-34b | | | | | Code Llama - Python | CodeLlama-7b-Python | | | | | | CodeLlama-13b-Python | | | | | | CodeLlama-34b-Python | | | | | Code Llama - Instruct | CodeLlama-7b-Instruct | | | | | | CodeLlama-13b-Instruct | | | | | | CodeLlama-34b-Instruct | | | | Code Llama[](#-Llama2)[llama.family](https://llama.family/)Code LlamaGithub[codellama](https://github.com/facebookresearch/codellama) ## ### Meta[Llama2](https://ai.meta.com/llama) MetaLLaMAMetaLlama2Llama27B13B70BLLaMALlama22token20484096Llama2 Chat100ChatGPT ### Llama * [LLaMA: Open and Efficient Foundation Language Models](https://arxiv.org/abs/2302.13971) * [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://arxiv.org/abs/2307.09288) * [Code Llama: Open Foundation Models for Code](https://ai.meta.com/research/publications/code-llama-open-foundation-models-for-code/) ### Llama2

## [AtomEcho](https://github.com/AtomEcho) [Coremesh](https://coremesh.net) @xzsGenius Llama2 @Z PotentialsLlama2 ## GitHub Issueissue [](https://chinesellama.feishu.cn/wiki/space/7257824476874768388?ccm_open_type=lark_wiki_spaceLink)

Wechat

Wechat

Owner

  • Login: Alexisxty
  • Kind: user

GitHub Events

Total
Last Year