https://github.com/alixunxing/chinese-llama-alpaca

中文LLaMA&Alpaca大语言模型+本地CPU/GPU部署 (Chinese LLaMA & Alpaca LLMs)
Science Score: 10.0%

This score indicates how likely this project is to be science-related based on various indicators:
○
CITATION.cff file
○
codemeta.json file
○
.zenodo.json file
○
DOI references
✓
Academic publication links
Links to: arxiv.org
○
Academic email domains
○
Institutional organization owner
○
JOSS paper metadata
○
Scientific vocabulary similarity
Low similarity (5.4%) to scientific vocabulary
Last synced: 10 months ago · JSON representation
Repository

中文LLaMA&Alpaca大语言模型+本地CPU/GPU部署 (Chinese LLaMA & Alpaca LLMs)
Basic Info

Host: GitHub
Owner: alixunxing
License: apache-2.0
Default Branch: main
Homepage: https://github.com/ymcui/Chinese-LLaMA-Alpaca/wiki
Size: 10.7 MB
Statistics

Stars: 0
Watchers: 0
Forks: 0
Open Issues: 0
Releases: 0
Fork of ymcui/Chinese-LLaMA-Alpaca
Created about 3 years ago · Last pushed about 3 years ago
https://github.com/alixunxing/Chinese-LLaMA-Alpaca/blob/main/

[****](./README.md) | [**English**](./README_EN.md)


    

    
    



    
    
    
    
    




ChatGPTGPT-4Large Language Model, LLMAGI

NLP**LLaMAAlpaca****LLaMA**Alpaca

******

****

-  LLaMA 
-  LLaMA7B13B
-  Alpaca7B13B
-  PCCPU/GPU

 7BCPUApple M1 Max

![](./pics/screencast.gif)

----

[VLE](https://github.com/iflytek/VLE) | [MiniRBT](https://github.com/iflytek/MiniRBT) | [LERT](https://github.com/ymcui/LERT) | [PERT](https://github.com/ymcui/PERT) | [MacBERT](https://github.com/ymcui/MacBERT) | [ELECTRA](https://github.com/ymcui/Chinese-ELECTRA) | [XLNet](https://github.com/ymcui/Chinese-XLNet) | [BERT](https://github.com/ymcui/Chinese-BERT-wwm) | [TextBrewer](https://github.com/airaria/TextBrewer) | [TextPruner](https://github.com/airaria/TextPruner)

## 

**[2023/04/13] Release v2.1HuggingFacetext-generation-webui[Release Note](https://github.com/ymcui/Chinese-LLaMA-Alpaca/releases/tag/v2.1)**

[2023/04/07] Release v2.013BLLaMAAlpaca[Release Note](https://github.com/ymcui/Chinese-LLaMA-Alpaca/releases/tag/v2.0)

[2023/04/03] notebookColab Pro(+)[](#)

[2023/03/31] Release v1.1llama.cpp[Release Note](https://github.com/ymcui/Chinese-LLaMA-Alpaca/releases/tag/v1.1)

[2023/03/28] LLaMAAlpaca7B

## 
|                                   |                                                          |
| ------------------------------------- | ------------------------------------------------------------ |
| [](#)        | LLaMAAlpaca                |
| [](#) | LoRALLaMA |
| [](#) |  |
| [](#) |              |
| [](#) | LLaMAAlpaca |
| [](#) |  |
| [FAQ](#FAQ) |  |


## 

### 

Facebook[LLaMA](https://github.com/facebookresearch/llama)Facebook**LoRA**LLaMA

LLaMA/Alpaca LoRALLaMA^[1][](#)

### LLaMA

LLaMA[](#)

|           |  |                 | ^[2] |                         LoRA                         | SHA256^[3] |
| :---------------- | :--: | :-------------------------: | :----------------: | :----------------------------------------------------------: | :------------------: |
| Chinese-LLaMA-7B  |  | LLaMA-7B^[1]  |        770M        | [[]](https://pan.baidu.com/s/1oORTdpr2TvlkxjpyWtb5Sw?pwd=33hb)
[[Google Drive]](https://drive.google.com/file/d/1iQp9T-BHjBjIrFWXq_kIm_cyNmpvv5WN/view?usp=sharing) |  39b86b......fe0e60  |
| Chinese-LLaMA-13B |  | LLaMA-13B^[1] |         1G         | [[]](https://pan.baidu.com/s/1BxFhYhDMipW7LwI58cGmQQ?pwd=ef3t)
[[Google Drive]](https://drive.google.com/file/d/12q9EH4mfKRnoKlbkkhzv1xDwWnroo9VS/view?usp=sharing) |  3d6dee......e5199b  |


### Alpaca

AlpacaLLaMA[](#)ChatGPTAlpacaLLaMA

|            |      |                 | ^[2] |                         LoRA                         | SHA256^[3] |
| :----------------- | :------: | :-------------------------: | :----------------: | :----------------------------------------------------------: | :------------------: |
| Chinese-Alpaca-7B  |  | LLaMA-7B^[1]  |        790M        | [[]](https://pan.baidu.com/s/1xV1UXjh1EPrPtXg6WyG7XQ?pwd=923e)
[[Google Drive]](https://drive.google.com/file/d/1JvFhBpekYiueWiUL3AF1TtaWDb3clY5D/view?usp=sharing) |  9bb5b6......ce2d87  |
| Chinese-Alpaca-13B |  | LLaMA-13B^[1] |        1.1G        | [[]](https://pan.baidu.com/s/1wYoSF58SnU9k0Lndd5VEYg?pwd=mm8i)
[[Google Drive]](https://drive.google.com/file/d/1gzMc0xMCpXsXmU1uxFlgQ8VRnWNtDjD8/view?usp=share_link) |  45c92e......682d91  |

### Model Hub

Model Hub[transformers](https://github.com/huggingface/transformers)[PEFT](https://github.com/huggingface/peft)LLaMAAlpaca LoRA`.from_pretrained()`

|              |                        |                                                          |
| ------------------ | :--------------------------------: | :----------------------------------------------------------: |
| Chinese-LLaMA-7B   |  ziqingyang/chinese-llama-lora-7b  | [Model Hub Link](https://huggingface.co/ziqingyang/chinese-llama-lora-7b) |
| Chinese-LLaMA-13B  | ziqingyang/chinese-llama-lora-13b  | [Model Hub Link](https://huggingface.co/ziqingyang/chinese-llama-lora-13b) |
| Chinese-Alpaca-7B  | ziqingyang/chinese-alpaca-lora-7b  | [Model Hub Link](https://huggingface.co/ziqingyang/chinese-alpaca-lora-7b) |
| Chinese-Alpaca-13B | ziqingyang/chinese-alpaca-lora-13b | [Model Hub Link](https://huggingface.co/ziqingyang/chinese-alpaca-lora-13b) |

### 

**[1]** LLaMA[LLaMA](https://github.com/facebookresearch/llama)[PR](https://github.com/facebookresearch/llama/pull/73/files)

**[2]** LLaMA

**[3]** SHA256[SHA256.md](./SHA256.md)

Chinese-LLaMA-7B

```
chinese_llama_lora_7b/
  - adapter_config.json		# LoRA
  - adapter_model.bin		# LoRA
  - special_tokens_map.json	# special_tokens_map
  - tokenizer_config.json	# tokenizer
  - tokenizer.model		# tokenizer 
```

4-bit

|                     |   7B   |  13B   |   33B   |   65B   |
| :------------------ | :----: | :----: | :-----: | :-----: |
| FP16  | 13 GB  | 24 GB  |  60 GB  | 120 GB  |
| 4-bit | 3.9 GB | 7.8 GB | 19.5 GB | 38.5 GB |

## 

### 

**Colab7B13B**

Google ColabProNotebook

- LLaMA/Alpaca7B13B
- Google Drive
- Google Colab`notebooks`

### 

1. 7B13-15G
2. LoRA[SHA256.md](./SHA256.md)LLaMA`tokenizer.model``tokenizer_checklist.chk``consolidated.*.pth``params.json`
3. 
   - `transformers`4.28.0
   - `sentencepiece`0.1.97
   - `peft`0.2.0
   - python3.9

```bash
pip install transformers
pip install sentencepiece
pip install peft
```

*FacebookHuggingFace`decapoda-research/llama-7b-hf`use at your own risk*


### Step 1: LLaMAHF

[transformers](https://huggingface.co/docs/transformers/installation#install-from-source)[convert_llama_weights_to_hf.py](https://github.com/huggingface/transformers/blob/main/src/transformers/models/llama/convert_llama_weights_to_hf.py)LLaMAHuggingFaceLLaMA`tokenizer.model``--input_dir``${input_dir}/${model_size}``--output_dir`HF

```bash
python src/transformers/models/llama/convert_llama_weights_to_hf.py \
    --input_dir path_to_original_llama_root_dir \
    --model_size 7B \
    --output_dir path_to_original_llama_hf_dir
```

### Step 2: LoRA

LLaMAHFLoRA

- PyTorch`.pth``merge_llama_with_chinese_lora.py`
  - [llama.cpp](#llamacpp)

- HuggingFace`.bin``merge_llama_with_chinese_lora_to_hf.py`@sgsdxzy 
  - [Transformers](#transformers)
  - [text-generation-webui](#text-generation-webui)


PyTorch

```bash
python scripts/merge_llama_with_chinese_lora.py \
    --base_model path_to_original_llama_hf_dir \
    --lora_model path_to_chinese_llama_or_alpaca_lora \
    --output_dir path_to_output_dir 
```



- `--base_model`HFLLaMAStep 1
- `--lora_model`LLaMA/Alpaca LoRA[Model Hub](#Model-Hub)
- `--output_dir``./`
- `--offload_dir`offload

## 



- [llama.cpp](#llamacpp)CPU
- [Transformers](#Transformers)transformersCPU/GPU
- [text-generation-webui](#text-generation-webui)UI

### llama.cpp

[llama.cpp](https://github.com/ggerganov/llama.cpp)MacOSLinux**CPU**WindowscmakeWindows[FAQ#6](https://github.com/ymcui/Chinese-LLaMA-Alpaca/tree/main#FAQ)**AlpacaFP16** Alpaca-7B

1. 7B13G
2. 4-bit7B4-6G
3. `make`MacOS/Linux`cmake`Windows
4. [llama.cpp](https://github.com/ggerganov/llama.cpp)Python 3.9~3.11


#### Step 1: llama.cpp

llama.cpp`./main``./quantize`

```bash
git clone https://github.com/ggerganov/llama.cpp && cd llama.cpp && make
```

####  Step 2: 

[](#)`.pth``tokenizer.model``zh-models``consolidated.*.pth``params.json``zh-models/7B`LLaMAAlpaca`tokenizer.model`[](#)

```
llama.cpp/zh-models/
   - 7B/
     - consolidated.00.pth
     - params.json
   - tokenizer.model
```

`.pth`ggmlFP16`zh-models/7B/ggml-model-f16.bin`

```bash
python convert.py zh-models/7B/
```

FP164-bit`zh-models/7B/ggml-model-q4_0.bin`

```bash
./quantize ./zh-models/7B/ggml-model-f16.bin ./zh-models/7B/ggml-model-q4_0.bin 2
```

`3``q4_1``q4_1``q4_0`[llama.cpp#PPL](https://github.com/ggerganov/llama.cpp#perplexity-measuring-model-quality)

#### Step 3: 

`./main``-m`4-bitFP16GGML

```bash
./main -m zh-models/7B/ggml-model-q4_0.bin --color -f prompts/alpaca.txt -ins -c 2048 --temp 0.2 -n 256 --repeat_penalty 1.3
```
 `>` prompt`cmd/ctrl+c``\``./main -h`

```
-ins ChatGPT
-f promptalpacaprompts/alpaca.txt
-c 512
-n 128
-b batch size8
-t 4
--repeat_penalty 
--temp 
--top_p, top_k 
```

### Transformers

Python[scripts/inference_hf.py](scripts/inference_hf.py) CPUGPUChinese-Alpaca-7B

```bash
CUDA_VISIBLE_DEVICES={device_id} python scripts/inference_hf.py \
    --base_model path_to_original_llama_hf_dir \
    --lora_model path_to_chinese_llama_or_alpaca_lora \
    --with_prompt \
    --interactive
```

`merge_llama_with_chinese_lora_to_hf.py`lora`--lora_model`

```bash
CUDA_VISIBLE_DEVICES={device_id} python scripts/inference_hf.py \
    --base_model path_to_merged_llama_or_alpaca_hf_dir \
    --with_prompt \
    --interactive
```



* `{device_id}`CUDACPU
* `--base_model {base_model} `**HF**LLaMAPyTorch[HF](#step-1-llamahf)
* `--lora_model {lora_model}` LLaMA/Alpaca LoRA[Model Hub](#Model-Hub)`--base_model`
* `--tokenizer_path {tokenizer_path}`tokenizer`--lora_model``--lora_model``--base_model`
* `--with_prompt`prompt**Alpaca**
* `--interactive`****llama.cpp
* `--data_file {file_name}``file_name`
* `--predictions_file {file_name}`json`file_name`



- llama.cpp
- 
- CPU7B32GBGPU7B20GB

### text-generation-webui

[text-generation-webui](https://github.com/oobabooga/text-generation-webui)****

```bash
# text-generation-webui
git clone https://github.com/oobabooga/text-generation-webui
cd text-generation-webui
pip install -r requirements.txt

# loraloras
ls loras/chinese-alpaca-lora-7b
adapter_config.json  adapter_model.bin  special_tokens_map.json  tokenizer_config.json  tokenizer.model

# HuggingFacellama-7Bmodels
ls models/llama-7b-hf
pytorch_model-00001-of-00002.bin pytorch_model-00002-of-00002.bin config.json pytorch_model.bin.index.json generation_config.json

# loratokenizermodels/llama-7b-hf
cp loras/chinese-alpaca-lora-7b/tokenizer.model models/llama-7b-hf/
cp loras/chinese-alpaca-lora-7b/special_tokens_map.json models/llama-7b-hf/
cp loras/chinese-alpaca-lora-7b/tokenizer_config.json models/llama-7b-hf/

# /modules/LoRA.py28
shared.model.resize_token_embeddings(len(shared.tokenizer))
shared.model = PeftModel.from_pretrained(shared.model, Path(f"{shared.args.lora_dir}/{lora_name}"), **params)

# https://github.com/oobabooga/text-generation-webui/wiki/Using-LoRAs
python server.py --model llama-7b-hf --lora chinese-alpaca-lora-7b
```

## 

promptAlpaca-7BAlpaca-13B[examples/README.md](./examples/README.md)

***4-bit***

|          |                                |  | Alpaca-7B | Alpaca-13B |
| ---------------- | :------------------------------------: | :----: | :-----------: | :------------: |
| ****    |                   -                    |  160   |    **49**     |    **71**    |
|          |            [QA.md](./examples/QA.md)            |   20   |      53       |    **77**    |
|        |           [OQA.md](./examples/OQA.md)           |   20   |      64       |    **73**    |
|    |     [REASONING.md](./examples/REASONING.md)     |   20   |      23       |    **50**    |
|  |    [LITERATURE.md](./examples/LITERATURE.md)    |   20   |      31       |    **54**    |
|  | [ENTERTAINMENT.md](./examples/ENTERTAINMENT.md) |   20   |      36       |    **65**    |
|      |    [GENERATION.md](./examples/GENERATION.md)    |   15   |      65       |    **78**    |
|          |   [TRANSLATION.md](./examples/TRANSLATION.md)   |   15   |      63       |    **79**    |
|          |      [DIALOGUE.md](./examples/DIALOGUE.md)      |   10   |      80       |    **83**    |
|          |          [CODE.md](./examples/CODE.md)          |   10   |      27       |    **49**    |
|        |        [ETHICS.md](./examples/ETHICS.md)        |   10   |      50       |   **100**    |





|            | 7B                                                       | 13B                                                      |
| -------------- | ------------------------------------------------------------ | ------------------------------------------------------------ |
|        | > 
**
> 
**
> 
** | > 

> 

> 
30~  |
|        | > 
**
> 
**
> 
**
> 
**
> 
** | > 

> 
 
> 
798
> 

> 
 |
|        | > 
**
> 
**
> 
**
> 
*1991* | > 

>  
  
> 
 
> 
  1986426
> 
  1979323 |
| githubgitlab | > github
GitHubGitHubGitHub
> gitgithub
GithubGitGitGitGithub
> githubgitlab
Git GitHub  GitLab 
> 
  GitHub  GitLab 
> github
  GitHub  GitLab  | > github
GitHub
> gitgithub
 git  github GitHUB  git  Git  GitHub
> githubgitlab
OSSgitlab  OSS Github  OSS  gitlab  Github
> 
GitHubgit lab
> github
 github  GitLab GitHub  |
|          | > 

> 
AIShape of Youby Ed Sheeran Bohemian Rhapsody by Queen
> Shape of you
Ed Sheeran
> 
Shape of YouPhotographPerfectSomeone Like You | > 

> Shape of You
Ed Sheeran
> 
Thinking Out LoudPhotographs
> 
Ariana GrandeThank U, NextTaylor SwiftLover
> 
 |







| Prompt                            | 7B                                                       | 13B                                                      |
| --------------------------------- | ------------------------------------------------------------ | ------------------------------------------------------------ |
|                       | 
1.
2.
3.
4.15-20 | 1. 
2. 
3. 
4. 
5. 
6.  |
|       |    |            |
|     |  |  |
| 100 | 100C100C             | 1013     |
|     | Rntgen1901X | 1914 |








| Prompt                                                   | 7B                                                       | 13B                                                      |
| -------------------------------------------------------- | ------------------------------------------------------------ | ------------------------------------------------------------ |
|                                  | 
1.
2.
3.
4. | 1. 
2. 
3. 
4. 
5.  |
|                                        |  |  |
|                      |  |  |
|  | 1.  - 
2.  - 
3. -  | Isaac Newton

 (Charles Darwin)

(Albert Einstein): |
|                            |  |  |








| Prompt                                                       | 7B                                                       | 13B                                                      |
| ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ |
|  |  | 





 |
|  |  |  |
|                | XX









 | 







 |









| Prompt                                                       | 7B                                                     | 13B                                                      |
| ------------------------------------------------------------ | ---------------------------------------------------------- | ------------------------------------------------------------ |
|                                  |  |  |
|  |                                |                                          |
|                      |                      |                            |
|                                  |      |  |
|                                    | Milo Torm          | Alexandre Dumas pre       |








| Prompt                                               | 7B                                                       | 13B                                                      |
| ---------------------------------------------------- | ------------------------------------------------------------ | ------------------------------------------------------------ |
|  | AI                   |  |
|          | Fukuromochan |  |
| doge                               | DogeoTsuneyama | "Doge"  |
| B                                | "Baidu Tudou (BT) culture", or "B"B | "Bilibili Culture ( )"bilibili |
|                            | 1.The Godfather (1972)(Francis Ford Coppola) 2. Shanghai Calling (2011), (Andrew Haigh) 3. Catch-22 (1958),(Mickey Rooney Heathcliff) |  - The Godfather - The Shawshank Redemption - (Gone with the Wind) - (Schindler's List) - (Interstellar) |






alignment



| Prompt                                 | 7B                                                       | 13B                                                      |
| -------------------------------------- | ------------------------------------------------------------ | ------------------------------------------------------------ |
|                          |                |  |
|                        | 
1.
2.
3.
4.
5. |  |
|                    |  |                  |
|                |                            |                                  |
|  | 
1. 
2. 
3. 
4.  |                |





## 

transformers[run_clm.py](https://github.com/huggingface/transformers/blob/main/examples/pytorch/language-modeling/run_clm.py)[Stanford Alpaca](https://github.com/tatsu-lab/stanford_alpaca)

### 

LLaMALLaMA

- [sentencepiece](https://github.com/google/sentencepiece)20KLLaMA32K
- tokenLLaMA**49953**
- fine-tuneAlpacaLLaMApad tokenAlpaca**49954**

[FAQ](#FAQ)

### 

20G[BERT-wwm](https://github.com/ymcui/Chinese-BERT-wwm)[MacBERT](https://github.com/ymcui/MacBERT)[LERT](https://github.com/ymcui/LERT)[PERT](https://github.com/ymcui/PERT)LLaMA

1. transformerembedding

2. LoRALoRAadapterembeddingLoRA

### 

1. [Stanford Alpaca](https://github.com/tatsu-lab/stanford_alpaca)LoRA
2. prompt[Stanford Alpaca](https://github.com/tatsu-lab/stanford_alpaca)inputinput`f"{instruction}+\n+{input}"`

### 

7B2M13B3M

|                  |  |                                                          |                                                   |
| -------------------- | :--: | :----------------------------------------------------------: | ----------------------------------------------------- |
|          | 500K | [](https://github.com/brightmart/nlp_chinese_corpus#5translation2019zh) | +                 |
| pCLUE            | 300K |      [](https://github.com/CLUEbenchmark/pCLUE)      | +                 |
| Alpaca     | 50K  |   [](https://github.com/tatsu-lab/stanford_alpaca)   | Alpaca                              |
| Alpaca     | 50K  |                    **[](./data)**                    | ChatGPT |
| Self-instruction | 1~2M |                                                      | ChatGPT     |

prompt`script/crawl_prompt.py`

```bash
python script/crawl_prompt.py output-file
```
- [Stanford Alpaca](https://github.com/tatsu-lab/stanford_alpaca#data-generation-process)20
- `gpt-3.5-turbo`OpenAI API key
- JSONJSON
- [OpenAI API](https://platform.openai.com/docs/guides/rate-limits/overview)

### 

|                  | -  | -  |          |
| :----------------------- | :--------------: | :--------------: | :--------------: |
| Batch Size               |       1024       |       1024       |       512        |
| Initial Learning Rate    |       2e-4       |       1e-4       |       1e-4       |
| Training Steps           |        3K        |        6K        |      6K-10K      |
| Max Length               |       512        |       512        |       512        |
| Trainable Parameters (%) |      2.97%       |      6.06%       |      6.22%       |
| Training Device          |     8  A100     |    16  A100     |    16  A100     |
| Distributed Training     | DeepSpeed Zero-2 | DeepSpeed Zero-2 | DeepSpeed Zero-2 |


## 

LLaMAAlpaca

- 
- 
- demo

## FAQ

##### 1

LLaMAFacebookLLaMArepo

##### 233B65B

33B65B


##### 3

1LLaMApretrain/finetune234RLHF54-bitFP16

##### 4LLaMA

LLaMA32K[LLaMA](https://arxiv.org/abs/2302.13971v1)XLM-R250KLLaMAbyte token1tokenLLaMA2-3

##### 5

4-bitFP16prompt

##### 6Windows

 Windows



- llama.cpp[PR#840](https://github.com/ggerganov/llama.cpp/pull/840)llama.cpp
- [Unicode (Windows) Support for llama.cpp](https://github.com/josStorer/llama.cpp-unicode-windows)@josStorer
- [#issue 11](https://github.com/ymcui/Chinese-LLaMA-Alpaca/issues/11)@LainNya@boholder@hyperzlib 



- [#issue 51](https://github.com/ymcui/Chinese-LLaMA-Alpaca/issues/51)@wscsjnhboy 

##### 7Chinese-LLaMA 13Bllama.cpp

13B[Issue#133](https://github.com/ymcui/Chinese-LLaMA-Alpaca/issues/133)issueChinese-LLaMAllama.cppChinese-LLaMA


## 


```
@misc{chinese-llama-alpaca,
  author = {Yiming Cui and Ziqing Yang},
  title = {Chinese LLaMA and Alpaca LLMs},
  year = {2023},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/ymcui/Chinese-LLaMA-Alpaca}},
}
```

## 



- Facebook LLaMA: https://github.com/facebookresearch/llama
- Stanford Alpaca: https://github.com/tatsu-lab/stanford_alpaca
- alpaca-lora by @tloen: https://github.com/tloen/alpaca-lora
- llama.cpp by @ggerganov: https://github.com/ggerganov/llama.cpp
- pCLUE and translation data by @brightmart: https://github.com/brightmart/nlp_chinese_corpus

Episode: Logo[midjourney](http://midjourney.com)Mac

## 






## 
GitHub Issue

- FAQissue
- issue[stable-bot](https://github.com/marketplace/stale)
-