https://github.com/artificialzeng/chatglm-6b

ChatGLM-6B: An Open Bilingual Dialogue Language Model | 开源双语对话语言模型

Last synced: 10 months ago · JSON representation

Repository

ChatGLM-6B: An Open Bilingual Dialogue Language Model | 开源双语对话语言模型

Basic Info

Host: GitHub
Owner: ArtificialZeng
License: apache-2.0
Language: Python
Default Branch: main
Homepage:
Size: 8.84 MB

Statistics

Stars: 0
Watchers: 1
Forks: 0
Open Issues: 0
Releases: 0

Fork of THUDM/ChatGLM-6B

Created about 3 years ago · Last pushed almost 3 years ago

https://github.com/ArtificialZeng/ChatGLM-6B/blob/main/

# ChatGLM-6B


    Blog   HF Repo   Twitter   [GLM@ACL 22] [GitHub]   [GLM-130B@ICLR 23] [GitHub] 



      Slack  WeChat


*Read this in [English](README_en.md).*

## 

ChatGLM-6B  [General Language Model (GLM)](https://github.com/THUDM/GLM)  62 INT4  6GB 
ChatGLM-6B  ChatGPT  1T 62  ChatGLM-6B [](https://chatglm.cn/blog)

 [P-Tuning v2](https://github.com/THUDM/P-tuning-v2)  [()](ptuning/README.md) INT4  7GB 

 ChatGLM-6B  [Badcase ](improve/README.md) 

-----

ChatGLM-6B [](MODEL_LICENSE)** ChatGLM-6B  iOS  Windows App **

 ChatGLM-6B [](README.md#)****

## 
**[2023/06/25]**  [ChatGLM2-6B](https://github.com/THUDM/ChatGLM2-6B)ChatGLM-6B ChatGLM**2**-6B 

1. **** ChatGLM  ChatGLM2-6B ChatGLM2-6B  [GLM](https://github.com/THUDM/GLM)  1.4T [](#)ChatGLM2-6B  MMLU+23%CEval+33%GSM8K+571% BBH+60%
2. **** [FlashAttention](https://github.com/HazyResearch/flash-attention) Context Length ChatGLM-6B  2K  32K 8K  ChatGLM2-6B 
3. **** [Multi-Query Attention](http://arxiv.org/abs/1911.02150) ChatGLM2-6B  42%INT4 6G  1K  8K

 [ChatGLM2-6B](https://github.com/THUDM/ChatGLM2-6B)

**[2023/06/14]**  [WebGLM](https://github.com/THUDM/WebGLM)KDD 2023

![](resources/webglm.jpg)

**[2023/05/17]**  [VisualGLM-6B](https://github.com/THUDM/VisualGLM-6B)

![](resources/visualglm.png)

 [cli_demo_vision.py](cli_demo_vision.py)  [web_demo_vision.py](web_demo_vision.py)  Demo VisualGLM-6B  [SwissArmyTransformer](https://github.com/THUDM/SwissArmyTransformer/)  torchvision [VisualGLM-6B](https://github.com/THUDM/VisualGLM-6B)

**[2023/05/15]**  v1.1  checkpoint



* Describe a time when you had to make a difficult decision.
  - v1.0:
  ![](resources/english-q1-old.png)
  - v1.1:
  ![](resources/english-q1-new.png)
* Describe the function of a computer motherboard
  - v1.0:
  ![](resources/english-q2-old.png)
  - v1.1:
  ![](resources/english-q2-new.png)
* Develop a plan to reduce electricity usage in a home.
  - v1.0:
  ![](resources/english-q3-old.png)
  - v1.1:
  ![](resources/english-q3-new.png)
* NFT
  - v1.0:
  ![](resources/english-q4-old.png)
  - v1.1:
  ![](resources/english-q4-new.png)


 [UPDATE.md](UPDATE.md)

## 
 ChatGLM 
* [lyraChatGLM](https://huggingface.co/TMElyralab/lyraChatGLM):  ChatGLM-6B  9000+ tokens/s 
* [ChatGLM-MNN](https://github.com/wangzhaode/ChatGLM-MNN):  MNN  ChatGLM-6B C++  GPU  CPU
* [JittorLLMs](https://github.com/Jittor/JittorLLMs)3G ChatGLM-6B FP16 LinuxwindowsMac
* [InferLLM](https://github.com/MegEngine/InferLLM) C++  x86Arm  4G

 ChatGLM-6B 
* [langchain-ChatGLM](https://github.com/imClumsyPanda/langchain-ChatGLM) langchain  ChatGLM 
* [](https://github.com/l15y/wenda) ChatGLM-6B  ChatPDF 
* [chatgpt_academic](https://github.com/binary-husky/chatgpt_academic): ChatGLM-6BLLMLLM
* [glm-bot](https://github.com/initialencounter/glm-bot)ChatGLMKoishiChatGLM

 ChatGLM-6B 
* [ChatGLM-6B ](https://www.heywhale.com/mw/project/6436d82948f7da1fee2be59e)
* [ChatGLM-6B  langchain  QA Bot](https://www.heywhale.com/mw/project/643977aa446c45f4592a1e59)


* [Measuring Massive Multitask Chinese Understanding](https://arxiv.org/abs/2304.12986)

 [PROJECT.md](PROJECT.md)

## 

### 

| ****   | ** GPU ** | ** GPU ** |
| -------------- | ------------------------- | --------------------------------- |
| FP16 | 13 GB                     | 14 GB                             |
| INT8           | 8 GB                     | 9 GB                             |
| INT4           | 6 GB                      | 7 GB                              |
### 

 pip `pip install -r requirements.txt` `transformers`  `4.27.1` `4.23.1` 

 cpu  `gcc`  `openmp` Linux  Windows  [TDM-GCC](https://jmeubank.github.io/tdm-gcc/)  `openmp` Windows  `gcc`  `TDM-GCC 10.3.0` Linux  `gcc 11.3.0` MacOS  [Q1](FAQ.md#q1)

###  

 ChatGLM-6B 

```python
>>> from transformers import AutoTokenizer, AutoModel
>>> tokenizer = AutoTokenizer.from_pretrained("THUDM/chatglm-6b", trust_remote_code=True)
>>> model = AutoModel.from_pretrained("THUDM/chatglm-6b", trust_remote_code=True).half().cuda()
>>> model = model.eval()
>>> response, history = model.chat(tokenizer, "", history=[])
>>> print(response)
! ChatGLM-6B,,
>>> response, history = model.chat(tokenizer, "", history=history)
>>> print(response)
,:

1. :,,
2. :,,,
3. :,,,,,
4. :,,,
5. :,,,
6. :,,,,

,,
```
 `from_pretrained`  `revision="v1.1.0"` `v1.1.0`  [Change Log](https://huggingface.co/THUDM/chatglm-6b#change-log)

### 
 `transformers`  [Hugging Face Hub](https://huggingface.co/THUDM/chatglm-6b)

 Hugging Face Hub [Git LFS](https://docs.github.com/zh/repositories/working-with-files/managing-large-files/installing-git-large-file-storage)
```Shell
git clone https://huggingface.co/THUDM/chatglm-6b
```

 Hugging Face Hub  checkpoint 
```Shell
GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/THUDM/chatglm-6b
```
[](https://cloud.tsinghua.edu.cn/d/fb9f16d6dc8f482596c2/) `chatglm-6b` 

 `THUDM/chatglm-6b`  `chatglm-6b` 

**Optional** 
```Shell
git checkout v1.1.0
```

## Demo & API

 [Gradio](https://gradio.app)  Demo  Demo

```shell
git clone https://github.com/THUDM/ChatGLM-6B
cd ChatGLM-6B
```

###  Demo

![web-demo](resources/web-demo.gif)

 Gradio`pip install gradio` [web_demo.py](web_demo.py) 

```shell
python web_demo.py
```

 Web Server Demo  Gradio  `demo.queue().launch(share=True, inbrowser=True)`  Gradio  `share=False` `share=True` 

 [@AdamBear](https://github.com/AdamBear)  Streamlit  Demo[#117](https://github.com/THUDM/ChatGLM-6B/pull/117).

###  Demo

![cli-demo](resources/cli-demo.png)

 [cli_demo.py](cli_demo.py)

```shell
python cli_demo.py
```

 `clear`  `stop` 

### API
 `pip install fastapi uvicorn` [api.py](api.py)
```shell
python api.py
```
 8000  POST 
```shell
curl -X POST "http://127.0.0.1:8000" \
     -H 'Content-Type: application/json' \
     -d '{"prompt": "", "history": []}'
```

```shell
{
  "response":" ChatGLM-6B",
  "history":[[""," ChatGLM-6B"]],
  "status":200,
  "time":"2023-03-23 21:38:40"
}
```

## 
### 
 FP16  13GB  GPU 

```python
#  4/8 bit 
model = AutoModel.from_pretrained("THUDM/chatglm-6b", trust_remote_code=True).quantize(8).half().cuda()
```

 2  3 8-bit  GPU  10GB4-bit  6GB  ChatGLM-6B  context-length 2048

ChatGLM-6B  4-bit  [GPT-Q](https://arxiv.org/abs/2210.17323) / Pull Request

 FP16  13GB INT4  5.2GB 
```python
# INT8 "THUDM/chatglm-6b-int4""THUDM/chatglm-6b-int8"
model = AutoModel.from_pretrained("THUDM/chatglm-6b-int4", trust_remote_code=True).half().cuda()
```
[](https://cloud.tsinghua.edu.cn/d/674208019e314311ab5c/)

### CPU 
 GPU  CPU  32GB 
```python
model = AutoModel.from_pretrained("THUDM/chatglm-6b", trust_remote_code=True).float()
```


```python
# INT8 "THUDM/chatglm-6b-int4""THUDM/chatglm-6b-int8"
model = AutoModel.from_pretrained("THUDM/chatglm-6b-int4",trust_remote_code=True).float()
```

 `Could not find module 'nvcuda.dll'`  `RuntimeError: Unknown platform: darwin` (MacOS) [](README.md#)

### Mac 
 Apple Silicon  AMD GPU Mac MPS  GPU  ChatGLM-6B Apple  [](https://developer.apple.com/metal/pytorch)  PyTorch-Nightly2.1.0.dev2023xxxx2.0.0

 MacOS [](README.md#) mps 
```python
model = AutoModel.from_pretrained("your local path", trust_remote_code=True).half().to('mps')
```

 ChatGLM-6B  13GB  16GB  MacBook Pro chatglm-6b-int4 GPU  kernel  CUDA  MacOS  CPU 
```python
# INT8 "THUDM/chatglm-6b-int4""THUDM/chatglm-6b-int8"
model = AutoModel.from_pretrained("THUDM/chatglm-6b-int4",trust_remote_code=True).float()
```
 CPU [ OpenMP](FAQ.md#q1)

### 
 GPU GPU GPU accelerate: `pip install accelerate`
```python
from utils import load_model_on_gpus
model = load_model_on_gpus("THUDM/chatglm-6b", num_gpus=2)
```
 GPU  `num_gpus`  GPU  `device_map`  

## 
 [P-tuning v2](https://github.com/THUDM/P-tuning-v2)  [ptuning/README.md](ptuning/README.md)

## ChatGLM-6B 

 `web_demo.py`  ChatGLM-6B 



![](examples/self-introduction.png)





![](examples/blog-outline.png)





![](examples/ad-writing-2.png)

![](examples/comments-writing.png)





![](examples/email-writing-1.png)

![](examples/email-writing-2.png)





![](examples/information-extraction.png)





![](examples/role-play.png)





![](examples/sport.png)





![](examples/tour-guide.png)



## 

 ChatGLM-6B 

- 6B ChatGLM-6B 
    
    
    ![](limitations/factual_error.png)
    
    ![](limitations/math_error.png)
    
    
  
- ChatGLM-6B 

- ChatGLM-6B /

- ChatGLM-6B   
    

    ![](limitations/self-confusion_google.jpg)
    
    ![](limitations/self-confusion_openai.jpg)
    
    ![](limitations/self-confusion_tencent.jpg)
    
    

## 

 [Apache-2.0](LICENSE) ChatGLM-6B  [Model License](MODEL_LICENSE)

## 



```
@article{zeng2022glm,
  title={Glm-130b: An open bilingual pre-trained model},
  author={Zeng, Aohan and Liu, Xiao and Du, Zhengxiao and Wang, Zihan and Lai, Hanyu and Ding, Ming and Yang, Zhuoyi and Xu, Yifan and Zheng, Wendi and Xia, Xiao and others},
  journal={arXiv preprint arXiv:2210.02414},
  year={2022}
}
```
```
@inproceedings{du2022glm,
  title={GLM: General Language Model Pretraining with Autoregressive Blank Infilling},
  author={Du, Zhengxiao and Qian, Yujie and Liu, Xiao and Ding, Ming and Qiu, Jiezhong and Yang, Zhilin and Tang, Jie},
  booktitle={Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  pages={320--335},
  year={2022}
}
```

Owner

Name: Dr. Artificial曾小健
Login: ArtificialZeng
Kind: user
Location: Beijing

Website: https://blog.csdn.net/sinat_37574187?type=blog
Repositories: 171
Profile: https://github.com/ArtificialZeng

LLM practitioner/engineer, AI/ML/DL Quant

ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/artificialzeng/chatglm-6b

Science Score: 10.0%

Repository

Basic Info

Statistics

https://github.com/ArtificialZeng/ChatGLM-6B/blob/main/

Owner

GitHub Events

Total

Last Year