Data

Tools

Indexes

Applications

Experiments

Open Source Science

Recent Releases of textgen

textgen - 1.1.2

1.1.2版本

  1. GPT模型支持了NEFTune给embedding加噪SFT训练方法，SFT中使用 --neft_alpha 参数启用 NEFTune，例如 --neft_alpha 5，示例代码：https://github.com/shibing624/textgen/blob/main/examples/gpt/trainingllamademo.py

  2. 兼容了 ChatGLM-6B 1,2,3 / Baichuan 1,2 / LLaMA 1,2 / BLOOM / Mistral / QWen 等GPT模型LoRA微调训练和预测，可以用于多轮对话微调训练

Full Changelog: https://github.com/shibing624/textgen/compare/1.1.1...1.1.2

- Python
Published by shibing624 over 2 years ago

textgen - 1.1.1

v1.1.1版本

  1. 支持多卡推理，推理速度加倍，调库textgen做batch推理，多卡推理更方便、快速。

多卡数据并行，batch推理： cd examples/gpt CUDA_VISIBLE_DEVICES=0,1 torchrun --nproc_per_node 2 inference_multigpu_demo.py --model_type chatglm --base_model THUDM/chatglm-6b

  1. 优化ChatGLM-6B/Baichuan/LLaMA2/BLOOM的多轮对话SFT代码，逻辑合并到textgen/gpt下，统一处理多模型微调，加入prompt模板支持。

Full Changelog: https://github.com/shibing624/textgen/compare/1.1.0...1.1.1

- Python
Published by shibing624 over 2 years ago

textgen - v1.1.0

v1.1.0版本

  1. 发布基于ShareGPT4数据集微调的中英文Vicuna-13B模型shibing624/vicuna-baichuan-13b-chat，和对应的LoRA模型shibing624/vicuna-baichuan-13b-chat-lora;
  2. 支持多轮对话微调；多轮对话样本格式：examples/data/sharegptzh100_format.jsonl

Full Changelog: https://github.com/shibing624/textgen/compare/1.0.2...1.1.0

- Python
Published by shibing624 over 2 years ago

textgen - v1.0.2

v1.0.2

新增支持ChatGLM2和LLaMA2模型的SFT微调训练

Full Changelog: https://github.com/shibing624/textgen/compare/1.0.1...1.0.2

- Python
Published by shibing624 over 2 years ago

textgen - v1.0

v1.0版本

训练 ChatGLM/LLaMA/Bloom 微调模型

  1. 支持自定义训练数据集和训练参数，数据集格式参考examples/data/zhcsctest.tsv或者shibing624/alpaca-zh
  2. 支持AdaLoRA、LoRA、PTuning、PrefixTuning等部分参数微调方法，也支持全参微调
  3. 支持多卡训练，支持混合精度训练

Full Changelog: https://github.com/shibing624/textgen/compare/0.2.7...1.0.0

- Python
Published by shibing624 over 2 years ago

textgen - v0.2.7

v0.2.7版本

  • 新增ChatGLM/LLaMA/Bloom模型的SFT微调训练，并发布适用于通用对话和中文纠错的LoRA模型。

| Model | Arch | Introduction | Train Script | Predict Script | |:----------------------------------------------------------------------------------------------------------|:-----------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------| | shibing624/chatglm-6b-csc-zh-lora | ChatGLM-6B | 在27万中文拼写纠错数据shibing624/CSC上微调了一版ChatGLM-6B，纠错效果有提升，发布微调后的LoRA权重 | training script | predict script | | shibing624/chatglm-6b-belle-zh-lora | ChatGLM-6B | 在100万条中文ChatGPT指令Belle数据集BelleGroup/train1MCN上微调了一版ChatGLM-6B，问答效果有提升，发布微调后的LoRA权重 | training script | predict script | | shibing624/llama-13b-belle-zh-lora | LLaMA-13B | 在100万条中文ChatGPT指令Belle数据集BelleGroup/train1MCN上微调了一版Llama-13B，问答效果有提升，发布微调后的LoRA权重 | training script | predict script | | shibing624/chinese-alpaca-plus-7b-hf | LLaMA-7B | 中文LLaMA-Plus, Alpaca-Plus 7B版本，在LLaMA-7B上扩充了中文词表并继续预训练120G文本（通用领域），在4M指令数据集上微调后得到的中文Alpaca-plus模型 | training script | predict script | | shibing624/chinese-alpaca-plus-13b-hf | LLaMA-13B | 中文LLaMA-Plus, Alpaca-Plus 13B版本，在LLaMA-13B上扩充了中文词表并继续预训练120G文本（通用领域），在4.3M指令数据集上微调后得到的中文Alpaca-plus模型 | training script | predict script |

Full Changelog: https://github.com/shibing624/textgen/compare/0.2.5...0.2.7

- Python
Published by shibing624 over 2 years ago

textgen - 0.2.5

What's Changed

  • pad labels to max length by @xingener in https://github.com/shibing624/textgen/pull/25

New Contributors

  • @xingener made their first contribution in https://github.com/shibing624/textgen/pull/25

Full Changelog: https://github.com/shibing624/textgen/compare/0.2.0...0.2.5

- Python
Published by shibing624 almost 3 years ago

textgen - 0.1.7

What's Changed

  • del the repeated torch.nn.DataParallel by @wiserxin in https://github.com/shibing624/textgen/pull/7

New Contributors

  • @wiserxin made their first contribution in https://github.com/shibing624/textgen/pull/7

Full Changelog: https://github.com/shibing624/textgen/compare/0.1.5...0.1.7

- Python
Published by shibing624 about 3 years ago

textgen - 0.1.5 add songnet and copyt5

Full Changelog: https://github.com/shibing624/textgen/compare/0.1.1...0.1.5

- Python
Published by shibing624 over 3 years ago

textgen - 0.1.1

Full Changelog: https://github.com/shibing624/textgen/compare/0.1.0...0.1.1

- Python
Published by shibing624 over 3 years ago

textgen - 0.1.0

Full Changelog: https://github.com/shibing624/textgen/compare/0.0.5...0.1.0

- Python
Published by shibing624 over 3 years ago

textgen - 0.0.5

Full Changelog: https://github.com/shibing624/textgen/compare/0.0.3...0.0.5

- Python
Published by shibing624 almost 4 years ago