https://github.com/bytedance/lightseq - Support HIP

In the hipdev branch, LightSeq supports CUDA backend and HIP backend（now support training only）. LightSeq transformer has a speedup about 7% comparing with FairsSeq transformer under the HIP backend. LightSeq HIP supports multiple NLP models, such as transformer, bert, gpt, etc. Users need no modification with python training. More information about the LightSeq HIP can be found here https://github.com/bytedance/lightseq/blob/hipdev/README_HIP.md

- C++
Published by neopro12 over 3 years ago

https://github.com/bytedance/lightseq - Release 3.0.1

What's Changed

• compatible gcq params by @HandH1998 in https://github.com/bytedance/lightseq/pull/409
• Fix gpu name by @godweiyang in https://github.com/bytedance/lightseq/pull/415

Full Changelog: https://github.com/bytedance/lightseq/compare/v3.0.0...v3.0.1

- C++
Published by godweiyang over 3 years ago

https://github.com/bytedance/lightseq - Release 3.0.0

It's been a long time since our last release (v2.2.0). For the past one year, we have focused on int8 quantization.

In this release, LightSeq supports int8 quantized training and inference. Compared with PyTorch QAT, LightSeq int8 training has a speedup of 3x without any performance loss. Compared with previous LightSeq fp16 inference, int8 engine has a speedup up to 1.7x.

LightSeq int8 engine supports multiple models, such as Transformer, BERT, GPT, etc. For int8 training, the users only need to apply quantization mode to the model using model.apply(enable_quant). For int8 inference, the users only need to use QuantTransformer instead of fp16 Transformer.

Other releases include supporting models like MoE, fix bugs, performance improvement, etc.

- C++
Published by godweiyang over 3 years ago

https://github.com/bytedance/lightseq - Release 2.2.0

Inference

Support more multi-language models #209

Fixes

Fix inference error on HDF5 #208 Fix training error when batch_size=1 #192 Other minor fixes: #205 #202 #193

- C++
Published by Taka152 over 4 years ago

https://github.com/bytedance/lightseq - Release 2.1.3

This version contains several features and bug fixes.

Training

relax restriction of layer norm hidden size #137 #161 support inference during training for transformer #141 #146 #147

Inference

Add inference support and examples for BERT #145

Fixes

fix save/load for training with pytorch #139 fix pos embedding index bug #144

- C++
Published by neopro12 almost 5 years ago

https://github.com/bytedance/lightseq - Release 2.1.0

This version contains several features and bug fixes.

Training

support BertEncoder #116 support torch amp and apex amp #100

Inference

support big models like gpt2-large and bart-large #82

Fixes

fix adam bug when param size < 1024 #98 fix training compiling fail in cuda < 11 #80

- C++
Published by Taka152 almost 5 years ago

https://github.com/bytedance/lightseq - Release 2.0.2

[inference] fix warp reduce bug in inference. #74

- C++
Published by Taka152 almost 5 years ago

https://github.com/bytedance/lightseq - Release 2.0.1

Merge codes about training and inference. Reorganize docs and README.

- C++
Published by neopro12 almost 5 years ago

https://github.com/bytedance/lightseq - Release 2.0.0

It's been a long time since our last release (v1.2.0). For the past six months, we have focused on training efficiency.

In this release, LightSeq supports fast training for models in the Transformer family!

We provide highly optimized custom operators for PyTorch and TensorFlow, which cover the entire training process for Transformer-based models. Users of LightSeq can use these operators to build their own models with efficient computation.

In addition, we integrate our custom operators into popular training libraries like Fairseq, Hugging Face, NeurST, which enables a 1.5X-3X end-to-end speedup campred to the native version.

With only a small amount of code, you can enjoy the excellent performance provided by LightSeq. Try it now!

Training

• support lightseq-train to accelerate fairseq training, including optimized transformer model, adam, and label smoothed loss
• huggingface bert training example
• neurst transformer training example for Tensorflow users

Inference

• support GPT python wrapper
• inference APIs are moved to lightseq.inference

This release has API change for inference, all inference API has moved to lightseq.inference. For example, use import lightseq.inference and model = lightseq.inference.Transformer("$PB_PATH", max_batch_size)

- C++
Published by neopro12 almost 5 years ago

https://github.com/bytedance/lightseq - Release 1.2.0

Support Python API and multilingual nmt

- C++
Published by neopro12 over 5 years ago

https://github.com/bytedance/lightseq - Release 1.1.0

Support sampling/diverse beam search and VAE.

- C++
Published by neopro12 over 5 years ago

https://github.com/bytedance/lightseq - Release 1.0.0

Byseqlib is a high performance inference library for SOTA NLU/NLG models.

- C++
Published by neopro12 over 6 years ago

https://github.com/bytedance/lightseq - initial releases

To provide test model weights ans inputs

- C++
Published by neopro12 over 6 years ago