Updated 5 months ago
https://github.com/bytedance/abq-llm
An acceleration library that supports arbitrary bit-width combinatorial quantization operations
An acceleration library that supports arbitrary bit-width combinatorial quantization operations