Text-Similarity

Overview

• Dataset: 中文/English 语料, ☞ 点这里
• Paper: 相关论文详解, ☞ 点这里
• The implemented method is as follows:：
  • TF-IDF
  • BM25
  • LSH
  • SIF/uSIF
  • FastText
  • RNN Base (Siamese RNN, Stack RNN)
  • CNN Base (Fast Text, Text CNN, Char CNN, VDCNN)
  • Bert Base
  • Albert
  • NEZHA
  • RoBERTa
  • SimCSE
  • Poly-Encoder
  • ColBERT
  • RE2（Simple-Effective-Text-Matching）

Usages

可以选择通过pip进行安装并使用（如下），或者直接下载源码到本地，集成到项目中： pip3 install text-sim

1：examples目录下有不同模型对应的 preprocess/train/evalute代码，可自行修改 2：如下示例从examples中引入actuator方法，准备好对应的模型配置文件即可执行 3：examples目录下的inference.py为训练好的模型推理代码 4：主体代码放在sim下，TensorFlow和Pytorch两个版本分开存放，引用方式基本保持一致 5：相关工具包括word2vec、tokenizer、data_format统一放在sim的tools下

TF-IDF

```python

Example

Sklearn version

from examples.runtfidfsklearn import actuator actuator("./corpus/chinese/breeno/train.tsv", query1="12 23 4160 276", query2="29 23 169 1495")

Custom version

from examples.run_tfidf import actuator actuator("./corpus/chinese/breeno/train.tsv", query1="12 23 4160 276", query2="29 23 169 1495")

工具调用

from sim.tf_idf import TFIdf

tokens_list = ["这是 一个 什么 样 的 工具", "..."] query = ["非常 好用 的 工具"]

tfidf = TFIdf(tokenslist, split=" ") print(tfidf.getscore(query, 0)) # score print(tfidf.getscorelist(query, 10)) # [(index, score), ...] print(tfidf.weight()) # list or numpy array ```

BM25

```python

Example

from examples.run_bm25 import actuator actuator("./corpus/chinese/breeno/train.tsv", query1="12 23 4160 276", query2="29 23 169 1495")

工具调用

from sim.bm25 import BM25

tokens_list = ["这是 一个 什么 样 的 工具", "..."] query = ["非常 好用 的 工具"]

bm25 = BM25(tokenslist, split=" ") print(bm25.getscore(query, 0)) # score print(bm25.getscorelist(query, 10)) # [(index, score), ...] print(bm25.weight()) # list or numpy array ```

LSH

```python from sim.lsh import E2LSH from sim.lsh import MinHash

e2lsh = E2LSH() min_hash = MinHash()

candidates = [[3.6216, 8.6661, -2.8073, -0.44699, 0], ...] query = [-2.7769, -5.6967, 5.9179, 0.37671, 1] print(e2lsh.search(candidates, query)) # index in candidates print(min_hash.search(candidates, query)) # index in candidates ```

SIF

• A Simple But Tough-To-Beat Baseline For Sentence Embeddings
• Unsupervised Random Walk Sentence Embeddings: A Strong but Simple Baseline ```python sentences = [["token1", "token2", "..."], ...] vector = [[[1, 1, 1], [2, 2, 2], [...]], ...] from sim.sifusif import SIF from sim.sifusif import uSIF

sif = SIF(ncomponents=5, componenttype="svd") sif.fit(tokenslist=sentences, vectorlist=vector)

usif = uSIF(ncomponents=5, n=1, componenttype="svd") usif.fit(tokenslist=sentences, vectorlist=vector) ```

FastText

• Bag of Tricks for Efficient Text Classification ```python # TensorFlow version from examples.tensorflow.runfasttext import actuator actuator(executetype="train", modeltype="bert", modeldir="./data/chinesewwmL-12H-768_A-12")

Pytorch version

from examples.pytorch.runfasttext import actuator actuator(executetype="train", modeltype="bert", modeldir="./data/chinesewwm_pytorch") ```

RNN Base

• Siamese Recurrent Architectures for Learning Sentence Similarity
• Learning Text Similarity with Siamese Recurrent Networks ```python # TensorFlow version from examples.tensorflow.runsiamesernn import actuator actuator("./data/config/siamsernn.json", executetype="train")

Pytorch version

from examples.pytorch.runsiamesernn import actuator actuator("./data/config/siamsernn.json", executetype="train") ```

CNN Base

• Convolutional Neural Networks for Sentence Classification
• Character-Aware Neural Language Models
• Highway Networks
• Very Deep Convolutional Networks for Text Classification ```python # TensorFlow version from examples.tensorflow.runcnnbase import actuator actuator(executetype="train", modeltype="bert", modeldir="./data/chinesewwmL-12H-768_A-12")

Pytorch version

from examples.pytorch.runcnnbase import actuator actuator(executetype="train", modeltype="bert", modeldir="./data/chinesewwm_pytorch") ```

Bert Base

• Attention Is All You Need ```python # TensorFlow version from examples.tensorflow.runbasicbert import actuator actuator(modeldir="./data/chinesewwmL-12H-768A-12", executetype="train")

Pytorch version

from examples.pytorch.runbasicbert import actuator actuator(modeldir="./data/chinesewwmpytorch", executetype="train") ```

Albert

• ALBERT: A Lite BERT For Self-superpised Learning Of Language Representations ```python # TensorFlow version from examples.tensorflow.runalbert import actuator actuator(modeldir="./data/albertsmallzhgoogle", executetype="train")

Pytorch version

from examples.pytorch.runalbert import actuator actuator(modeldir="./data/albertchinesesmall", execute_type="train") ```

NEZHA

• NEZHA: Neural Contextualized Representation For Chinese Language Understanding ```python # TensorFlow version from examples.tensorflow.runnezha import actuator actuator(modeldir="./data/NEZHA-Base-WWM", execute_type="train")

Pytorch version

from examples.pytorch.runnezha import actuator actuator(modeldir="./data/nezha-base-wwm", execute_type="train") ```

RoBERTa

• RoBERTa: A Robustly Optimized BERT Pretraining Approach ```python # TensorFlow version from examples.tensorflow.runbasicbert import actuator actuator(modeldir="./data/chineserobertaL-6H-384A-12", executetype="train")

Pytorch version

from examples.pytorch.runbasicbert import actuator actuator(modeldir="./data/chinese-roberta-wwm-ext", executetype="train") ```

SimCSE

• SimCSE: Simple Contrastive Learning of Sentence Embeddings ```python # TensorFlow version from examples.tensorflow.runsimcse import actuator actuator(modeldir="./data/chinesewwmL-12H-768A-12", executetype="train", modeltype="bert")

Pytorch version

from examples.pytorch.runsimcse import actuator actuator(modeldir="./data/chinesewwmpytorch", executetype="train", modeltype="bert") ```

Poly-Encoder

• Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring ```python # TensorFlow version from examples.tensorflow.runpolyencoder import actuator actuator(modeldir="./data/chinesewwmL-12H-768A-12", executetype="train", model_type="bert")

Pytorch version

from examples.pytorch.runpolyencoder import actuator actuator(modeldir="./data/chinesewwmpytorch", executetype="train", model_type="bert") ```

ColBERT

• ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT ```python # TensorFlow version from examples.tensorflow.runcolbert import actuator actuator(modeldir="./data/chinesewwmL-12H-768A-12", executetype="train", modeltype="bert")

Pytorch version

from examples.pytorch.runcolbert import actuator actuator(modeldir="./data/chinesewwmpytorch", executetype="train", modeltype="bert") ```

RE2

• Simple and Effective Text Matching with Richer Alignment Features ```python # TensorFlow version from examples.tensorflow.runre2 import actuator actuator("./data/config/re2.json", executetype="train")

Pytorch version

from examples.pytorch.runre2 import actuator actuator("./data/config/re2.json", executetype="train") ```

Cite

@misc{text-similarity, title={text-similarity}, author={Bocong Deng}, year={2021}, howpublished={\url{https://github.com/DengBoCong/text-similarity}}, }

Reference

• bert4keras
• albert_zh
• HuggingFace
• Self-Attention with Relative Position Representations