Overview | Tutorials | Examples | Installation | FAQ | API Docs | How to Cite

<!---->

Welcome to ktrain

a "Swiss Army knife" for machine learning

News and Announcements

• 2024-02-20

  - ktrain 0.41.x is released and removes the ktrain.text.qa.generative_qa module. Our OnPrem.LLM package should be used for Generative Question-Answering tasks. See example notebook.

Overview

ktrain is a lightweight wrapper for the deep learning library TensorFlow Keras (and other libraries) to help build, train, and deploy neural networks and other machine learning models. Inspired by ML framework extensions like fastai and ludwig, ktrain is designed to make deep learning and AI more accessible and easier to apply for both newcomers and experienced practitioners. With only a few lines of code, ktrain allows you to easily and quickly:

• employ fast, accurate, and easy-to-use pre-canned models for text, vision, graph, and tabular data:

  • text data:
    • Text Classification: BERT, DistilBERT, NBSVM, fastText, and other models _{^{[example notebook]}}
    • Text Regression: BERT, DistilBERT, Embedding-based linear text regression, fastText, and other models _{^{[example notebook]}}
    • Sequence Labeling (NER): Bidirectional LSTM with optional CRF layer and various embedding schemes such as pretrained BERT and fasttext word embeddings and character embeddings _{^{[example notebook]}}
    • Ready-to-Use NER models for English, Chinese, and Russian with no training required _{^{[example notebook]}}
    • Sentence Pair Classification for tasks like paraphrase detection _{^{[example notebook]}}
    • Unsupervised Topic Modeling with LDA _{^{[example notebook]}}
    • Document Similarity with One-Class Learning: given some documents of interest, find and score new documents that are thematically similar to them using One-Class Text Classification _{^{[example notebook]}}
    • Document Recommendation Engines and Semantic Searches: given a text snippet from a sample document, recommend documents that are semantically-related from a larger corpus _{^{[example notebook]}}
    • Text Summarization: summarize long documents - no training required _{^{[example notebook]}}
    • Extractive Question-Answering: ask a large text corpus questions and receive exact answers using BERT _{^{[example notebook]}}
    • Generative Question-Answering: ask a large text corpus questions and receive answers with citations using local or OpenAI models _{^{[example notebook]}}
    • Easy-to-Use Built-In Search Engine: perform keyword searches on large collections of documents _{^{[example notebook]}}
    • Zero-Shot Learning: classify documents into user-provided topics without training examples _{^{[example notebook]}}
    • Language Translation: translate text from one language to another _{^{[example notebook]}}
    • Text Extraction: Extract text from PDFs, Word documents, etc. _{^{[example notebook]}}
    • Speech Transcription: Extract text from audio files _{^{[example notebook]}}
    • Universal Information Extraction: extract any kind of information from documents by simply phrasing it in the form of a question _{^{[example notebook]}}
    • Keyphrase Extraction: extract keywords from documents _{^{[example notebook]}}
    • Sentiment Analysis: easy-to-use wrapper to pretrained sentiment analysis _{^{[example notebook]}}
    • Generative AI with GPT: Provide instructions to a lightweight ChatGPT-like model running on your own own machine to solve various tasks. _{^{[example notebook]}}
  • vision data:
  • image classification (e.g., ResNet, Wide ResNet, Inception) _{^{[example notebook]}}
  • image regression for predicting numerical targets from photos (e.g., age prediction) _{^{[example notebook]}}
  • image captioning with a pretrained model _{^{[example notebook]}}
  • object detection with a pretrained model _{^{[example notebook]}}
  • graph data:
  • node classification with graph neural networks (GraphSAGE) _{^{[example notebook]}}
  • link prediction with graph neural networks (GraphSAGE) _{^{[example notebook]}}
  • tabular data:
  • tabular classification (e.g., Titanic survival prediction) _{^{[example notebook]}}
  • tabular regression (e.g., predicting house prices) _{^{[example notebook]}}
  • causal inference using meta-learners _{^{[example notebook]}}

• estimate an optimal learning rate for your model given your data using a Learning Rate Finder

• utilize learning rate schedules such as the triangular policy, the 1cycle policy, and SGDR to effectively minimize loss and improve generalization

• build text classifiers for any language (e.g., Arabic Sentiment Analysis with BERT, Chinese Sentiment Analysis with NBSVM)

• easily train NER models for any language (e.g., Dutch NER )

• load and preprocess text and image data from a variety of formats

• inspect data points that were misclassified and provide explanations to help improve your model

• leverage a simple prediction API for saving and deploying both models and data-preprocessing steps to make predictions on new raw data

• built-in support for exporting models to ONNX and TensorFlow Lite (see example notebook for more information)

Tutorials

Please see the following tutorial notebooks for a guide on how to use ktrain on your projects: * Tutorial 1: Introduction * Tutorial 2: Tuning Learning Rates * Tutorial 3: Image Classification * Tutorial 4: Text Classification * Tutorial 5: Learning from Unlabeled Text Data * Tutorial 6: Text Sequence Tagging for Named Entity Recognition * Tutorial 7: Graph Node Classification with Graph Neural Networks * Tutorial 8: Tabular Classification and Regression * Tutorial A1: Additional tricks, which covers topics such as previewing data augmentation schemes, inspecting intermediate output of Keras models for debugging, setting global weight decay, and use of built-in and custom callbacks. * Tutorial A2: Explaining Predictions and Misclassifications * Tutorial A3: Text Classification with Hugging Face Transformers * Tutorial A4: Using Custom Data Formats and Models: Text Regression with Extra Regressors

Some blog tutorials and other guides about ktrain are shown below:

ktrain: A Lightweight Wrapper for Keras to Help Train Neural Networks

BERT Text Classification in 3 Lines of Code

Text Classification with Hugging Face Transformers in TensorFlow 2 (Without Tears)

Build an Open-Domain Question-Answering System With BERT in 3 Lines of Code

Finetuning BERT using ktrain for Disaster Tweets Classification by Hamiz Ahmed

Indonesian NLP Examples with ktrain by Sandy Khosasi

Examples

Using ktrain on Google Colab? See these Colab examples: - text classification: a simple demo of Multiclass Text Classification with BERT - text classification: a simple demo of Multiclass Text Classification with Hugging Face Transformers - sequence-tagging (NER): NER example using transformer word embeddings - question-answering: End-to-End Question-Answering using the 20newsgroups dataset. - image classification: image classification with Cats vs. Dogs

Tasks such as text classification and image classification can be accomplished easily with only a few lines of code.

Example: Text Classification of IMDb Movie Reviews Using BERT _{^{[see notebook]}}

```python import ktrain from ktrain import text as txt

load data

(xtrain, ytrain), (xtest, ytest), preproc = txt.textsfromfolder('data/aclImdb', maxlen=500, preprocessmode='bert', traintest_names=['train', 'test'], classes=['pos', 'neg'])

load model

model = txt.textclassifier('bert', (xtrain, y_train), preproc=preproc)

wrap model and data in ktrain.Learner object

learner = ktrain.getlearner(model, traindata=(xtrain, ytrain), valdata=(xtest, ytest), batchsize=6)

find good learning rate

learner.lrfind() # briefly simulate training to find good learning rate learner.lrplot() # visually identify best learning rate

train using 1cycle learning rate schedule for 3 epochs

learner.fit_onecycle(2e-5, 3) ```

Example: Classifying Images of Dogs and Cats Using a Pretrained ResNet50 model <sub><sup>[see notebook]</sup></sub>

```python import ktrain from ktrain import vision as vis

load data

(traindata, valdata, preproc) = vis.imagesfromfolder( datadir='data/dogscats', dataaug = vis.getdataaug(horizontalflip=True), traintestnames=['train', 'valid'], targetsize=(224,224), colormode='rgb')

load model

model = vis.imageclassifier('pretrainedresnet50', traindata, valdata, freeze_layers=80)

wrap model and data in ktrain.Learner object

learner = ktrain.getlearner(model=model, traindata=traindata, valdata=valdata, workers=8, usemultiprocessing=False, batch_size=64)

find good learning rate

learner.lrfind() # briefly simulate training to find good learning rate learner.lrplot() # visually identify best learning rate

train using triangular policy with ModelCheckpoint and implicit ReduceLROnPlateau and EarlyStopping

learner.autofit(1e-4, checkpointfolder='/tmp/savedweights') ```

Example: Sequence Labeling for Named Entity Recognition using a randomly initialized Bidirectional LSTM CRF model <sub><sup>[see notebook]</sup></sub>

```python import ktrain from ktrain import text as txt

load data

(trn, val, preproc) = txt.entitiesfromtxt('data/nerdataset.csv', sentencecolumn='Sentence #', wordcolumn='Word', tagcolumn='Tag', dataformat='gmb', usechar=True) # enable character embeddings

load model

model = txt.sequence_tagger('bilstm-crf', preproc)

wrap model and data in ktrain.Learner object

learner = ktrain.getlearner(model, traindata=trn, val_data=val)

conventional training for 1 epoch using a learning rate of 0.001 (Keras default for Adam optmizer)

learner.fit(1e-3, 1) ```

Example: Node Classification on Cora Citation Graph using a GraphSAGE model <sub><sup>[see notbook]</sup></sub>

```python import ktrain from ktrain import graph as gr

load data with supervision ratio of 10%

(trn, val, preproc) = gr.graphnodesfromcsv( 'cora.content', # node attributes/labels 'cora.cites', # edge list samplesize=20, holdoutpct=None, holdoutforinductive=False, trainpct=0.1, sep='\t')

load model

model=gr.graphnodeclassifier('graphsage', trn)

wrap model and data in ktrain.Learner object

learner = ktrain.getlearner(model, traindata=trn, valdata=val, batchsize=64)

find good learning rate

learner.lrfind(maxepochs=100) # briefly simulate training to find good learning rate learner.lr_plot() # visually identify best learning rate

train using triangular policy with ModelCheckpoint and implicit ReduceLROnPlateau and EarlyStopping

learner.autofit(0.01, checkpointfolder='/tmp/savedweights') ```

Example: Text Classification with Hugging Face Transformers on 20 Newsgroups Dataset Using DistilBERT <sub><sup>[see notebook]</sup></sub>

```python

load text data

categories = ['alt.atheism', 'soc.religion.christian','comp.graphics', 'sci.med'] from sklearn.datasets import fetch20newsgroups trainb = fetch20newsgroups(subset='train', categories=categories, shuffle=True) testb = fetch20newsgroups(subset='test',categories=categories, shuffle=True) (xtrain, ytrain) = (trainb.data, trainb.target) (xtest, ytest) = (testb.data, test_b.target)

build, train, and validate model (Transformer is wrapper around transformers library)

import ktrain from ktrain import text MODELNAME = 'distilbert-base-uncased' t = text.Transformer(MODELNAME, maxlen=500, classnames=trainb.targetnames) trn = t.preprocesstrain(xtrain, ytrain) val = t.preprocesstest(xtest, ytest) model = t.getclassifier() learner = ktrain.getlearner(model, traindata=trn, valdata=val, batchsize=6) learner.fitonecycle(5e-5, 4) learner.validate(classnames=t.getclasses()) # classnames must be string values

Output from learner.validate()

precision recall f1-score support

alt.atheism 0.92 0.93 0.93 319

comp.graphics 0.97 0.97 0.97 389

sci.med 0.97 0.95 0.96 396

soc.religion.christian 0.96 0.96 0.96 398

accuracy 0.96 1502

macro avg 0.95 0.96 0.95 1502

weighted avg 0.96 0.96 0.96 1502

```

Example: Tabular Classification for Titanic Survival Prediction Using an MLP _{^{[see notebook]}}

```python import ktrain from ktrain import tabular import pandas as pd traindf = pd.readcsv('train.csv', indexcol=0) traindf = traindf.drop(['Name', 'Ticket', 'Cabin'], 1) trn, val, preproc = tabular.tabularfromdf(traindf, labelcolumns=['Survived'], randomstate=42) learner = ktrain.getlearner(tabular.tabularclassifier('mlp', trn), traindata=trn, valdata=val) learner.lrfind(showplot=True, maxepochs=5) # estimate learning rate learner.fitonecycle(5e-3, 10)

evaluate held-out labeled test set

tst = preproc.preprocesstest(pd.readcsv('heldout.csv', indexcol=0)) learner.evaluate(tst, classnames=preproc.get_classes()) ```

Additional examples can be found here.

Installation

1. Make sure pip is up-to-date with: pip install -U pip

2. Install TensorFlow 2 if it is not already installed (e.g., pip install tensorflow).

3. Install ktrain: pip install ktrain

4. If using tensorflow>=2.16:

   • Install tf_keras: pip install tf_keras
   • Set the environment variable TF_USE_LEGACY_KERAS to true before importing ktrain

The above should be all you need on Linux systems and cloud computing environments like Google Colab and AWS EC2. If you are using ktrain on a Windows computer, you can follow these more detailed instructions that include some extra steps.

Notes about TensorFlow Versions

• As of tensorflow>=2.11, you must only use legacy optimizers such as tf.keras.optimizers.legacy.Adam. The newer tf.keras.optimizers.Optimizer base class is not supported at this time. For instance, when using TensorFlow 2.11 and above, please use tf.keras.optimzers.legacy.Adam() instead of the string "adam" in model.compile. ktrain does this automatically when using out-of-the-box models (e.g., models from the transformers library).
• As mentioned above, due to breaking changes in TensorFlow 2.16, you will need to install the tf_keras package and also set the environment variable TF_USE_LEGACY_KERAS=True before importing ktrain (e.g., add export TF_USE_LEGACY_KERAS=1 in .bashrc or add os.environ['TF_USE_LEGACY_KERAS']="1" at top of your code, etc.).

Additional Notes About Installation

• Some optional, extra libraries used for some operations can be installed as needed. (Notice that ktrain is using forked versions of the eli5 and stellargraph libraries in order to support TensorFlow2.) python # for graph module: pip install https://github.com/amaiya/stellargraph/archive/refs/heads/no_tf_dep_082.zip # for text.TextPredictor.explain and vision.ImagePredictor.explain: pip install https://github.com/amaiya/eli5-tf/archive/refs/heads/master.zip # for tabular.TabularPredictor.explain: pip install shap # for text.zsl (ZeroShotClassifier), text.summarization, text.translation, text.speech: pip install torch # for text.speech: pip install librosa # for tabular.causal_inference_model: pip install causalnlp # for text.summarization.core.LexRankSummarizer: pip install sumy # for text.kw.KeywordExtractor pip install textblob # for text.generative_ai pip install onprem

• ktrain purposely pins to a lower version of transformers to include support for older versions of TensorFlow. If you need a newer version of transformers, it is usually safe for you to upgrade transformers, as long as you do it after installing ktrain.

• As of v0.30.x, TensorFlow installation is optional and only required if training neural networks. Although ktrain uses TensorFlow for neural network training, it also includes a variety of useful pretrained PyTorch models and sklearn models, which can be used out-of-the-box without having TensorFlow installed, as summarized in this table:

| Feature | TensorFlow | PyTorch | Sklearn | --- | :-: | :-: | :-: | | training any neural network (e.g., text or image classification) | ✅ | ❌ | ❌ | | End-to-End Question-Answering (pretrained) | ✅ | ✅ | ❌ | | QA-Based Information Extraction (pretrained) | ✅ | ✅ | ❌ | | Zero-Shot Classification (pretrained) | ❌ | ✅ | ❌ | | Language Translation (pretrained) | ❌ | ✅ | ❌ | | Summarization (pretrained) | ❌ | ✅ | ❌ | | Speech Transcription (pretrained) | ❌ | ✅ |❌ | | Image Captioning (pretrained) | ❌ | ✅ |❌ | | Object Detection (pretrained) | ❌ | ✅ |❌ | | Sentiment Analysis (pretrained) | ❌ | ✅ |❌ | | GenerativeAI (sentence-transformers) | ❌ | ✅ |❌ | | Topic Modeling (sklearn) | ❌ | ❌ | ✅ | | Keyphrase Extraction (textblob/nltk/sklearn) | ❌ | ❌ | ✅ |

As noted above, end-to-end question-answering and information extraction in ktrain can be used with either TensorFlow (using framework='tf') or PyTorch (using framework='pt').

How to Cite

Please cite the following paper when using ktrain: ``` @article{maiya2020ktrain, title={ktrain: A Low-Code Library for Augmented Machine Learning}, author={Arun S. Maiya}, year={2020}, eprint={2004.10703}, archivePrefix={arXiv}, primaryClass={cs.LG}, journal={arXiv preprint arXiv:2004.10703}, }

```

---

Creator: Arun S. Maiya

Email: arun [at] maiya [dot] net