Data

Tools

Indexes

Applications

Experiments

Open Source Science

cpt

Compact Prediction Tree: A Lossless Model for Accurate Sequence Prediction (cython implementation)

https://github.com/bluesheeptoken/cpt

Science Score: 36.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ✓
    Committers with academic emails
    1 of 8 committers (12.5%) from academic institutions
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (15.1%) to scientific vocabulary
Last synced: 10 months ago · JSON representation

Repository

Compact Prediction Tree: A Lossless Model for Accurate Sequence Prediction (cython implementation)

Basic Info
Statistics
  • Stars: 41
  • Watchers: 6
  • Forks: 8
  • Open Issues: 2
  • Releases: 1
Created over 7 years ago · Last pushed 11 months ago
Metadata Files
Readme Contributing License

README.md

CPT

PyPI version Downloads License

What is it ?

This project is a cython open-source implementation of the Compact Prediction Tree algorithm using multithreading.

CPT is a sequence prediction model. It is a highly explainable model specialized in predicting the next element of a sequence over a finite alphabet.

This implementation is based on the following research papers:

  • http://www.philippe-fournier-viger.com/ADMA2013CompactPrediction_trees.pdf
  • http://www.philippe-fournier-viger.com/spmf/PAKDD2015CompactPrediction_tree+.pdf

Installation

You can simply use pip install cpt.

Simple example

You can test the model with the following code:

```python from cpt.cpt import Cpt model = Cpt()

model.fit([['hello', 'world'], ['hello', 'this', 'is', 'me'], ['hello', 'me'] ])

model.predict([['hello'], ['hello', 'this']])

Output: ['me', 'is']

``` For an example with the compatibility with sklearn, you should check the documentation.

Features

Train

The model can be trained with the fit method.

If needed the model can be retrained with the same method. It adds new sequences to the model and do not remove the old ones.

Multithreading

The predictions are launched by default with multithreading with OpenMP.

The predictions can also be launched in a single thread with the option multithread=False in the predict method.

You can control the number of threads by setting the following environment variable OMP_NUM_THREADS.

Pickling

You can pickle the model to save it, and load it later via pickle library. ```python from cpt.cpt import Cpt import pickle

model = Cpt() model.fit([['hello', 'world']])

dumped = pickle.dumps(model)

unpickled_model = pickle.loads(dumped)

print(model == unpickled_model) ```

Explainability

The CPT class has several methods to explain the predictions.

You can see which elements are considered as noise (with a low presence in sequences) with model.compute_noisy_items(noise_ratio).

You can retrieve trained sequences with model.retrieve_sequence(id).

You can find similar sequences with find_similar_sequences(sequence).

You can not yet retrieve automatically all similar sequences with the noise reduction technique.

Tuning

CPT has 3 meta parameters that need to be tuned. You can check how to tune them in the documentation. To tune you can use the model_selection module from sklearn, you can find an example here on how to.

Benchmark

The benchmark has been made on the FIFA dataset, the data can be found on the SPMF website.

Using multithreading, CPT was able to perform around 5000 predictions per second.

Without multithreading, CPT predicted around 1650 sequences per second.

Details on the benchmark can be found here.

Further reading

A study has been made on how to reduce dataset size, and so training / testing time using PageRank on the dataset.

The study has been published in IJIKM review here. An overall performance improvement of 10-40% has been observed with this technique on the prediction time without any accuracy loss.

One of the co-author of CPT has also published an algorithm subseq for sequence prediction. An implementation can be found here

Support

If you enjoy the project and wish to support me, a buymeacoffee link is available.

Owner

  • Name: Louis FRULEUX
  • Login: bluesheeptoken
  • Kind: user
  • Location: Paris
  • Company: Teads

GitHub Events

Total
  • Issues event: 1
  • Delete event: 2
  • Issue comment event: 7
  • Push event: 21
  • Pull request event: 6
  • Fork event: 1
  • Create event: 5
Last Year
  • Issues event: 1
  • Delete event: 2
  • Issue comment event: 7
  • Push event: 21
  • Pull request event: 6
  • Fork event: 1
  • Create event: 5

Committers

Last synced: over 2 years ago

All Time
  • Total Commits: 299
  • Total Committers: 8
  • Avg Commits per committer: 37.375
  • Development Distribution Score (DDS): 0.217
Past Year
  • Commits: 3
  • Committers: 2
  • Avg Commits per committer: 1.5
  • Development Distribution Score (DDS): 0.333
Top Committers
Name Email Commits
Bluesheeptoken l****1@g****m 234
kimci86 k****6@h****r 38
Louis Fruleux l****x@t****v 19
louis l****x@e****r 3
Louis Fruleux l****x@P****l 2
Louis Fruleux l****x@i****l 1
Louis Fruleux l****x@p****e 1
Sasha Kacanski s****i@g****m 1
Committer Domains (Top 20 + Academic)
par-mbp-louisf.home: 1 ip-192-168-1-32.eu-west-1.compute.internal: 1 etudiant.mines-nantes.fr: 1 teads.tv: 1

Issues and Pull Requests

Last synced: 10 months ago

All Time
  • Total issues: 34
  • Total pull requests: 77
  • Average time to close issues: about 2 months
  • Average time to close pull requests: 7 days
  • Total issue authors: 14
  • Total pull request authors: 4
  • Average comments per issue: 3.26
  • Average comments per pull request: 0.64
  • Merged pull requests: 69
  • Bot issues: 0
  • Bot pull requests: 2
Past Year
  • Issues: 1
  • Pull requests: 11
  • Average time to close issues: 8 months
  • Average time to close pull requests: about 1 month
  • Issue authors: 1
  • Pull request authors: 2
  • Average comments per issue: 2.0
  • Average comments per pull request: 1.09
  • Merged pull requests: 6
  • Bot issues: 0
  • Bot pull requests: 2
View more stats
Top Authors
Issue Authors
  • bluesheeptoken (13)
  • catchthemonster (5)
  • kimci86 (3)
  • uty07 (2)
  • navanchauhan (2)
  • sebastian-ho (1)
  • abdulbasitds (1)
  • QhenryQ (1)
  • rt3722 (1)
  • borgeser (1)
  • Sandy4321 (1)
  • hiddevanesch (1)
  • crbl1122 (1)
Pull Request Authors
  • bluesheeptoken (69)
  • catchthemonster (3)
  • kimci86 (3)
  • dependabot[bot] (2)
  • ShikovEgor (1)
Top Labels
Issue Labels
linux (2) pip-installation (2) source-installation (2) enhancement (2) performance optimization (2) python-3.9 (1) wontfix (1) documentation (1) macOS (1) good first issue (1)
Pull Request Labels
performance optimization (4) enhancement (2) dependencies (2) github_actions (2) documentation (1)

Packages

  • Total packages: 1
  • Total downloads:
    • pypi 24,835 last-month
  • Total dependent packages: 0
  • Total dependent repositories: 1
  • Total versions: 13
  • Total maintainers: 1
pypi.org: cpt

Compact Prediction Tree: A Lossless Model for Accurate Sequence Prediction

  • Versions: 13
  • Dependent Packages: 0
  • Dependent Repositories: 1
  • Downloads: 24,835 Last month
Rankings
Downloads: 2.1%
Dependent packages count: 10.0%
Stargazers count: 10.4%
Average: 11.5%
Forks count: 13.3%
Dependent repos count: 21.7%
Maintainers (1)
Bluesheeptoken
Last synced: 10 months ago

Dependencies

doc/requirements.txt pypi
  • cython >=0.20
  • numpydoc *
  • sphinx_rtd_theme *
.github/workflows/publish-pypi.yml actions
  • actions/checkout v3 composite
  • actions/checkout v2 composite
  • actions/download-artifact v2 composite
  • actions/setup-python v4 composite
  • actions/setup-python v2 composite
  • actions/upload-artifact v2 composite
  • pypa/gh-action-pypi-publish v1.4.2 composite
setup.py pypi