https://github.com/alan-turing-institute/ptype

Probabilistic type inference

Science Score: 33.0%

This score indicates how likely this project is to be science-related based on various indicators:

○
CITATION.cff file
○
codemeta.json file
○
.zenodo.json file
✓
DOI references
Found 4 DOI reference(s) in README
✓
Academic publication links
Links to: springer.com
✓
Committers with academic emails
2 of 21 committers (9.5%) from academic institutions
○
Institutional organization owner
○
JOSS paper metadata
○
Scientific vocabulary similarity
Low similarity (12.9%) to scientific vocabulary

Keywords

hut23 hut23-438

Last synced: 10 months ago · JSON representation

Repository

Probabilistic type inference

Basic Info

Host: GitHub
Owner: alan-turing-institute
License: mit
Language: Jupyter Notebook
Default Branch: develop
Homepage: https://pypi.org/project/ptype/
Size: 20.6 MB

Statistics

Stars: 29
Watchers: 3
Forks: 4
Open Issues: 9
Releases: 0

Topics

hut23 hut23-438

Created almost 7 years ago · Last pushed over 4 years ago

Metadata Files

Readme License

README.rst

.. image:: https://github.com/alan-turing-institute/ptype/workflows/build-publish/badge.svg?branch=release
    :target: https://github.com/alan-turing-institute/ptype/actions?query=workflow%3Abuild-publish+branch%3Arelease
    :alt: build-publish on release

.. image:: https://github.com/alan-turing-institute/ptype/workflows/build/badge.svg?branch=develop
    :target: https://github.com/alan-turing-institute/ptype/actions?query=workflow%3Abuild+branch%3Adevelop
    :alt: build on develop

.. image:: https://badge.fury.io/py/ptype.svg
    :target: https://badge.fury.io/py/ptype
    :alt: PyPI version

.. image:: https://readthedocs.org/projects/ptype/badge/?version=stable
    :target: https://ptype.readthedocs.io/en/stable/
    :alt: Documentation status

.. image:: https://pepy.tech/badge/ptype
    :target: https://pepy.tech/project/ptype
    :alt: Downloads

.. image:: https://mybinder.org/badge_logo.svg
    :target: https://mybinder.org/v2/gh/alan-turing-institute/ptype/release?filepath=notebooks
    :alt: Binder

============
Introduction
============

.. sectnum::

.. contents::

ptype is a probabilistic approach to *type inference*, which is the task of identifying the data type (e.g. Boolean, date, integer or string) of a given column of data.

Existing approaches often fail on type inference for messy datasets where data is missing or anomalous. With ptype_, our goal is to develop a robust method that can deal with such data.

.. figure:: https://raw.githubusercontent.com/alan-turing-institute/ptype/release/notes/motivation.png
    :width: 400

    Normal, missing and anomalous values are denoted by green, yellow and red, respectively in the right hand figure.

.. _ptype: https://link.springer.com/content/pdf/10.1007/s10618-020-00680-1.pdf

ptype uses `Probabilistic Finite-State Machines`_ (PFSMs) to model known data types, missing and anomalous data. Given a column of data, we can infer a plausible column type, and also identify any values which (conditional on that type) are deemed missing or anomalous. In contrast to more familiar finite-state machines, such as regular expressions, that either accept or reject a given data value, PFSMs assign probabilities to different values. They therefore offer the advantage of generating weighted predictions when a column of messy data is consistent with more than one type assignment.

.. _`Probabilistic Finite-State Machines`: https://en.wikipedia.org/wiki/Probabilistic_automaton

If you use this package, please cite the `ptype paper`_, using the following BibTeX entry:

.. _`ptype paper`: http://doi.org/10.1007/s10618-020-00680-1

::

    @article{ceritli2020ptype,
      title={ptype: probabilistic type inference},
      author={Ceritli, Taha and Williams, Christopher KI and Geddes, James},
      journal={Data Mining and Knowledge Discovery},
      year={2020},
      volume = {34},
      number = {3},
      pages={870–-904},
      doi = {10.1007/s10618-020-00680-1},
    }

====================
Install requirements
====================

You can simply install ptype from PyPI:

.. code:: bash

    pip install ptype

=====
Usage
=====

See demo notebooks in ``notebooks`` folder. View them online via Binder_.

.. _Binder: https://mybinder.org/v2/gh/alan-turing-institute/ptype/release?filepath=notebooks

Owner

Name: The Alan Turing Institute
Login: alan-turing-institute
Kind: organization
Email: info@turing.ac.uk

Website: https://turing.ac.uk
Repositories: 477
Profile: https://github.com/alan-turing-institute

The UK's national institute for data science and artificial intelligence.

GitHub Events

Total

Issues event: 1

Last Year

Issues event: 1

Committers

Last synced: over 2 years ago

All Time

Total Commits: 993
Total Committers: 21
Avg Commits per committer: 47.286
Development Distribution Score (DDS): 0.163

Past Year

Commits: 0
Committers: 0
Avg Commits per committer: 0.0
Development Distribution Score (DDS): 0.0

Top Committers

Name	Email	Commits
Roly Perera	r**a@t**k	831
Taha Yusuf Ceritli	t**i@s**k	138
Gertjan van den Burg	g**g@g**m	6
runner	r**r@M**l	1
runner	r**r@M**l	1
runner	r**r@M**l	1
runner	r**r@M**l	1
runner	r**r@M**l	1
runner	r**r@M**l	1
runner	r**r@M**l	1
runner	r**r@M**l	1
runner	r**r@M**l	1
runner	r**r@M**l	1
runner	r**r@M**l	1
runner	r**r@M**l	1
Roly Perera	r**a@d**g	1
runner	r**r@M**l	1
runner	r**r@M**l	1
runner	r**r@M**l	1
runner	r**r@M**l	1
runner	r**r@M**l	1

Committer Domains (Top 20 + Academic)

dynamicaspects.org: 1 sms.ed.ac.uk: 1 turing.ac.uk: 1

Issues and Pull Requests

Last synced: 11 months ago

All Time

Total issues: 28
Total pull requests: 72
Average time to close issues: about 2 months
Average time to close pull requests: about 1 hour
Total issue authors: 4
Total pull request authors: 3
Average comments per issue: 1.18
Average comments per pull request: 0.29
Merged pull requests: 70
Bot issues: 0
Bot pull requests: 0

Past Year

Issues: 0
Pull requests: 0
Average time to close issues: N/A
Average time to close pull requests: N/A
Issue authors: 0
Pull request authors: 0
Average comments per issue: 0
Average comments per pull request: 0
Merged pull requests: 0
Bot issues: 0
Bot pull requests: 0

View more stats

Top Authors

Issue Authors

rolyp (21)
tahaceritli (5)
GjjvdBurg (1)
gihyukko (1)
mhauru (1)

Pull Request Authors

rolyp (43)
tahaceritli (28)
GjjvdBurg (1)

Top Labels

Issue Labels

task:core-api (10) task:use-cases (6) task:python-package (5) task:internal-design (5) bug (4) task:test-coverage (2)

Pull Request Labels

Packages

Total packages: 1
Total downloads:
- pypi 165 last-month

Total dependent packages: 0
Total dependent repositories: 1
Total versions: 19
Total maintainers: 2

pypi.org: ptype

Probabilistic type inference

Homepage: https://github.com/alan-turing-institute/ptype
Documentation: https://ptype.readthedocs.io/
License: MIT
Latest release: 0.2.17
published over 4 years ago

Versions: 19
Dependent Packages: 0
Dependent Repositories: 1
Downloads: 165 Last month

Rankings

Downloads: 7.7%

Dependent packages count: 10.1%

Stargazers count: 11.7%

Average: 13.3%

Forks count: 15.3%

Dependent repos count: 21.6%

Maintainers (2)

rolyp tahayc

Last synced: 11 months ago

ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/alan-turing-institute/ptype

Science Score: 33.0%

Keywords

Repository

Basic Info

Statistics

Topics

Metadata Files

README.rst

Owner

GitHub Events

Total

Last Year

Committers

All Time

Past Year

Top Committers

Committer Domains (Top 20 + Academic)

Issues and Pull Requests

All Time

Past Year

Top Authors

Issue Authors

Pull Request Authors

Top Labels

Issue Labels

Pull Request Labels

Packages

pypi.org: ptype

Rankings

Maintainers (2)