Data

Tools

Indexes

Applications

Experiments

Open Source Science

dataproperty

A Python library for extract property from data.

https://github.com/thombashi/dataproperty

Science Score: 26.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Committers with academic emails
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (10.0%) to scientific vocabulary

Keywords

data property python python-library
Last synced: 6 months ago · JSON representation

Repository

A Python library for extract property from data.

Basic Info
Statistics
  • Stars: 16
  • Watchers: 2
  • Forks: 5
  • Open Issues: 2
  • Releases: 46
Topics
data property python python-library
Created about 10 years ago · Last pushed 8 months ago
Metadata Files
Readme License

README.rst 

.. contents:: **DataProperty**
   :backlinks: top
   :local:


Summary
=======
A Python library for extract property from data.


.. image:: https://badge.fury.io/py/DataProperty.svg
    :target: https://badge.fury.io/py/DataProperty
    :alt: PyPI package version

.. image:: https://anaconda.org/conda-forge/DataProperty/badges/version.svg
    :target: https://anaconda.org/conda-forge/DataProperty
    :alt: conda-forge package version

.. image:: https://img.shields.io/pypi/pyversions/DataProperty.svg
   :target: https://pypi.org/project/DataProperty
    :alt: Supported Python versions

.. image:: https://img.shields.io/pypi/implementation/DataProperty.svg
    :target: https://pypi.org/project/DataProperty
    :alt: Supported Python implementations

.. image:: https://github.com/thombashi/DataProperty/actions/workflows/ci.yml/badge.svg
    :target: https://github.com/thombashi/DataProperty/actions/workflows/ci.yml
    :alt: CI status of Linux/macOS/Windows

.. image:: https://coveralls.io/repos/github/thombashi/DataProperty/badge.svg?branch=master
    :target: https://coveralls.io/github/thombashi/DataProperty?branch=master
    :alt: Test coverage

.. image:: https://github.com/thombashi/DataProperty/actions/workflows/github-code-scanning/codeql/badge.svg
    :target: https://github.com/thombashi/DataProperty/actions/workflows/github-code-scanning/codeql
    :alt: CodeQL


Installation
============

Installation: pip
------------------------------
::

    pip install DataProperty

Installation: conda
------------------------------
::

    conda install -c conda-forge dataproperty

Installation: apt
------------------------------
::

    sudo add-apt-repository ppa:thombashi/ppa
    sudo apt update
    sudo apt install python3-dataproperty


Usage
=====

Extract property of data
------------------------

e.g. Extract a ``float`` value property
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: python

    >>> from dataproperty import DataProperty
    >>> DataProperty(-1.1)
    data=-1.1, type=REAL_NUMBER, align=right, ascii_width=4, int_digits=1, decimal_places=1, extra_len=1

e.g. Extract a ``int`` value property
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: python

    >>> from dataproperty import DataProperty
    >>> DataProperty(123456789)
    data=123456789, type=INTEGER, align=right, ascii_width=9, int_digits=9, decimal_places=0, extra_len=0

e.g. Extract a ``str`` (ascii) value property
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: python

    >>> from dataproperty import DataProperty
    >>> DataProperty("sample string")
    data=sample string, type=STRING, align=left, length=13, ascii_width=13, extra_len=0

e.g. Extract a ``str`` (multi-byte) value property
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: python

    >>> from dataproperty import DataProperty
    >>> str(DataProperty("吾輩は猫である"))
    data=吾輩は猫である, type=STRING, align=left, length=7, ascii_width=14, extra_len=0

e.g. Extract a time (``datetime``) value property
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: python

    >>> import datetime
    >>> from dataproperty import DataProperty
    >>> DataProperty(datetime.datetime(2017, 1, 1, 0, 0, 0))
    data=2017-01-01 00:00:00, type=DATETIME, align=left, ascii_width=19, extra_len=0

e.g. Extract a ``bool`` value property
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: python

    >>> from dataproperty import DataProperty
    >>> DataProperty(True)
    data=True, type=BOOL, align=left, ascii_width=4, extra_len=0


Extract data property for each element from a matrix
----------------------------------------------------
``DataPropertyExtractor.to_dp_matrix`` method returns a matrix of ``DataProperty`` instances from a data matrix.
An example data set and the result are as follows:

:Sample Code:
    .. code:: python

        import datetime
        from dataproperty import DataPropertyExtractor

        dp_extractor = DataPropertyExtractor()
        dt = datetime.datetime(2017, 1, 1, 0, 0, 0)
        inf = float("inf")
        nan = float("nan")

        dp_matrix = dp_extractor.to_dp_matrix([
            [1, 1.1, "aa", 1, 1, True, inf, nan, dt],
            [2, 2.2, "bbb", 2.2, 2.2, False, "inf", "nan", dt],
            [3, 3.33, "cccc", -3, "ccc", "true", inf, "NAN", "2017-01-01T01:23:45+0900"],
        ])

        for row, dp_list in enumerate(dp_matrix):
            for col, dp in enumerate(dp_list):
                print("row={:d}, col={:d}, {}".format(row, col, str(dp)))

:Output:
    ::

        row=0, col=0, data=1, type=INTEGER, align=right, ascii_width=1, int_digits=1, decimal_places=0, extra_len=0
        row=0, col=1, data=1.1, type=REAL_NUMBER, align=right, ascii_width=3, int_digits=1, decimal_places=1, extra_len=0
        row=0, col=2, data=aa, type=STRING, align=left, ascii_width=2, length=2, extra_len=0
        row=0, col=3, data=1, type=INTEGER, align=right, ascii_width=1, int_digits=1, decimal_places=0, extra_len=0
        row=0, col=4, data=1, type=INTEGER, align=right, ascii_width=1, int_digits=1, decimal_places=0, extra_len=0
        row=0, col=5, data=True, type=BOOL, align=left, ascii_width=4, extra_len=0
        row=0, col=6, data=Infinity, type=INFINITY, align=left, ascii_width=8, extra_len=0
        row=0, col=7, data=NaN, type=NAN, align=left, ascii_width=3, extra_len=0
        row=0, col=8, data=2017-01-01 00:00:00, type=DATETIME, align=left, ascii_width=19, extra_len=0
        row=1, col=0, data=2, type=INTEGER, align=right, ascii_width=1, int_digits=1, decimal_places=0, extra_len=0
        row=1, col=1, data=2.2, type=REAL_NUMBER, align=right, ascii_width=3, int_digits=1, decimal_places=1, extra_len=0
        row=1, col=2, data=bbb, type=STRING, align=left, ascii_width=3, length=3, extra_len=0
        row=1, col=3, data=2.2, type=REAL_NUMBER, align=right, ascii_width=3, int_digits=1, decimal_places=1, extra_len=0
        row=1, col=4, data=2.2, type=REAL_NUMBER, align=right, ascii_width=3, int_digits=1, decimal_places=1, extra_len=0
        row=1, col=5, data=False, type=BOOL, align=left, ascii_width=5, extra_len=0
        row=1, col=6, data=Infinity, type=INFINITY, align=left, ascii_width=8, extra_len=0
        row=1, col=7, data=NaN, type=NAN, align=left, ascii_width=3, extra_len=0
        row=1, col=8, data=2017-01-01 00:00:00, type=DATETIME, align=left, ascii_width=19, extra_len=0
        row=2, col=0, data=3, type=INTEGER, align=right, ascii_width=1, int_digits=1, decimal_places=0, extra_len=0
        row=2, col=1, data=3.33, type=REAL_NUMBER, align=right, ascii_width=4, int_digits=1, decimal_places=2, extra_len=0
        row=2, col=2, data=cccc, type=STRING, align=left, ascii_width=4, length=4, extra_len=0
        row=2, col=3, data=-3, type=INTEGER, align=right, ascii_width=2, int_digits=1, decimal_places=0, extra_len=1
        row=2, col=4, data=ccc, type=STRING, align=left, ascii_width=3, length=3, extra_len=0
        row=2, col=5, data=True, type=BOOL, align=left, ascii_width=4, extra_len=0
        row=2, col=6, data=Infinity, type=INFINITY, align=left, ascii_width=8, extra_len=0
        row=2, col=7, data=NaN, type=NAN, align=left, ascii_width=3, extra_len=0
        row=2, col=8, data=2017-01-01T01:23:45+0900, type=STRING, align=left, ascii_width=24, length=24, extra_len=0


Full example source code can be found at *examples/py/to_dp_matrix.py*


Extract properties for each column from a matrix
------------------------------------------------------
``DataPropertyExtractor.to_column_dp_list`` method returns a list of ``DataProperty`` instances from a data matrix. The list represents the properties for each column.
An example data set and the result are as follows:

Example data set and result are as follows:

:Sample Code:
    .. code:: python

        import datetime
        from dataproperty import DataPropertyExtractor

        dp_extractor = DataPropertyExtractor()
        dt = datetime.datetime(2017, 1, 1, 0, 0, 0)
        inf = float("inf")
        nan = float("nan")

        data_matrix = [
            [1, 1.1,  "aa",   1,   1,     True,   inf,   nan,   dt],
            [2, 2.2,  "bbb",  2.2, 2.2,   False,  "inf", "nan", dt],
            [3, 3.33, "cccc", -3,  "ccc", "true", inf,   "NAN", "2017-01-01T01:23:45+0900"],
        ]

        dp_extractor.headers = ["int", "float", "str", "num", "mix", "bool", "inf", "nan", "time"]
        col_dp_list = dp_extractor.to_column_dp_list(dp_extractor.to_dp_matrix(dp_matrix))

        for col_idx, col_dp in enumerate(col_dp_list):
            print(str(col_dp))

:Output:
    ::

        column=0, type=INTEGER, align=right, ascii_width=3, bit_len=2, int_digits=1, decimal_places=0
        column=1, type=REAL_NUMBER, align=right, ascii_width=5, int_digits=1, decimal_places=(min=1, max=2)
        column=2, type=STRING, align=left, ascii_width=4
        column=3, type=REAL_NUMBER, align=right, ascii_width=4, int_digits=1, decimal_places=(min=0, max=1), extra_len=(min=0, max=1)
        column=4, type=STRING, align=left, ascii_width=3, int_digits=1, decimal_places=(min=0, max=1)
        column=5, type=BOOL, align=left, ascii_width=5
        column=6, type=INFINITY, align=left, ascii_width=8
        column=7, type=NAN, align=left, ascii_width=3
        column=8, type=STRING, align=left, ascii_width=24


Full example source code can be found at *examples/py/to_column_dp_list.py*


Dependencies
============
- Python 3.9+
- `Python package dependencies (automatically installed) `__

Optional dependencies
---------------------
- `loguru `__
    - Used for logging if the package installed

Owner

  • Name: Tsuyoshi Hombashi
  • Login: thombashi
  • Kind: user
  • Location: Japan

💮 🦉💤💤

GitHub Events

Total
  • Create event: 5
  • Issues event: 1
  • Release event: 2
  • Watch event: 1
  • Delete event: 1
  • Issue comment event: 3
  • Push event: 13
  • Pull request event: 3
Last Year
  • Create event: 5
  • Issues event: 1
  • Release event: 2
  • Watch event: 1
  • Delete event: 1
  • Issue comment event: 3
  • Push event: 13
  • Pull request event: 3

Committers

Last synced: 8 months ago

All Time
  • Total Commits: 1,405
  • Total Committers: 3
  • Avg Commits per committer: 468.333
  • Development Distribution Score (DDS): 0.002
Past Year
  • Commits: 37
  • Committers: 1
  • Avg Commits per committer: 37.0
  • Development Distribution Score (DDS): 0.0
Top Committers
Name Email Commits
Tsuyoshi Hombashi t****i@g****m 1,402
Hugo van Kemenade h****k 2
Yegor Yefremov y****s@g****m 1

Issues and Pull Requests

Last synced: 8 months ago

All Time
  • Total issues: 3
  • Total pull requests: 57
  • Average time to close issues: 6 months
  • Average time to close pull requests: about 13 hours
  • Total issue authors: 3
  • Total pull request authors: 4
  • Average comments per issue: 3.33
  • Average comments per pull request: 0.21
  • Merged pull requests: 53
  • Bot issues: 0
  • Bot pull requests: 1
Past Year
  • Issues: 0
  • Pull requests: 1
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 1
  • Average comments per issue: 0
  • Average comments per pull request: 0.0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 1
View more stats
Top Authors
Issue Authors
  • benjaminrigaud (1)
  • thombashi (1)
  • yegorich (1)
Pull Request Authors
  • thombashi (54)
  • dependabot[bot] (2)
  • hugovk (1)
  • yegorich (1)
Top Labels
Issue Labels
documentation (1)
Pull Request Labels
bug (13) enhancement (12) refactoring (9) documentation (6) dependencies (2) github_actions (2)

Packages

  • Total packages: 2
  • Total downloads:
    • pypi 2,123,220 last-month
  • Total docker downloads: 44
  • Total dependent packages: 2
    (may contain duplicates)
  • Total dependent repositories: 77
    (may contain duplicates)
  • Total versions: 156
  • Total maintainers: 1
pypi.org: dataproperty

Python library for extract property from data.

  • Versions: 147
  • Dependent Packages: 0
  • Dependent Repositories: 77
  • Downloads: 2,123,220 Last month
  • Docker Downloads: 44
Rankings
Downloads: 0.5%
Dependent repos count: 1.7%
Dependent packages count: 7.4%
Average: 7.7%
Forks count: 14.3%
Stargazers count: 14.9%
Maintainers (1)
thombashi
Last synced: 6 months ago
conda-forge.org: dataproperty
  • Versions: 9
  • Dependent Packages: 2
  • Dependent Repositories: 0
Rankings
Dependent packages count: 19.5%
Dependent repos count: 34.0%
Average: 36.9%
Stargazers count: 46.3%
Forks count: 47.7%
Last synced: 6 months ago

Dependencies

requirements/docs_requirements.txt pypi
  • Sphinx *
  • sphinx_rtd_theme *
requirements/requirements.txt pypi
  • mbstrdecoder >=1.0.0,<2
  • typepy >=1.2.0,<2
requirements/test_requirements.txt pypi
  • pytest >=6.0.1 test
  • pytest-md-report >=0.1 test
  • termcolor * test
setup.py pypi
  • line.strip *
.github/workflows/ci.yml actions
  • actions/checkout v4 composite
  • actions/setup-python v5 composite
  • actions/upload-artifact v4 composite
.github/workflows/on_push_default_branch.yml actions
  • actions/checkout v4 composite
  • actions/download-artifact v4 composite
  • actions/setup-python v5 composite
  • actions/upload-artifact v4 composite
  • pypa/gh-action-pypi-publish release/v1 composite
  • sigstore/gh-action-sigstore-python v3.0.0 composite
.github/workflows/publish.yml actions
  • actions/checkout v4 composite
  • actions/download-artifact v4 composite
  • actions/setup-python v5 composite
  • actions/upload-artifact v4 composite
  • pypa/gh-action-pypi-publish release/v1 composite
  • sigstore/gh-action-sigstore-python v3.0.0 composite
  • softprops/action-gh-release v2 composite
pyproject.toml pypi