Data

Tools

Indexes

Applications

Experiments

Open Source Science

preprocessing

python pre-processing package for data analysis

https://github.com/spotlightdata/preprocessing

Science Score: 13.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Committers with academic emails
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (14.9%) to scientific vocabulary
Last synced: 11 months ago · JSON representation

Repository

python pre-processing package for data analysis

Basic Info
  • Host: GitHub
  • Owner: SpotlightData
  • License: mit
  • Language: Python
  • Default Branch: master
  • Size: 12.7 MB
Statistics
  • Stars: 8
  • Watchers: 3
  • Forks: 3
  • Open Issues: 4
  • Releases: 0
Created almost 9 years ago · Last pushed over 2 years ago
Metadata Files
Readme License

README.rst 

.. image:: logo.png
   :align: center
   :alt: Spotlight Data Logo

'preprocessing'
===============
.. image:: https://readthedocs.org/projects/preprocessing/badge/?version=latest
   :target: http://preprocessing.readthedocs.io/en/latest/?badge=latest
   :alt: Documentation Status

Summary
-------

Text pre-processing package to aid in NLP package development for Python3. With this package you 
can order text cleaning functions in the order you prefer rather than relying on the order of an 
arbitrary NLP package.

Installation
------------

pip:

.. code-block:: console

   pip install preprocessing

PyPI - You can also download the source distribution from:

`https://pypi.python.org/pypi/preprocessing/ 
`_

You can then perform:

.. code-block:: console

   pip install 

on the tar file, or

.. code-block:: console
   
   python setup.py install

on/inside, respectively, the extracted package to install *preprocessing*.

Example
-------

Once you have the package installed, implementing it with Python3 takes the following form:

.. code-block:: python

   import preprocessing.text as ptext
   from preprocessing.text import keyword_tokenize, remove_unbound_punct, remove_urls

   text_string = "important string at: http://example.com"

   clean_string = ptext.preprocess_text(text_string, [
       remove_urls,
       remove_unbound_punct,
       keyword_tokenize
   ])

>>> print(clean_string)
"important string"

Should the functions be performed in a different order (i.e. keyword_tokenize -> remove_urls -> 
remove_non_bound_punct) :

>>> print(clean_string)
"important string http example.com"

Organisation
------------

This package is comprised of a single module with no intended subpackages currently. The 
*preprocessing* package is dependent on NLTK for tokenizers and stopwords. However, ignoring this,
the package only has built-in dependencies from Python 3.

Contributing
------------

If you feel like contributing:

* `Check for open issues `_ or open a new issue
* Fork the preprocessing repository to start making your changes
* Write a test which shows the bug was fixed or that the feature works as expected
* Send a pull request and remember to add yourself to `CONTRIBUTORS.md `_

License
-------

This project is licensed under the MIT license (see `LICENSE `_)

Owner

  • Name: Spotlight Data
  • Login: SpotlightData
  • Kind: organization
  • Location: Nottingham UK

Spotlight Data's Open source projects.

GitHub Events

Total
Last Year

Committers

Last synced: almost 3 years ago

All Time
  • Total Commits: 68
  • Total Committers: 2
  • Avg Commits per committer: 34.0
  • Development Distribution Score (DDS): 0.279
Past Year
  • Commits: 0
  • Committers: 0
  • Avg Commits per committer: 0.0
  • Development Distribution Score (DDS): 0.0
Top Committers
Name Email Commits
mwtmurphy m****h@s****k 49
mwtmurphy m****y@g****m 19
Committer Domains (Top 20 + Academic)
spotlightdata.co.uk: 1

Issues and Pull Requests

Last synced: over 1 year ago

All Time
  • Total issues: 2
  • Total pull requests: 2
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Total issue authors: 2
  • Total pull request authors: 2
  • Average comments per issue: 0.5
  • Average comments per pull request: 0.0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 1
Past Year
  • Issues: 0
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • djlindsay87 (1)
  • mwtmurphy (1)
Pull Request Authors
  • cerob (2)
  • dependabot[bot] (1)
Top Labels
Issue Labels
enhancement (1)
Pull Request Labels
dependencies (1)

Packages

  • Total packages: 1
  • Total downloads:
    • pypi 8,781 last-month
  • Total docker downloads: 558
  • Total dependent packages: 0
  • Total dependent repositories: 89
  • Total versions: 2
  • Total maintainers: 1
pypi.org: preprocessing

pre-processing package for text strings

  • Versions: 2
  • Dependent Packages: 0
  • Dependent Repositories: 89
  • Downloads: 8,781 Last month
  • Docker Downloads: 558
Rankings
Dependent repos count: 1.6%
Docker downloads count: 1.7%
Downloads: 4.1%
Average: 9.3%
Dependent packages count: 10.0%
Forks count: 19.1%
Stargazers count: 19.3%
Maintainers (1)
spotlightdata
Last synced: 12 months ago

Dependencies

requirements.txt pypi
  • nltk ==3.2.4
  • sphinx-rtd-theme ==0.2.4
setup.py pypi