rosetta
Tools, wrappers, etc... for data science with a concentration on text processing
Science Score: 10.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
○codemeta.json file
-
○.zenodo.json file
-
○DOI references
-
○Academic publication links
-
✓Committers with academic emails
2 of 12 committers (16.7%) from academic institutions -
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (18.4%) to scientific vocabulary
Repository
Tools, wrappers, etc... for data science with a concentration on text processing
Basic Info
- Host: GitHub
- Owner: columbia-applied-data-science
- License: other
- Language: Jupyter Notebook
- Default Branch: master
- Size: 1.41 MB
Statistics
- Stars: 206
- Watchers: 21
- Forks: 46
- Open Issues: 16
- Releases: 0
Metadata Files
README.md
Rosetta
Tools for data science with a focus on text processing.
- Focuses on "medium data", i.e. data too big to fit into memory but too small to necessitate the use of a cluster.
- Integrates with existing scientific Python stack as well as select outside tools.
Examples
- See the
examples/directory. - The docs contain plots of example output.
Packages
cmdutils
- Unix-like command line utilities. Filters (read from stdin/write to stdout) for files.
- Focus on stream processing and csv files.
parallel
- Wrappers for Python multiprocessing that add ease of use
- Memory-friendly multiprocessing
text
- Stream text from disk to formats used in common ML processes
- Write processed text to sparse formats
- Helpers for ML tools (e.g. Vowpal Wabbit, Gensim, etc...)
- Other general utilities
workflow
- High-level wrappers that have helped with our workflow and provide additional examples of code use
modeling
- General ML modeling utilities
Install
Check out the master branch from the rosettarepo. Then, (so long as you have pip).
cd rosetta
make
make test
If you update the source, you can do
make reinstall
make test
The above make targets use pip, so you can of course do pip uninstall at any time.
Getting the source (above) is the preferred method since the code changes often, but if you don't use Git you can download a tagged release (tarball) here. Then
pip install rosetta-X.X.X.tar.gz
Development
Code
You can get the latest sources with
git clone git://github.com/columbia-applied-data-science/rosetta
Contributing
Feel free to contribute a bug report or a request by opening an issue
The preferred method to contribute is to fork and send a pull request. Before doing this, read CONTRIBUTING.md
Dependencies
- Major dependencies on Pandas and numpy.
- Minor dependencies on Gensim and statsmodels.
- Some examples need scikit-learn.
- Minor dependencies on docx
- Minor dependencies on the unix utilities pdftotext and catdoc
Testing
From the base repo directory, rosetta/, you can run all tests with
make test
Documentation
Documentation for releases is hosted at pypi. This does NOT auto-update.
History
Rosetta refers to the Rosetta Stone, the ancient Egyptian tablet discovered just over 200 years ago. The tablet contained fragmented text in three different languages and the uncovering of its meaning is considered an essential key to our understanding of Ancient Egyptian civilization. We would like this project to provide individuals the necessary tools to process and unearth insight in the ever-growing volumes of textual data of today.
Owner
- Name: columbia-applied-data-science
- Login: columbia-applied-data-science
- Kind: organization
- Website: columbia-applied-data-science.github.com
- Repositories: 3
- Profile: https://github.com/columbia-applied-data-science
GitHub Events
Total
Last Year
Committers
Last synced: almost 3 years ago
Top Committers
| Name | Commits | |
|---|---|---|
| dkrasner | d****r@m****u | 95 |
| Ian Langmore | i****e@g****m | 72 |
| Matt DeLand | m****d@g****m | 16 |
| Thomas Nyberg | t****3@c****u | 15 |
| Thomas Nyberg | t****g@g****m | 6 |
| Louis Clark | s****b@g****m | 5 |
| Matt DeLand | m****d@g****m | 4 |
| dkrasner | k****l@g****m | 4 |
| David van Coevorden | d****o@g****m | 2 |
| rafacarrascosa | r****a@g****m | 1 |
| Thomas Nyberg | n****g@o****g | 1 |
| Joshua Brooks | j****k@g****m | 1 |
Committer Domains (Top 20 + Academic)
Issues and Pull Requests
Last synced: about 1 year ago
All Time
- Total issues: 21
- Total pull requests: 35
- Average time to close issues: 29 days
- Average time to close pull requests: 15 days
- Total issue authors: 11
- Total pull request authors: 9
- Average comments per issue: 3.86
- Average comments per pull request: 1.49
- Merged pull requests: 30
- Bot issues: 0
- Bot pull requests: 0
Past Year
- Issues: 0
- Pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Issue authors: 0
- Pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Top Authors
Issue Authors
- dkrasner (6)
- langmore (5)
- ApproximateIdentity (2)
- AllardJM (1)
- Eickho (1)
- metasyn (1)
- jquacinella (1)
- davaco (1)
- BrianMiner (1)
- binhngoc17 (1)
- bhaskar2khaneja (1)
Pull Request Authors
- ApproximateIdentity (9)
- dkrasner (8)
- mdeland (7)
- langmore (6)
- rafacarrascosa (1)
- joshbrooks (1)
- davefol (1)
- zigeuner (1)
- davaco (1)
Top Labels
Issue Labels
Pull Request Labels
Packages
- Total packages: 2
-
Total downloads:
- pypi 1,659 last-month
-
Total dependent packages: 1
(may contain duplicates) -
Total dependent repositories: 49
(may contain duplicates) - Total versions: 12
- Total maintainers: 2
pypi.org: rosetta
Tools, wrappers, etc... for data science with a concentration on text processing
- Homepage: https://github.com/columbia-applied-data-science/rosetta
- Documentation: https://rosetta.readthedocs.io/
- License: BSD
-
Latest release: 0.2.5
published over 12 years ago
Rankings
proxy.golang.org: github.com/columbia-applied-data-science/rosetta
- Documentation: https://pkg.go.dev/github.com/columbia-applied-data-science/rosetta#section-documentation
- License: other
-
Latest release: v0.3.0
published almost 11 years ago
Rankings
Dependencies
- numpydoc *
- MySQL-python *
- docx *
- gensim *
- pandas *
- pymongo *
- pyth *
- scikit-learn *
- scipy *
- statsmodels *