gobbli
gobbli: A uniform interface to deep learning for text in Python - Published in JOSS (2021)
Science Score: 93.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
✓DOI references
Found 4 DOI reference(s) in README and JOSS metadata -
✓Academic publication links
Links to: zenodo.org -
○Committers with academic emails
-
○Institutional organization owner
-
✓JOSS paper metadata
Published in Journal of Open Source Software
Keywords
Repository
Deep learning with text doesn't have to be scary.
Basic Info
Statistics
- Stars: 275
- Watchers: 25
- Forks: 22
- Open Issues: 13
- Releases: 12
Topics
Metadata Files
README.md
This is a library designed to provide a uniform interface to various deep learning models for text via programmatically created Docker containers.
Usage
See the docs for prerequisites, a quickstart, and the API reference. In brief, you need Docker installed with appropriate permissions for your user account to run Docker commands and Python 3.7. Then run the following:
pip install gobbli
You may also want to check out the benchmarks to see some comparisons of gobbli's implementation of various models in different situations.
Interactive
gobbli provides streamlit apps to perform some interactive tasks in a web browser, such as data exploration and model evaluation. Once you've installed the library, you can run the bundled apps using the gobbli command line application. Check the docs for more information.
Development
Assuming you have all prerequisites noted above, you need to install the package and all required + optional dependencies in development mode:
pip install -e ".[augment,tokenize,interactive]"
Install additional dev dependencies:
pip install -r requirements.txt
Run linting, autoformatting, and tests:
./run_ci.sh
To avoid manually fixing some of these errors, consider enabling isort and black support in your favorite editor.
If you're running tests in an environment with less than 12GB of memory, you'll want to pass the --low-resource argument when running tests to avoid out of memory errors.
NOTE: If running on a Mac, even with adequate memory available, you may encounter Out of Memory errors (exit status 137) when running the tests. This is due to not enough memory being allocated to your Docker daemon. Try going to Docker for Mac -> Preferences -> Advanced and raising "Memory" to 12GiB or more.
If you want to run the tests GPU(s) enabled, see the --use-gpu and --nvidia-visible-devices arguments under py.test --help. If your local machine doesn't have an NVIDIA GPU, but you have access to one that does via SSH, you can use the test_remote_gpu.sh script to run the tests with GPU enabled over SSH.
Docs
To generate the docs, install the docs requirements:
pip install -r docs/requirements.txt
Since doc structure is auto-generated from the library, you must have the library (and all its dependencies) installed as well.
Then, run the following from the repository root:
./generate_docs.sh
Then browse the generated documentation in docs/_build/html.
Attribution
gobbli wouldn't exist without the public release of several state-of-the-art models. The library incorporates:
- BERT, released by Google
- MT-DNN, released by Microsoft
- Universal Sentence Encoder, released by Google
- fastText, released by Facebook
- transformers, released by Hugging Face
- spaCy, by Explosion
- scikit-learn
Original work on the library was funded by RTI International.
Logo design by Marcia Underwood.
Owner
- Name: RTI International
- Login: RTIInternational
- Kind: organization
- Location: Worldwide
- Website: https://www.rti.org/
- Repositories: 26
- Profile: https://github.com/RTIInternational
JOSS Publication
gobbli: A uniform interface to deep learning for text in Python
Authors
RTI International
RTI International
Editor
Arfon Smith
Tags
deep learning data science classification natural language processingGitHub Events
Total
- Watch event: 1
Last Year
- Watch event: 1
Committers
Last synced: 7 months ago
Top Committers
| Name | Commits | |
|---|---|---|
| Jason Nance | j****e@r****g | 218 |
| Jason Nance | j****e@r****g | 6 |
| Peter B | 5****r | 4 |
Committer Domains (Top 20 + Academic)
Issues and Pull Requests
Last synced: 6 months ago
All Time
- Total issues: 14
- Total pull requests: 18
- Average time to close issues: 2 days
- Average time to close pull requests: about 1 month
- Total issue authors: 9
- Total pull request authors: 4
- Average comments per issue: 1.64
- Average comments per pull request: 0.22
- Merged pull requests: 13
- Bot issues: 0
- Bot pull requests: 4
Past Year
- Issues: 0
- Pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Issue authors: 0
- Pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Top Authors
Issue Authors
- jasonnance (6)
- pmbaumgartner (1)
- jacobdanovitch (1)
- rcalsaverini (1)
- cfculhane (1)
- theccalderon (1)
- swayson (1)
- erichare (1)
- AschHarwood (1)
Pull Request Authors
- jasonnance (11)
- dependabot[bot] (4)
- pmbaumgartner (2)
- TrellixVulnTeam (1)
Top Labels
Issue Labels
Pull Request Labels
Packages
- Total packages: 1
-
Total downloads:
- pypi 41 last-month
- Total dependent packages: 0
- Total dependent repositories: 1
- Total versions: 11
- Total maintainers: 1
pypi.org: gobbli
Uniform interface to deep learning approaches via Docker containers.
- Homepage: https://github.com/RTIInternational/gobbli/
- Documentation: https://gobbli.readthedocs.io/
- License: LICENSE.txt
-
Latest release: 0.2.4
published over 4 years ago
Rankings
Maintainers (1)
Dependencies
- boto3 *
- colorlog *
- numpy *
- pytorch-pretrained-bert ==v0.6.0
- torch ==0.4.1
- tqdm *
- PyYAML ==5.3
- matplotlib ==3.1.3
- selenium ==3.141.0
- tabulate ==0.8.6
- umap-learn ==0.3.10
- autodoc ==0.5.0
- gobbli *
- mock ==3.0.5
- sphinx ==2.1.0
- sphinx-autobuild ==0.7.1
- sphinx-autodoc-typehints ==1.6.0
- sphinx-paramlinks ==0.3.7
- tensorflow >=1.11.0
- cupy-cuda101 ==7.0.0
- pandas ==0.25.0
- requests ==2.25.1
- sentencepiece ==0.1.86
- spacy ==2.2.1
- spacy-transformers ==0.5.1
- thinc_gpu_ops ==0.0.4
- urllib3 >=1.25.4,<1.27
- pandas ==0.25.0
- sentencepiece ==0.1.86
- transformers ==2.8.0
- tensorflow-hub ==0.7.0
- aiohttp ==3.5.4
- black ==19.3b0
- epc ==0.0.5
- flake8 ==3.7.7
- importmagic ==0.1.7
- isort ==4.3.20
- mypy ==0.720
- pytest ==4.5.0
- ray ==0.8.5
- setuptools ==41.0.1
- twine ==1.13.0
- wheel ==0.33.6
- altair *
- click *
- docker *
- humanize *
- numpy *
- pandas *
- ray *
- scikit-learn *
- python ${PYTHON_VERSION} build
- python 3.7 build
- gobbli-benchmark latest
- gobbli-ci latest
- gobbli-ci latest
- pytorch/pytorch 1.3-cuda10.1-cudnn7-runtime build
- pytorch/pytorch 1.3-cuda10.1-cudnn7-runtime build
- tensorflow/tensorflow 1.11.0${GPU build
- ubuntu 18.04 build
- allenlao/pytorch-mt-dnn v0.1 build
- nvidia/cuda 9.0-cudnn7-runtime-ubuntu16.04 build
- pytorch/pytorch 1.3-cuda10.1-cudnn7-devel build
- pytorch/pytorch 1.3-cuda10.1-cudnn7-runtime build
- tensorflow/tensorflow 2.0.1${GPU build