Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/dmlc/gluon-nlp

NLP made easy

https://github.com/dmlc/gluon-nlp

Science Score: 10.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ○
    codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ✓
    Committers with academic emails
    5 of 97 committers (5.2%) from academic institutions
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (16.0%) to scientific vocabulary

Keywords

deep-learning gluon gluonnlp machine-learning mxnet natural-language-generation natural-language-inference natural-language-processing natural-language-understanding nlg nlp nlu numpy

Keywords from Contributors

action-recognition gan person-reid pose-estimation semantic-segmentation tensor deep-neural-networks hyperparameter-optimization automl transfer-learning
Last synced: 6 months ago · JSON representation

Repository

NLP made easy

Basic Info
  • Host: GitHub
  • Owner: dmlc
  • License: apache-2.0
  • Language: Python
  • Default Branch: master
  • Homepage: https://nlp.gluon.ai/
  • Size: 10.9 MB
Statistics
  • Stars: 2,559
  • Watchers: 94
  • Forks: 530
  • Open Issues: 272
  • Releases: 15
Archived
Topics
deep-learning gluon gluonnlp machine-learning mxnet natural-language-generation natural-language-inference natural-language-processing natural-language-understanding nlg nlp nlu numpy
Created almost 8 years ago · Last pushed over 2 years ago
Metadata Files
Readme License Code of conduct Codeowners

README.md

GluonNLP Logo

GluonNLP: Your Choice of Deep Learning for NLP

GluonNLP is a toolkit that helps you solve NLP problems. It provides easy-to-use tools that helps you load the text data, process the text data, and train models.

See our documents at https://nlp.gluon.ai/master/index.html.

Features

  • Easy-to-use Text Processing Tools and Modular APIs
  • Pretrained Model Zoo
  • Write Models with Numpy-like API
  • Fast Inference via Apache TVM (incubating) (Experimental)
  • AWS Integration via SageMaker

Installation

First of all, install the MXNet 2 release such as MXNet 2 Alpha. You may use the following commands:

```bash

Install the version with CUDA 10.2

python3 -m pip install -U --pre "mxnet-cu102>=2.0.0a"

Install the version with CUDA 11

python3 -m pip install -U --pre "mxnet-cu110>=2.0.0a"

Install the cpu-only version

python3 -m pip install -U --pre "mxnet>=2.0.0a" ```

To install GluonNLP, use

```bash python3 -m pip install -U -e .

Also, you may install all the extra requirements via

python3 -m pip install -U -e ."[extras]" ```

If you find that you do not have the permission, you can also install to the user folder:

bash python3 -m pip install -U -e . --user

For Windows users, we recommend to use the Windows Subsystem for Linux.

Access the Command-line Toolkits

To facilitate both the engineers and researchers, we provide command-line-toolkits for downloading and processing the NLP datasets. For more details, you may refer to GluonNLP Datasets and GluonNLP Data Processing Tools.

```bash

CLI for downloading / preparing the dataset

nlp_data help

CLI for accessing some common data processing scripts

nlp_process help

Also, you can use python -m to access the toolkits

python3 -m gluonnlp.cli.data help python3 -m gluonnlp.cli.process help

```

Run Unittests

You may go to tests to see how to run the unittests.

Use Docker

You can use Docker to launch a JupyterLab development environment with GluonNLP installed.

```

GPU Instance

docker pull gluonai/gluon-nlp:gpu-latest docker run --gpus all --rm -it -p 8888:8888 -p 8787:8787 -p 8786:8786 --shm-size=2g gluonai/gluon-nlp:gpu-latest

CPU Instance

docker pull gluonai/gluon-nlp:cpu-latest docker run --rm -it -p 8888:8888 -p 8787:8787 -p 8786:8786 --shm-size=2g gluonai/gluon-nlp:cpu-latest ```

For more details, you can refer to the guidance in tools/docker.

Owner

  • Name: Distributed (Deep) Machine Learning Community
  • Login: dmlc
  • Kind: organization

A Community of Awesome Machine Learning Projects

GitHub Events

Total
  • Watch event: 15
  • Fork event: 2
Last Year
  • Watch event: 15
  • Fork event: 2

Committers

Last synced: about 2 years ago

All Time
  • Total Commits: 838
  • Total Committers: 97
  • Avg Commits per committer: 8.639
  • Development Distribution Score (DDS): 0.844
Past Year
  • Commits: 0
  • Committers: 0
  • Avg Commits per committer: 0.0
  • Development Distribution Score (DDS): 0.0
Top Committers
Name Email Commits
Sheng Zha s****a 131
Haibin Lin l****c@g****m 131
Leonard Lausen l****d@l****l 107
Leonard Lausen l****n@a****m 86
Shuai Zheng s****c 44
Xingjian Shi x****b@c****k 43
barry-jin 6****n 33
cgwang w****u@g****m 18
Xingjian Shi x****b@u****k 15
Aston Zhang 2****g 12
Zheyu Ye 3****e 12
liuzh91 l****1@g****m 12
Ziyue Huang z****e@a****g 9
zburning 7****1@q****m 9
ht w****a@a****p 8
Sheng Zha z****g@a****m 8
Sergey Sokolov S****v@g****m 6
WuKangjian 1****5@q****m 6
hhexiy h****y@g****m 6
MoisesHer 5****r 5
paperplanet z****2@g****m 5
夏鲁豫 f****x@o****m 5
Zhi Sun 4****4 4
rongruosong s****8@g****m 4
Yongyi (Ethan) Wu w****y@c****u 4
Mu Li m****i@a****m 4
Heewon Jeon(gogamza) g****a@g****m 4
Tao Lv t****v@i****m 3
Siyuan Liu l****2@g****m 3
Shuai Zheng s****c@c****k 3
and 67 more...
Committer Domains (Top 20 + Academic)
amazon.com: 6 intel.com: 4 qq.com: 3 126.com: 3 connect.ust.hk: 2 lausen.nl: 1 ust.hk: 1 apache.org: 1 akane.waseda.jp: 1 cmu.edu: 1 emory.edu: 1 163.com: 1 post.eurasia.edu: 1 bluewin.ch: 1 sbcglobal.net: 1 ucdavis.edu: 1 de-vries.id.au: 1 craftworkz.be: 1 hello.com: 1 hexiaoting.com: 1 research.iiit.ac.in: 1

Issues and Pull Requests

Last synced: 10 months ago

All Time
  • Total issues: 44
  • Total pull requests: 59
  • Average time to close issues: 4 months
  • Average time to close pull requests: about 1 month
  • Total issue authors: 28
  • Total pull request authors: 18
  • Average comments per issue: 2.61
  • Average comments per pull request: 3.51
  • Merged pull requests: 43
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 0
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • sxjscience (9)
  • barry-jin (5)
  • szha (3)
  • araitats (2)
  • leezu (2)
  • hb0313 (1)
  • m0dulo (1)
  • zheyuye (1)
  • Ishitori (1)
  • makua-bernal (1)
  • stratosphere1492 (1)
  • fierceX (1)
  • bartekkuncer (1)
  • preeyank5 (1)
  • eric-haibin-lin (1)
Pull Request Authors
  • barry-jin (25)
  • szha (6)
  • leezu (5)
  • sxjscience (3)
  • bgawrych (3)
  • bartekkuncer (3)
  • DOUDOU0314 (2)
  • xinyual (2)
  • johnhe-dev (1)
  • TrellixVulnTeam (1)
  • gongel (1)
  • AetherPrior (1)
  • pigooosuke (1)
  • hutao965 (1)
  • DominikaJedynak (1)
Top Labels
Issue Labels
bug (26) enhancement (13) good first issue (7) help wanted (5) documentation (2) discussion (1) API change (1)
Pull Request Labels

Packages

  • Total packages: 2
  • Total downloads:
    • pypi 19,894 last-month
  • Total docker downloads: 499
  • Total dependent packages: 9
    (may contain duplicates)
  • Total dependent repositories: 224
    (may contain duplicates)
  • Total versions: 45
  • Total maintainers: 4
pypi.org: gluonnlp

MXNet Gluon NLP Toolkit

  • Versions: 25
  • Dependent Packages: 9
  • Dependent Repositories: 224
  • Downloads: 19,894 Last month
  • Docker Downloads: 499
Rankings
Dependent repos count: 1.0%
Downloads: 1.1%
Dependent packages count: 1.1%
Stargazers count: 1.4%
Average: 1.4%
Docker downloads count: 1.8%
Forks count: 2.3%
Maintainers (4)
mxnet mxnet-ci szha leezu
Last synced: 6 months ago
proxy.golang.org: github.com/dmlc/gluon-nlp
  • Versions: 20
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent packages count: 5.5%
Average: 5.7%
Dependent repos count: 5.9%
Last synced: 6 months ago

Dependencies

scripts/benchmarks/requirements.txt pypi
  • py3nvml *
  • torch *
  • torchvision *
  • transformers *
.github/workflows/buildwebsite.yml actions
  • actions/checkout v2 composite
  • actions/setup-python v2 composite
  • actions/upload-artifact v2 composite
  • aws-actions/configure-aws-credentials v1 composite
  • peter-evans/create-or-update-comment v1 composite
.github/workflows/data-pipeline.yml actions
  • actions/checkout v2 composite
  • aws-actions/configure-aws-credentials v1 composite
.github/workflows/nightly-test.yml actions
  • actions/checkout v2 composite
  • actions/upload-artifact v2 composite
  • aws-actions/configure-aws-credentials v1 composite
.github/workflows/unittests-gpu.yml actions
  • actions/checkout v2 composite
  • actions/setup-python v2 composite
  • actions/upload-artifact v2 composite
  • aws-actions/configure-aws-credentials v1 composite
.github/workflows/unittests.yml actions
  • actions/cache v2 composite
  • actions/checkout v2 composite
  • actions/setup-python v2 composite
  • actions/upload-artifact v2 composite
setup.py pypi