WordTokenizers.jl - v0.5.6

WordTokenizers v0.5.6

Diff since v0.5.5

Merged pull requests: - Adopt ColPrac? (#54) (@oxinabox) - use a normal function in init to intialize the data deps (#56) (@KristofferC)

Scientific Software - Peer-reviewed - Julia
Published by github-actions[bot] almost 6 years ago

WordTokenizers.jl - v0.5.5

WordTokenizers v0.5.5

Diff since v0.5.4

Merged pull requests: - Update paper.bib (#47) (@kthyng) - Update paper.md (#48) (@kthyng) - Install TagBot as a GitHub Action (#49) (@JuliaTagBot) - Adding support for unigram sentencepiece model (#51) (@tejasvaidhyadev) - Update to version 0.5.5. (#53) (@Ayushk4)

Scientific Software - Peer-reviewed - Julia
Published by github-actions[bot] almost 6 years ago

WordTokenizers.jl - WordTokenizers.jl: Basic tools for tokenizing natural language in Julia

Scientific Software - Peer-reviewed - Julia
Published by oxinabox over 6 years ago

WordTokenizers.jl - v0.5.4

v0.5.4 (2020-02-06)

Diff since v0.5.3

Merged pull requests:

• Update paper based on JOSS review (#45) (oxinabox)
• Add installation guide to README (#43) (Ayushk4)
• Change example setting tokenizer to TinySegmenter.jl's tokenizer (#42) (Ayushk4)
• Fixing a number of typos in paper and readme (#41) (leios)
• Minor Fixes in JOSS paper (#40) (Ayushk4)
• very minor grammar fixes in README (#39) (danielskatz)
• Add plot comparing speeds of tokenizers to JOSS paper. (#36) (Ayushk4)
• Support and Contribution guidelines (#35) (Ayushk4)
• JOSS paper update (#34) (Ayushk4)
• Write paper for JOSS (#7) (oxinabox)

Scientific Software - Peer-reviewed - Julia
Published by julia-tagbot[bot] over 6 years ago

WordTokenizers.jl - v0.5.3

v0.5.3 (2019-06-28)

Diff since v0.5.2

Merged pull requests:

• Handle final periods (#33) (Ayushk4)
• Toktok fix patch (#31) (Ayushk4)
• Update for Julia-1.1 (#30) (Ayushk4)
• Fix sentence splitter: sentences ending with acronyms (#24) (nickto)

Scientific Software - Peer-reviewed - Julia
Published by julia-tagbot[bot] almost 7 years ago

WordTokenizers.jl - v0.5.2

v0.5.2 (2019-06-21)

Diff since v0.5.1

Closed issues:

• Tokenize begins with full stop. (#28)

Merged pull requests:

• Fix TokTok.jl (#29) (Ayushk4)

Scientific Software - Peer-reviewed - Julia
Published by julia-tagbot[bot] about 7 years ago

WordTokenizers.jl - v0.5.1

v0.5.1 (2019-06-06)

Diff since v0.5.0

Closed issues:

• Make a release (#26)
• Add a Twitter tokenizer (#3)

Merged pull requests:

• Add Tweet Tokenizer (#13) (Ayushk4)

Scientific Software - Peer-reviewed - Julia
Published by julia-tagbot[bot] about 7 years ago

WordTokenizers.jl - v0.5.0

v0.5.0 (2019-06-06)

Diff since v0.4.0

Closed issues:

• Add TokTok tokenizer (#15)

Merged pull requests:

• Fix inconsistency between tabs and spaces (#25) (Ayushk4)
• appveyor badge fix (#23) (aquatiko)
• Fix indentation in nltk_word.jl (#22) (Ayushk4)
• Fix indentation. (#21) (Ayushk4)
• Minor doc fixes in fast.jl (#20) (Ayushk4)
• some cleanup, inclusing changing TokenBuffer to use replaces rather than splits (#19) (oxinabox)
• add toktok tokenizer (#18) (aquatiko)

Scientific Software - Peer-reviewed - Julia
Published by julia-tagbot[bot] about 7 years ago

WordTokenizers.jl - v0.4.0

• fix typo in name of reversable_tokenize

Scientific Software - Peer-reviewed - Julia
Published by oxinabox over 7 years ago

WordTokenizers.jl - v0.3.1

• Add reversable tokenizer

Scientific Software - Peer-reviewed - Julia
Published by oxinabox over 7 years ago

WordTokenizers.jl - v0.3.0

Muuuuuch faster.

Scientific Software - Peer-reviewed - Julia
Published by oxinabox over 7 years ago

WordTokenizers.jl - v0.2.0

• Julia 0.7 / 1.0 compat

Scientific Software - Peer-reviewed - Julia
Published by oxinabox over 7 years ago

WordTokenizers.jl - v0.1.0

Scientific Software - Peer-reviewed - Julia
Published by oxinabox about 8 years ago