Data

Tools

Indexes

Applications

Experiments

Open Source Science

langdetect

Port of Google's language-detection library to Python.

https://github.com/mimino666/langdetect

Science Score: 36.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ✓
    Committers with academic emails
    1 of 11 committers (9.1%) from academic institutions
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (11.9%) to scientific vocabulary
Last synced: 10 months ago · JSON representation

Repository

Port of Google's language-detection library to Python.

Basic Info
  • Host: GitHub
  • Owner: Mimino666
  • License: other
  • Language: Python
  • Default Branch: master
  • Size: 979 KB
Statistics
  • Stars: 1,835
  • Watchers: 24
  • Forks: 210
  • Open Issues: 70
  • Releases: 1
Created about 12 years ago · Last pushed over 1 year ago
Metadata Files
Readme License

README.md

langdetect

Build Status

Port of Nakatani Shuyo's language-detection library (version from 03/03/2014) to Python.

Installation

$ pip install langdetect

Supported Python versions 2.7, 3.4+.

Languages

langdetect supports 55 languages out of the box (ISO 639-1 codes):

af, ar, bg, bn, ca, cs, cy, da, de, el, en, es, et, fa, fi, fr, gu, he,
hi, hr, hu, id, it, ja, kn, ko, lt, lv, mk, ml, mr, ne, nl, no, pa, pl,
pt, ro, ru, sk, sl, so, sq, sv, sw, ta, te, th, tl, tr, uk, ur, vi, zh-cn, zh-tw

Basic usage

To detect the language of the text:

```python

from langdetect import detect detect("War doesn't show who's right, just who's left.") 'en' detect("Ein, zwei, drei, vier") 'de' ```

To find out the probabilities for the top languages:

```python

from langdetect import detectlangs detectlangs("Otec matka syn.") [sk:0.572770823327, pl:0.292872522702, cs:0.134356653968] ```

NOTE

Language detection algorithm is non-deterministic, which means that if you try to run it on a text which is either too short or too ambiguous, you might get different results everytime you run it.

To enforce consistent results, call following code before the first language detection:

python from langdetect import DetectorFactory DetectorFactory.seed = 0

How to add new language?

You need to create a new language profile. The easiest way to do it is to use the langdetect.jar tool, which can generate language profiles from Wikipedia abstract database files or plain text.

Wikipedia abstract database files can be retrieved from "Wikipedia Downloads" (http://download.wikimedia.org/). They form '(language code)wiki-(version)-abstract.xml' (e.g. 'enwiki-20101004-abstract.xml' ).

usage: java -jar langdetect.jar --genprofile -d [directory path] [language codes]

  • Specify the directory which has abstract databases by -d option.
  • This tool can handle gzip compressed file.

Remark: The database filename in Chinese is like 'zhwiki-(version)-abstract-zh-cn.xml' or zhwiki-(version)-abstract-zh-tw.xml', so that it must be modified 'zh-cnwiki-(version)-abstract.xml' or 'zh-twwiki-(version)-abstract.xml'.

To generate language profile from a plain text, use the genprofile-text command.

usage: java -jar langdetect.jar --genprofile-text -l [language code] [text file path]

For more details see language-detection Wiki.

Original project

This library is a direct port of Google's language-detection library from Java to Python. All the classes and methods are unchanged, so for more information see the project's website or wiki.

Presentation of the language detection algorithm: http://www.slideshare.net/shuyo/language-detection-library-for-java.

Owner

  • Name: Michal Danilák
  • Login: Mimino666
  • Kind: user
  • Location: Slovakia
  • Company: Monitora Media s.r.o.

CTO @monitora-media

GitHub Events

Total
  • Commit comment event: 1
  • Issues event: 4
  • Watch event: 113
  • Issue comment event: 10
  • Push event: 1
  • Pull request event: 3
  • Fork event: 12
Last Year
  • Commit comment event: 1
  • Issues event: 4
  • Watch event: 113
  • Issue comment event: 10
  • Push event: 1
  • Pull request event: 3
  • Fork event: 12

Committers

Last synced: about 1 year ago

All Time
  • Total Commits: 56
  • Total Committers: 11
  • Avg Commits per committer: 5.091
  • Development Distribution Score (DDS): 0.375
Past Year
  • Commits: 1
  • Committers: 1
  • Avg Commits per committer: 1.0
  • Development Distribution Score (DDS): 0.0
Top Committers
Name Email Commits
Michal Danilak m****r@g****m 35
hugovk h****k 12
lingwndr c****r@g****m 1
Pierre-Hugues Husson p****h@p****e 1
Marc Abramowitz m****c@m****m 1
Jonathan Klaassen j****n@g****m 1
Joe Wreschnig j****g@g****m 1
Autom3 A****3 1
Arumugam a****n@y****m 1
Patrick Robertson p****n@b****k 1
Jay Chan j****y@e****m 1
Committer Domains (Top 20 + Academic)
eventbrite.com: 1 bangor.ac.uk: 1 marc-abramowitz.com: 1 phh.me: 1

Issues and Pull Requests

Last synced: 10 months ago

All Time
  • Total issues: 74
  • Total pull requests: 36
  • Average time to close issues: 5 months
  • Average time to close pull requests: 6 months
  • Total issue authors: 67
  • Total pull request authors: 28
  • Average comments per issue: 1.7
  • Average comments per pull request: 0.69
  • Merged pull requests: 11
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 4
  • Pull requests: 2
  • Average time to close issues: 5 minutes
  • Average time to close pull requests: 2 minutes
  • Issue authors: 4
  • Pull request authors: 2
  • Average comments per issue: 0.25
  • Average comments per pull request: 0.5
  • Merged pull requests: 1
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • DonaldTsang (3)
  • anubhav0fnu (3)
  • dpriskorn (2)
  • ziky90 (2)
  • andy-d (2)
  • rspeer (1)
  • wojciech-galan (1)
  • paulfranceschi (1)
  • mimi89999 (1)
  • topspinj (1)
  • kvi24 (1)
  • versant2612 (1)
  • grestonian (1)
  • vors (1)
  • joseberlines (1)
Pull Request Authors
  • hugovk (7)
  • Ou7law007 (2)
  • Autom3 (2)
  • danilo-augusto (2)
  • bildfell (2)
  • whnr (1)
  • grischard (1)
  • KorayUlusan (1)
  • lingwndr (1)
  • Anushavasa15 (1)
  • phhusson (1)
  • joewreschnig (1)
  • veelion (1)
  • Sinkler (1)
  • cikay (1)
Top Labels
Issue Labels
Pull Request Labels

Packages

  • Total packages: 2
  • Total downloads:
    • pypi 7,390,868 last-month
  • Total docker downloads: 12,438,168
  • Total dependent packages: 189
    (may contain duplicates)
  • Total dependent repositories: 2,006
    (may contain duplicates)
  • Total versions: 14
  • Total maintainers: 1
pypi.org: langdetect

Language detection library ported from Google's language-detection.

  • Versions: 11
  • Dependent Packages: 183
  • Dependent Repositories: 1,998
  • Downloads: 7,390,868 Last month
  • Docker Downloads: 12,438,168
Rankings
Dependent packages count: 0.2%
Downloads: 0.2%
Dependent repos count: 0.2%
Docker downloads count: 0.6%
Average: 1.1%
Stargazers count: 1.7%
Forks count: 3.7%
Maintainers (1)
Michal.Danilk
Last synced: 10 months ago
conda-forge.org: langdetect
  • Versions: 3
  • Dependent Packages: 6
  • Dependent Repositories: 8
Rankings
Dependent packages count: 9.0%
Stargazers count: 10.6%
Average: 11.3%
Dependent repos count: 12.1%
Forks count: 13.3%
Last synced: 10 months ago

Dependencies

requirements.txt pypi
  • six *
setup.py pypi
  • six *