Data

Tools

Indexes

Applications

Experiments

Open Source Science

citationparser

https://github.com/mromanello/citationparser

Science Score: 28.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ✓
    CITATION.cff file
    Found CITATION.cff file
  • ○
    codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ✓
    Committers with academic emails
    1 of 2 committers (50.0%) from academic institutions
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (6.7%) to scientific vocabulary
Last synced: 11 months ago · JSON representation ·

Repository

Basic Info
  • Host: GitHub
  • Owner: mromanello
  • Language: Python
  • Default Branch: master
  • Homepage:
  • Size: 57.6 KB
Statistics
  • Stars: 7
  • Watchers: 4
  • Forks: 3
  • Open Issues: 1
  • Releases: 0
Created about 15 years ago · Last pushed over 7 years ago
Metadata Files
Readme Citation

README.rst 

CitationParser
==============

.. image:: https://badge.fury.io/py/citation-parser.svg
    :target: https://badge.fury.io/py/citation-parser

.. image:: https://travis-ci.org/mromanello/CitationParser.svg?branch=master
    :target: https://travis-ci.org/mromanello/CitationParser

Canonical references (e.g. "Hom. *Il.* 1,124-125") use punctuation symbols in a consistent way, meaning that we can define a formal grammar to process them. 

When encountering the reference "Hom. *Il.* 1,124-125", the human reader will parse it as follows:

* the text preceding the numbers contains information about work and author being cited
* the hyphen is used to specify a range of text passages, e.g. lines 124 to 125;
* the semicolon separates a reference from another within the sanme citation (is common to chain together references to mutiple of the same work or of different works);
* the comma separates the hierarchical levels of the work being cited. In the example above 1,124-5 stands for from Book 1, Line 124 to Book 1, Line 125
* when the citation scope is a range, the identical hierarchical level are collapsed: 1.124 - 1.125 can be written as both 1.124-125 or 1.124 s. without any loss of information for the human reader.

The CitationParser is composed by a lexer, a parser and a tree parser written in ANTLR and compiled into Python code. The parsed reference is then serialized into JSON.

An example:

    >>> cp = CitationParser()
    >>> cp.parse("Hom. Il. 1,124-125")
    [{'work': u'Hom. Il.', 'scp': {'start': ['1', '124'], 'end': ['1', '125']}

Owner

  • Name: Matteo Romanello
  • Login: mromanello
  • Kind: user
  • Location: Lausanne (CH)
  • Company: @dhlab-epfl (previously @dains and @kcl-ddh)

Researcher in Computational Humanities.

Citation (citation_parser/__init__.py) 

#!/usr/bin/python
# -*- coding: utf-8 -*-
# author: Matteo Romanello, matteo.romanello@gmail.com

from pyCTS import CTS_URN
from operator import itemgetter
import antlr3
import sys
import re
from antlr.cp_lexer import cp_lexer
from antlr.cp_parser import cp_parser
from antlr.cp_treeparser import cp_treeparser
try:
    import simplejson as json
except ImportError:
    import json as json
import logging

logger = logging.getLogger(__name__)
logger.addHandler(logging.NullHandler())

class CitationParser:
    """A wrapper for the ANTLR components (lexer, parser and tree parser)."""
    def __init__(self):
        logger.debug("citation_parser instance initialised")

    def parse(self, text):
        """TODO."""
        if(type(text) is not type("ü".decode('UTF-8'))):
            logger.info("converting input string to utf-8")
            text = text.decode('UTF-8')
        logger.debug(text)
        char_stream = antlr3.ANTLRStringStream(text)
        try:
            lexer = cp_lexer(char_stream)
            tokenstream = antlr3.CommonTokenStream(lexer)
            parser = cp_parser(tokenstream)
            try:
                r = parser.doc()
                root = r.tree
                print root.toStringTree()
                logger.debug(root.toStringTree())
                try:
                    nodes = antlr3.tree.CommonTreeNodeStream(root)
                    nodes.setTokenStream(tokenstream)
                    tp = cp_treeparser(nodes)
                    tp.doc()
                    logger.debug(tp.refs)
                    json_string = json.dumps(tp.refs)
                    return json.loads(json_string)
                except Exception as e:
                    logger.error("%s"%e)
                    logger.error("there was a problem w/ the TreeParser: exiting")
            except Exception as e:
                logger.error("%s"%e)
                logger.error("there was a problem with the parser: exiting")
        except Exception as e:
            logger.error("%s"%e)
            logger.error("there was a problem with the lexer: exiting")

    @staticmethod
    def scope2urn(urn, scope):
        """Build a passage-level CTS URN out of a URN and a parsed scope."""
        urns = []
        for s in scope:
            scp = s["scp"]
            if "start" in scp and "end" in scp:
                if len(scp["end"][-1]) < len(scp["start"][-1]):
                    diff = len(scp["start"][-1]) - len(scp["end"][-1])
                    scp["end"][-1] = scp["start"][-1][:diff] + scp["end"][-1]

                start = ".".join(scp["start"])
                end = ".".join(scp["end"])
                urns.append("{}:{}-{}".format(urn, start, end))
            else:
                start = ".".join(scp["start"])
                urns.append("{}:{}".format(urn, start))
        return urns

GitHub Events

Total
Last Year

Committers

Last synced: over 3 years ago

All Time
  • Total Commits: 24
  • Total Committers: 2
  • Avg Commits per committer: 12.0
  • Development Distribution Score (DDS): 0.042
Top Committers
Name Email Commits
mromanello m****o@g****m 23
matteo m****o@t****h 1
Committer Domains (Top 20 + Academic)
tsf-492-wpa-0-071.epfl.ch: 1

Issues and Pull Requests

Last synced: 12 months ago

All Time
  • Total issues: 1
  • Total pull requests: 1
  • Average time to close issues: over 6 years
  • Average time to close pull requests: N/A
  • Total issue authors: 1
  • Total pull request authors: 1
  • Average comments per issue: 0.0
  • Average comments per pull request: 0.0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 0
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • mromanello (1)
Pull Request Authors
  • ralic (1)
Top Labels
Issue Labels
Pull Request Labels

Packages

  • Total packages: 1
  • Total downloads:
    • pypi 21 last-month
  • Total dependent packages: 0
  • Total dependent repositories: 1
  • Total versions: 1
  • Total maintainers: 1
pypi.org: citation-parser

A parser for canonical references.

  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 1
  • Downloads: 21 Last month
Rankings
Dependent packages count: 10.1%
Forks count: 16.9%
Stargazers count: 19.4%
Average: 19.5%
Dependent repos count: 21.6%
Downloads: 29.6%
Maintainers (1)
mromanello
Last synced: 12 months ago

Dependencies

setup.py pypi
  • antlr-python-runtime *
  • pyCTS *