Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/climatecompatiblegrowth/research_index_backend

Provisioning the graph database which forms the backend of the research index

https://github.com/climatecompatiblegrowth/research_index_backend

Science Score: 26.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (9.4%) to scientific vocabulary
Last synced: 9 months ago · JSON representation

Repository

Provisioning the graph database which forms the backend of the research index

Basic Info
  • Host: GitHub
  • Owner: ClimateCompatibleGrowth
  • License: mit
  • Language: Jupyter Notebook
  • Default Branch: main
  • Size: 329 KB
Statistics
  • Stars: 0
  • Watchers: 1
  • Forks: 0
  • Open Issues: 17
  • Releases: 0
Created about 2 years ago · Last pushed about 1 year ago
Metadata Files
Readme License

readme.md

Ingest and Natural Language Processing of Research Outputs

The package is not yet deployed to PyPI. Only an editable (development) install is possible.

  1. Provide a list of DOIs in a CSV file format list_of_dois.csv
  2. Clone the repository git clonehttps://github.com/ClimateCompatibleGrowth/research_index_backend.git
  3. Change directory cd research_index_backend
  4. Install the package pip install -e . as an editable package (development install)

  5. Obtain an OpenAIRE Graph refresh token and create a .env file with the following parameters: MG_HOST= MG_PORT= MG_PORT_ALT= MG_USER= MG_PASS= ORCID_NAME_SIMILARITY_THRESHOLD= NAME_SIMILARITY_THRESHOLD= OPENAIRE_API="https://api.openaire.eu" OPENAIRE_SERVICE="https://services.openaire.eu" REFRESH_TOKEN=

  6. Provision Memgraph graph database and set up environment variables

Once the VM is up and running, SSH into the VM, download and install memgraph

    $ curl -O https://download.memgraph.com/memgraph/v2.14.1/ubuntu-20.04/memgraph_2.14.1-1_amd64.deb
    $ sudo dpkg -i /memgraph_2.14.1-1_amd64.deb

  1. Run the backend:

    research_index --help
usage: research_index [-h] [-i] [-l LIMIT] [-u] list_of_dois

positional arguments:
  list_of_dois          Path to CSV file containing list of DOIs

options:
  -h, --help            Show this help message and exit
  -i, --initialise      Delete existing data and create new database
  -l, --limit N         Limit number of DOIs to process (default: 50)
  -u, --update-metadata Update metadata for existing DOIs
  -w, --write-metadata  Save JSON responses to disk 

Examples:
  -> Process 10 DOIs from file:
  $ research_index list_of_dois.csv -l 10  # Process 10 DOIs from file

  -> Update metadata for existing DOIs and save metadata
  $ research_index list_of_dois.csv --update-metadata --write-metadata

Development

The package is maintained using hatch.

To run the tests run:

    hatch test

Owner

  • Name: Climate Compatible Growth
  • Login: ClimateCompatibleGrowth
  • Kind: organization
  • Location: United Kingdom

GitHub Events

Total
  • Issues event: 17
  • Delete event: 9
  • Issue comment event: 13
  • Push event: 47
  • Pull request review comment event: 18
  • Pull request review event: 18
  • Pull request event: 22
  • Create event: 11
Last Year
  • Issues event: 17
  • Delete event: 9
  • Issue comment event: 13
  • Push event: 47
  • Pull request review comment event: 18
  • Pull request review event: 18
  • Pull request event: 22
  • Create event: 11

Issues and Pull Requests

Last synced: 10 months ago

All Time
  • Total issues: 8
  • Total pull requests: 7
  • Average time to close issues: 25 days
  • Average time to close pull requests: 9 days
  • Total issue authors: 2
  • Total pull request authors: 2
  • Average comments per issue: 0.13
  • Average comments per pull request: 0.71
  • Merged pull requests: 5
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 8
  • Pull requests: 7
  • Average time to close issues: 25 days
  • Average time to close pull requests: 9 days
  • Issue authors: 2
  • Pull request authors: 2
  • Average comments per issue: 0.13
  • Average comments per pull request: 0.71
  • Merged pull requests: 5
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • willu47 (13)
  • FrancisTembo (5)
Pull Request Authors
  • willu47 (8)
  • FrancisTembo (5)
Top Labels
Issue Labels
enhancement (4) bug (3)
Pull Request Labels
enhancement (2) bug (1)

Dependencies

pyproject.toml pypi
  • gqlalchemy *
  • pandas *
  • rdflib *
  • requests *
  • tqdm *
environment.yaml pypi
  • anyio ==3.6.2
  • appnope ==0.1.3
  • argon2-cffi ==21.3.0
  • argon2-cffi-bindings ==21.2.0
  • arrow ==1.2.3
  • asttokens ==2.2.1
  • attrs ==22.2.0
  • backcall ==0.2.0
  • beautifulsoup4 ==4.11.2
  • bleach ==6.0.0
  • certifi ==2022.12.7
  • cffi ==1.15.1
  • charset-normalizer ==3.0.1
  • comm ==0.1.2
  • contourpy ==1.0.7
  • cycler ==0.11.0
  • debugpy ==1.6.6
  • decorator ==5.1.1
  • defusedxml ==0.7.1
  • et-xmlfile ==1.1.0
  • executing ==1.2.0
  • fastjsonschema ==2.16.2
  • filelock ==3.9.0
  • fonttools ==4.38.0
  • fqdn ==1.5.1
  • huggingface-hub ==0.12.0
  • idna ==3.4
  • ipykernel ==6.21.2
  • ipython ==8.10.0
  • ipython-genutils ==0.2.0
  • ipywidgets ==8.0.4
  • isoduration ==20.11.0
  • jedi ==0.18.2
  • jinja2 ==3.1.2
  • jsonpointer ==2.3
  • jsonschema ==4.17.3
  • jupyter ==1.0.0
  • jupyter-client ==8.0.3
  • jupyter-console ==6.6.1
  • jupyter-core ==5.2.0
  • jupyter-events ==0.6.3
  • jupyter-server ==2.3.0
  • jupyter-server-terminals ==0.4.4
  • jupyterlab-pygments ==0.2.2
  • jupyterlab-widgets ==3.0.5
  • kiwisolver ==1.4.4
  • markupsafe ==2.1.2
  • matplotlib ==3.7.0
  • matplotlib-inline ==0.1.6
  • mistune ==2.0.5
  • nbclassic ==0.5.2
  • nbclient ==0.7.2
  • nbconvert ==7.2.9
  • nbformat ==5.7.3
  • nest-asyncio ==1.5.6
  • notebook ==6.5.2
  • notebook-shim ==0.2.2
  • openpyxl ==3.1.0
  • packaging ==23.0
  • pandas ==1.5.3
  • pandocfilters ==1.5.0
  • parso ==0.8.3
  • pexpect ==4.8.0
  • pickleshare ==0.7.5
  • platformdirs ==3.0.0
  • prometheus-client ==0.16.0
  • prompt-toolkit ==3.0.37
  • psutil ==5.9.4
  • ptyprocess ==0.7.0
  • pure-eval ==0.2.2
  • pycparser ==2.21
  • pygments ==2.14.0
  • pyparsing ==3.0.9
  • pyrsistent ==0.19.3
  • python-dateutil ==2.8.2
  • python-json-logger ==2.0.7
  • pytz ==2022.7.1
  • pyyaml ==6.0
  • pyzmq ==25.0.0
  • qtconsole ==5.4.0
  • qtpy ==2.3.0
  • regex ==2022.10.31
  • requests ==2.28.2
  • rfc3339-validator ==0.1.4
  • rfc3986-validator ==0.1.1
  • seaborn ==0.12.2
  • send2trash ==1.8.0
  • sniffio ==1.3.0
  • soupsieve ==2.4
  • stack-data ==0.6.2
  • terminado ==0.17.1
  • tinycss2 ==1.2.1
  • tokenizers ==0.13.2
  • tornado ==6.2
  • tqdm ==4.64.1
  • traitlets ==5.9.0
  • transformers ==4.26.1
  • uri-template ==1.2.0
  • urllib3 ==1.26.14
  • wcwidth ==0.2.6
  • webcolors ==1.12
  • webencodings ==0.5.1
  • websocket-client ==1.5.1
  • widgetsnbextension ==4.0.5