https://github.com/biolink/kgx

KGX is a Python library for exchanging Knowledge Graphs

Science Score: 36.0%

This score indicates how likely this project is to be science-related based on various indicators:

○
CITATION.cff file
✓
codemeta.json file
Found codemeta.json file
✓
.zenodo.json file
Found .zenodo.json file
○
DOI references
○
Academic publication links
✓
Committers with academic emails
2 of 26 committers (7.7%) from academic institutions
○
Institutional organization owner
○
JOSS paper metadata
○
Scientific vocabulary similarity
Low similarity (12.8%) to scientific vocabulary

Keywords

biolink-model knowledge-graph ncats-translator

Keywords from Contributors

obofoundry monarchinitiative owl rdf linkml ontology semantic-web neo4j standards linked-data

Last synced: 6 months ago · JSON representation

Repository

KGX is a Python library for exchanging Knowledge Graphs

Basic Info

Host: GitHub
Owner: biolink
License: bsd-3-clause
Language: Python
Default Branch: master
Homepage: https://biolink.github.io/kgx/
Size: 10.1 MB

Statistics

Stars: 131
Watchers: 21
Forks: 31
Open Issues: 54
Releases: 54

Topics

biolink-model knowledge-graph ncats-translator

Created almost 8 years ago · Last pushed 6 months ago

Metadata Files

Readme Changelog Contributing License Code of conduct

Knowledge Graph Exchange

KGX (Knowledge Graph Exchange) is a Python library and set of command line utilities for exchanging Knowledge Graphs (KGs) that conform to or are aligned to the Biolink Model.

The core datamodel is a Property Graph (PG), represented internally in Python using a networkx MultiDiGraph model.

KGX allows conversion to and from:

RDF serializations (read/write) and SPARQL endpoints (read)
Neo4j endpoints (read) or Neo4j dumps (write)
CSV/TSV and JSON (see associated data formats and example script to load CSV/TSV to Neo4j)
Reasoner Standard API format
OBOGraph JSON format

KGX will also provide validation, to ensure the KGs are conformant to the Biolink Model: making sure nodes are categorized using Biolink classes, edges are labeled using valid Biolink relationship types, and valid properties are used.

Internal representation is a property graph, specifically a networkx MultiDiGraph.

The structure of this graph is expected to conform to the Biolink Model standard, as specified in the KGX format specification.

In addition to the main code-base, KGX also provides a series of command line operations.

Example usage

Validate: bash poetry run kgx validate -i tsv tests/resources/merge/test2_nodes.tsv tests/resources/merge/test2_edges.tsv

Merge: bash poetry run kgx merge —merge-config tests/resources/test-merge.yaml

Graph Summary: bash poetry run kgx graph-summary -i tests/resources/graph_nodes.tsv -o summary.txt

Transform: bash poetry run kgx transform —transform-config tests/resources/test-transform-tsv-rdf.yaml

Error Detection and Reporting

Non-redundant JSON-formatted structured error logging is now provided in KGX Transformer, Validator, GraphSummary and MetaKnowledgeGraph operations. See the various unit tests for the general design pattern (using the Validator as an example here):

```python from kgx.validator import Validator from kgx.transformer import Transformer

Validator.setbiolinkmodel("2.11.0")

Validator assumes the currently set Biolink Release

validator = Validator()

transformer = Transformer(stream=True)

transformer.transform( inputargs = { "filename": [ "graphnodes.tsv", "graphedges.tsv", ], "format": "tsv", }, outputargs={ "format": "null" }, inspector=validator, )

Both the Validator and the Transformer can independently capture errors

The Validator, from the overall semantics of the graph...

Here, we just report severe Errors from the Validator (no Warnings)

validator.writereport(open("validationerrors.json", "w"), "Error")

The Transformer, from the syntax of the input files...

Here, we catch all Errors and Warnings (by not providing a filter)

transformer.writereport(open("inputerrors.json", "w")) ```

The JSON error outputs will look something like this:

```json { "ERROR": { "MISSINGEDGEPROPERTY": { "Required edge property 'id' is missing": [ "A:123->X:1", "B:456->Y:2" ], "Required edge property 'object' is missing": [ "A:123->X:1" ], "Required edge property 'predicate' is missing": [ "A:123->X:1" ], "Required edge property 'subject' is missing": [ "A:123->X:1", "B:456->Y:2" ] } }, "WARNING": { "DUPLICATE_NODE": { "Node 'id' duplicated in input data": [ "MONDO:0010011", "REACT:R-HSA-5635838" ] } } }

```

This system reduces the significant redundancies of earlier line-oriented KGX logging text output files, in that graph entities with the same class of error are simply aggregated in lists of names/identifiers at the leaf level of the JSON structure.

The top level JSON tags originate from the MessageLevel class and the second level tags from the ErrorType class in the error_detection module, while the third level messages are hard coded as log_error method messages in the code.

It is likely that additional error conditions within KGX can be efficiently captured and reported in the future using this general framework.

Installation

Installing from PyPI

KGX is available on PyPI and can be installed using pip as follows,

bash pip install kgx

To install a particular version of KGX, be sure to specify the version number,

bash pip install kgx==0.5.0

Installing from GitHub

Clone the GitHub repository and then install,

bash git clone https://github.com/biolink/kgx cd kgx poetry install

Setting up a testing environment for Neo4j

This release of KGX supports graph source and sink transactions with the 4.3 release of Neo4j.

KGX has a suite of tests that rely on Docker containers to run Neo4j specific tests.

To set up the required containers, first install Docker on your local machine.

Once Docker is up and running, run the following commands:

bash docker run -d --rm --name kgx-neo4j-integration-test -p 7474:7474 -p 7687:7687 --env NEO4J_AUTH=neo4j/test neo4j:4.3

bash docker run -d --rm --name kgx-neo4j-unit-test -p 8484:7474 -p 8888:7687 --env NEO4J_AUTH=neo4j/test neo4j:4.3

Note: Setting up the Neo4j container is optional. If there is no container set up then the tests that rely on them are skipped.

Owner

Name: biolink
Login: biolink
Kind: organization

Repositories: 10
Profile: https://github.com/biolink

GitHub Events

Total

Create event: 12
Release event: 3
Issues event: 4
Watch event: 12
Delete event: 5
Issue comment event: 6
Push event: 60
Pull request review event: 4
Pull request review comment event: 2
Pull request event: 25
Fork event: 4

Last Year

Create event: 12
Release event: 3
Issues event: 4
Watch event: 12
Delete event: 5
Issue comment event: 6
Push event: 60
Pull request review event: 4
Pull request review comment event: 2
Pull request event: 25
Fork event: 4

Committers

Last synced: 9 months ago

All Time

Total Commits: 1,496
Total Committers: 26
Avg Commits per committer: 57.538
Development Distribution Score (DDS): 0.643

Past Year

Commits: 57
Committers: 7
Avg Commits per committer: 8.143
Development Distribution Score (DDS): 0.316

Top Committers

Name	Email	Commits
Deepak Unni	d**3@g**m	534
Sierra Taylor Moxon	s**r@g**m	381
Richard Bruskiewich	r**d@s**m	231
Lance Hannestad	l**t@s**a	195
cmungall	c**m@b**g	31
Richard Bruskiewich	r**h@d**m	26
Kevin Schaper	k**r@g**m	20
Kenneth	k**z@g**m	17
caufieldjh	j**d@g**m	16
dependabot[bot]	4****]	13
Harshad Hegde	h**b@g**m	9
Phil Owen	p**n@o**m	4
iQuxLE	c**5@g**m	4
gregr	g**i@g**m	2
hsolbrig	s**g@e**t	2
Andrew Su	a**u@s**u	1
Damien Goutte-Gattat	d**t@i**g	1
Daniel Himmelstein	d**n@g**m	1
Eric Douglass	e**s@l**v	1
Evan Morris	e**s@g**m	1
HyeongSik Kim	y**6@g**m	1
Kent Shefchek	k**k@g**m	1
Nico Matentzoglu	n**u@g**m	1
cbizon	b**n@r**g	1
Yaphetkg	y**g@r**g	1
glass-ships	g**s@o**m	1

Committer Domains (Top 20 + Academic)

renci.org: 2 lbl.gov: 1 incenp.org: 1 scripps.edu: 1 earthlink.net: 1 delphinai.com: 1 berkeleybop.org: 1 sfu.ca: 1 starinformatics.com: 1

Issues and Pull Requests

Last synced: 6 months ago

All Time

Total issues: 61
Total pull requests: 98
Average time to close issues: over 1 year
Average time to close pull requests: about 2 months
Total issue authors: 13
Total pull request authors: 16
Average comments per issue: 2.36
Average comments per pull request: 0.49
Merged pull requests: 80
Bot issues: 0
Bot pull requests: 23

Past Year

Issues: 5
Pull requests: 26
Average time to close issues: 14 days
Average time to close pull requests: 4 days
Issue authors: 5
Pull request authors: 6
Average comments per issue: 0.4
Average comments per pull request: 0.23
Merged pull requests: 19
Bot issues: 0
Bot pull requests: 10

View more stats

Top Authors

Issue Authors

cmungall (31)
lhannest (13)
deepakunni3 (3)
saramsey (3)
caufieldjh (2)
realmarcin (2)
shenfc (1)
DnlRKorn (1)
cthoyt (1)
chrisroederucdenver (1)
RichardBruskiewich (1)
ptgolden (1)
hrshdhgd (1)

Pull Request Authors

dependabot[bot] (23)
deepakunni3 (19)
lhannest (17)
sierra-moxon (8)
hrshdhgd (6)
cmungall (5)
iQuxLE (4)
kevinschaper (4)
gregr (3)
andrewsu (2)
matentzn (2)
yy20716 (1)
caufieldjh (1)
putmantime (1)
hsolbrig (1)

Top Labels

Issue Labels

enhancement (6) discussion (2) bug (2) question (1) help wanted (1)

Pull Request Labels

dependencies (23) python (5) github_actions (1)

Packages

Total packages: 2
Total downloads:
- pypi 3,276 last-month

Total dependent packages: 3
(may contain duplicates)
Total dependent repositories: 25
(may contain duplicates)
Total versions: 81
Total maintainers: 3

pypi.org: kgx

A Python library and set of command line utilities for exchanging Knowledge Graphs (KGs) that conform to or are aligned to the Biolink Model.

Documentation: https://kgx.readthedocs.io/
License: BSD
Latest release: 2.5.1
published 10 months ago

Versions: 55
Dependent Packages: 3
Dependent Repositories: 25
Downloads: 3,276 Last month

Rankings

Dependent packages count: 2.4%

Dependent repos count: 2.9%

Average: 4.5%

Downloads: 8.3%

Maintainers (3)

cmungall smoxon deepakunni3

Last synced: 6 months ago

proxy.golang.org: github.com/biolink/kgx

Documentation: https://pkg.go.dev/github.com/biolink/kgx#section-documentation
License: bsd-3-clause
Latest release: v2.5.1+incompatible
published 10 months ago

Versions: 26
Dependent Packages: 0
Dependent Repositories: 0

Rankings

Dependent packages count: 5.4%

Average: 5.6%

Dependent repos count: 5.8%

Last synced: 6 months ago

Dependencies

.github/workflows/run_tests.yml actions

SonarSource/sonarcloud-github-action master composite
actions/checkout v2 composite
actions/setup-python v2 composite
jakejarvis/wait-action master composite

Dockerfile docker

python 3.9 build

.github/workflows/pypi-release.yml actions

actions/checkout v2 composite
actions/setup-python v2.2.2 composite
pypa/gh-action-pypi-publish v1.2.2 composite

poetry.lock pypi

124 dependencies

pyproject.toml pypi

sphinx-click * develop
sphinx-rtd-theme * develop
Click *
SPARQLWrapper ^1.8.2
Sphinx *
bmt ^1.1.0
cachetools ^5.0.0
deprecation ^2.1.0
docker ^6.0.0
docutils ^0.18.1
ijson ^3.1.3
inflection ^0.5.1
jsonlines ^3.1.0
jsonstreams ^0.6.0
linkml ^1.5.0
linkml-runtime ^1.5.0
mypy *
neo4j ^4.4.10
networkx *
ordered-set ^4.0.2
pandas >1.0.3
prefixcommons ^0.1.4
prologterms ^0.0.6
pytest *
python ^3.9
python-dateutil ^2.8.1
pyyaml *
rdflib *
recommonmark *
shexjsg *
stringcase ^1.2.0
terminaltables ^3.1.0
tox ^3.0
validators ^0.20.0

https://github.com/biolink/kgx

Science Score: 36.0%

Keywords

Keywords from Contributors

Repository

Basic Info

Statistics

Topics

Metadata Files

README.md

Knowledge Graph Exchange

Example usage

Error Detection and Reporting

Validator assumes the currently set Biolink Release

Both the Validator and the Transformer can independently capture errors

The Validator, from the overall semantics of the graph...

Here, we just report severe Errors from the Validator (no Warnings)

The Transformer, from the syntax of the input files...

Here, we catch all Errors and Warnings (by not providing a filter)

Installation

Installing from PyPI

Installing from GitHub

Setting up a testing environment for Neo4j

Owner

GitHub Events

Total

Last Year

Committers

All Time

Past Year

Top Committers

Committer Domains (Top 20 + Academic)

Issues and Pull Requests

All Time

Past Year

Top Authors

Issue Authors

Pull Request Authors

Top Labels

Issue Labels

Pull Request Labels

Packages

pypi.org: kgx

Rankings

Maintainers (3)

proxy.golang.org: github.com/biolink/kgx

Rankings

Dependencies