Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/biolink/resource-ingest-guide-schema

A LinkML schema for describing the scope, rationale, and modeling approach for ingesting content from a single source.

https://github.com/biolink/resource-ingest-guide-schema

Science Score: 44.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ✓
    CITATION.cff file
    Found CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (14.5%) to scientific vocabulary

Keywords

etl linkml model schema
Last synced: 4 months ago · JSON representation ·

Repository

A LinkML schema for describing the scope, rationale, and modeling approach for ingesting content from a single source.

Basic Info
Statistics
  • Stars: 1
  • Watchers: 1
  • Forks: 1
  • Open Issues: 3
  • Releases: 1
Topics
etl linkml model schema
Created 5 months ago · Last pushed 4 months ago
Metadata Files
Readme Contributing License Code of conduct Citation

README.md

Resource Ingest Guide Schema

A LinkML schema for describing Reference Ingest Guides (RIGs) - structured documents that capture the scope, rationale, and modeling approach for ingesting content from external sources into Biolink Model-compliant data repositories.

Overview

This repository provides:

  • LinkML Schema: Formal specification for Reference Ingest Guides in src/resource_ingest_guide_schema/schema/
  • Documentation Generator: Automated conversion of RIG YAML files to human-readable markdown
  • Validation Tools: Schema validation for RIG files using LinkML
  • Template System: Standardized templates and creation tools for new RIGs
  • Example RIGs: Real-world examples from CTD, DISEASES, and Clinical Trials KP

What are Reference Ingest Guides (RIGs)?

RIGs are structured documents that describe:

  • Source Information: Details about data sources (access, formats, licensing)
  • Ingest Information: What content is included/excluded and filtering rationale
  • Target Information: How data is modeled in the output knowledge graph
  • Provenance Information: Contributors and related artifacts

RIGs help ensure reproducible, well-documented data ingestion processes for biomedical knowledge graphs.

Website

https://biolink.github.io/resource-ingest-guide-schema

Repository Structure

├── src/ │ ├── resource_ingest_guide_schema/ │ │ └── schema/ # LinkML schema definition │ ├── docs/ │ │ ├── files/ # Static documentation files │ │ ├── rigs/ # Example RIG YAML files │ │ └── doc-templates/ # Jinja2 templates for docs │ └── scripts/ # Python utilities for RIG processing ├── docs/ # Generated documentation ├── tests/ # Test suite └── project/ # Generated LinkML artifacts

Developer Documentation

Prerequisites

This project uses uv for dependency management. Install it with:

```bash

On macOS and Linux

curl -LsSf https://astral.sh/uv/install.sh | sh

On Windows

powershell -c "irm https://astral.sh/uv/install.ps1 | iex"

Or with pip

pip install uv ```

Getting Started

Note that the following commands assume you are in the project root directory, and the equivalent just commands may be substituted for make (namely just test instead of make test)

  1. Install dependencies: bash uv sync --extra dev

  2. Run tests: bash make test

  3. Generate documentation: bash make gendoc

  4. Create a new RIG: bash make new-rig INFORES=infores:example NAME="Example Data Source"

Working with RIGs

Creating a New RIG

```bash

Create a new RIG from the template

make new-rig INFORES=infores:mydatasource NAME="My Data Source RIG"

This creates src/docs/rigs/mydatasource_rig.yaml

Edit the file to fill in your specific information

```

Validating RIGs

```bash

Validate all RIG files against the schema

make validate-rigs

Validate a specific RIG

uv run linkml-validate --schema src/resourceingestguideschema/schema/resourceingestguideschema.yaml src/docs/rigs/my_rig.yaml ```

Building Documentation

```bash

Generate all documentation including RIG index and markdown versions

make gendoc

Test documentation locally

make testdoc # Builds docs and starts local server ```

Development Workflow

1. Schema Development

The LinkML schema is defined in src/resource_ingest_guide_schema/schema/resource_ingest_guide_schema.yaml. After making changes:

```bash

Regenerate Python datamodel and other artifacts

make gen-project

Test the schema

make test-schema

Lint the schema

make lint ```

2. Script Development

Python utilities are in src/scripts/: - create_rig.py: Generate new RIG from template - rig_to_markdown.py: Convert RIG YAML to markdown - generate_rig_index.py: Create RIG index table

To test script changes: ```bash

Run scripts directly

uv run python src/scripts/createrig.py --help uv run python src/scripts/rigto_markdown.py --input-dir src/docs/rigs --output-dir docs ```

3. Documentation Development

Templates are in src/docs/doc-templates/ and static files in src/docs/files/:

```bash

Regenerate docs after template changes

make gendoc

View changes locally

make serve # or make testdoc ```

Available Commands

| Command | Description | |---------|-------------| | make help | Show all available commands | | make install | Install dependencies with uv | | make test | Run full test suite | | make test-schema | Test schema generation | | make test-python | Run Python tests | | make lint | Lint the LinkML schema | | make gen-project | Generate LinkML artifacts (Python, JSON Schema, etc.) | | make gendoc | Generate documentation including RIG processing | | make serve | Start local documentation server | | make testdoc | Build docs and start server | | make new-rig | Create new RIG (requires INFORES and NAME) | | make validate-rigs | Validate all RIG files | | make clean | Clean generated files | | make deploy | Deploy documentation |

Project Structure Details

Key Directories

  • src/resource_ingest_guide_schema/schema/: LinkML schema definition
  • src/docs/rigs/: Example RIG YAML files (CTD, DISEASES, Clinical Trials KP)
  • src/docs/files/: Static documentation files copied to output
  • src/docs/doc-templates/: Jinja2 templates for documentation generation
  • src/scripts/: Python utilities for RIG creation and processing
  • docs/: Generated documentation output (do not edit directly)
  • project/: Generated LinkML artifacts (Python models, JSON Schema, etc.)

Generated Artifacts

The make gen-project command generates: - Python datamodel: src/resource_ingest_guide_schema/datamodel/ - JSON Schema: project/jsonschema/ - OWL ontology: project/owl/ - GraphQL schema: project/graphql/ - SQL DDL: project/sqlschema/ - And more: See project/ directory

Contributing

  1. Fork the repository
  2. Create a feature branch
  3. Make changes following the existing patterns
  4. Ensure tests pass: make test
  5. Update documentation if needed: make gendoc
  6. Submit a pull request

Adding New RIG Examples

  1. Create YAML file in src/docs/rigs/
  2. Follow the schema structure (see existing examples)
  3. Validate: make validate-rigs
  4. Regenerate docs: make gendoc
  5. The RIG will automatically appear in the documentation index

Schema Changes

  1. Modify src/resource_ingest_guide_schema/schema/resource_ingest_guide_schema.yaml
  2. Regenerate artifacts: make gen-project
  3. Update any affected RIG files
  4. Test: make test
  5. Update documentation as needed

Owner

  • Name: biolink
  • Login: biolink
  • Kind: organization

Citation (CITATION.cff) 

cff-version: '1.1.0'
message: 'Please cite the following works when using this software.'
authors:
  - family-names: 'Unni'
    given-names: 'Deepak R.'
  - family-names: 'Moxon'
    given-names: 'Sierra A. T.'
  - family-names: 'Bada'
    given-names: 'Michael'
  - family-names: 'Brush'
    given-names: 'Matthew'
  - family-names: 'Bruskiewich'
    given-names: 'Richard'
  - family-names: 'Caufield'
    given-names: 'J. Harry'
  - family-names: 'Clemons'
    given-names: 'Paul A.'
  - family-names: 'Dancik'
    given-names: 'Vlado'
  - family-names: 'Dumontier'
    given-names: 'Michel'
  - family-names: 'Fecho'
    given-names: 'Karamarie'
  - family-names: 'Glusman'
    given-names: 'Gustavo'
  - family-names: 'Hadlock'
    given-names: 'Jennifer J.'
  - family-names: 'Harris'
    given-names: 'Nomi L.'
  - family-names: 'Joshi'
    given-names: 'Arpita'
  - family-names: 'Putman'
    given-names: 'Tim'
  - family-names: 'Qin'
    given-names: 'Guangrong'
  - family-names: 'Ramsey'
    given-names: 'Stephen A.'
  - family-names: 'Shefchek'
    given-names: 'Kent A.'
  - family-names: 'Solbrig'
    given-names: 'Harold'
  - family-names: 'Soman'
    given-names: 'Karthik'
  - family-names: 'Thessen'
    given-names: 'Anne E.'
  - family-names: 'Haendel'
    given-names: 'Melissa A.'
  - family-names: 'Bizon'
    given-names: 'Chris'
  - family-names: 'Mungall'
    given-names: 'Christopher J.'
  - family-names: 'Acevedo'
    given-names: 'Liliana'
  - family-names: 'Ahalt'
    given-names: 'Stanley C.'
  - family-names: 'Alden'
    given-names: 'John'
  - family-names: 'Alkanaq'
    given-names: 'Ahmed'
  - family-names: 'Amin'
    given-names: 'Nada'
  - family-names: 'Avila'
    given-names: 'Ricardo'
  - family-names: 'Balhoff'
    given-names: 'Jim'
  - family-names: 'Baranzini'
    given-names: 'Sergio E.'
  - family-names: 'Baumgartner'
    given-names: 'Andrew'
  - family-names: 'Baumgartner'
    given-names: 'William'
  - family-names: 'Belhu'
    given-names: 'Basazin'
  - family-names: 'Brandes'
    given-names: 'MacKenzie'
  - family-names: 'Brandon'
    given-names: 'Namdi'
  - family-names: 'Burtt'
    given-names: 'Noel'
  - family-names: 'Byrd'
    given-names: 'William'
  - family-names: 'Callaghan'
    given-names: 'Jackson'
  - family-names: 'Cano'
    given-names: 'Marco Alvarado'
  - family-names: 'Carrell'
    given-names: 'Steven'
  - family-names: 'Celebi'
    given-names: 'Remzi'
  - family-names: 'Champion'
    given-names: 'James'
  - family-names: 'Chen'
    given-names: 'Zhehuan'
  - family-names: 'Chen'
    given-names: 'Mei‐Jan'
  - family-names: 'Chung'
    given-names: 'Lawrence'
  - family-names: 'Cohen'
    given-names: 'Kevin'
  - family-names: 'Conlin'
    given-names: 'Tom'
  - family-names: 'Corkill'
    given-names: 'Dan'
  - family-names: 'Costanzo'
    given-names: 'Maria'
  - family-names: 'Cox'
    given-names: 'Steven'
  - family-names: 'Crouse'
    given-names: 'Andrew'
  - family-names: 'Crowder'
    given-names: 'Camerron'
  - family-names: 'Crumbley'
    given-names: 'Mary E.'
  - family-names: 'Dai'
    given-names: 'Cheng'
  - family-names: 'Dančík'
    given-names: 'Vlado'
  - family-names: 'De Miranda Azevedo'
    given-names: 'Ricardo'
  - family-names: 'Deutsch'
    given-names: 'Eric'
  - family-names: 'Dougherty'
    given-names: 'Jennifer'
  - family-names: 'Duby'
    given-names: 'Marc P.'
  - family-names: 'Duvvuri'
    given-names: 'Venkata'
  - family-names: 'Edwards'
    given-names: 'Stephen'
  - family-names: 'Emonet'
    given-names: 'Vincent'
  - family-names: 'Fehrmann'
    given-names: 'Nathaniel'
  - family-names: 'Flannick'
    given-names: 'Jason'
  - family-names: 'Foksinska'
    given-names: 'Aleksandra M.'
  - family-names: 'Gardner'
    given-names: 'Vicki'
  - family-names: 'Gatica'
    given-names: 'Edgar'
  - family-names: 'Glen'
    given-names: 'Amy'
  - family-names: 'Goel'
    given-names: 'Prateek'
  - family-names: 'Gormley'
    given-names: 'Joseph'
  - family-names: 'Greyber'
    given-names: 'Alon'
  - family-names: 'Haaland'
    given-names: 'Perry'
  - family-names: 'Hanspers'
    given-names: 'Kristina'
  - family-names: 'He'
    given-names: 'Kaiwen'
  - family-names: 'He'
    given-names: 'Kaiwen'
  - family-names: 'Henrickson'
    given-names: 'Jeff'
  - family-names: 'Hinderer'
    given-names: 'Eugene W.'
  - family-names: 'Hoatlin'
    given-names: 'Maureen'
  - family-names: 'Hoffman'
    given-names: 'Andrew'
  - family-names: 'Huang'
    given-names: 'Sui'
  - family-names: 'Huang'
    given-names: 'Conrad'
  - family-names: 'Hubal'
    given-names: 'Robert'
  - family-names: 'Huellas‐Bruskiewicz'
    given-names: 'Kenneth'
  - family-names: 'Huls'
    given-names: 'Forest B.'
  - family-names: 'Hunter'
    given-names: 'Lawrence'
  - family-names: 'Hyde'
    given-names: 'Greg'
  - family-names: 'Issabekova'
    given-names: 'Tursynay'
  - family-names: 'Jarrell'
    given-names: 'Matthew'
  - family-names: 'Jenkins'
    given-names: 'Lindsay'
  - family-names: 'Johs'
    given-names: 'Adam'
  - family-names: 'Kang'
    given-names: 'Jimin'
  - family-names: 'Kanwar'
    given-names: 'Richa'
  - family-names: 'Kebede'
    given-names: 'Yaphet'
  - family-names: 'Kim'
    given-names: 'Keum Joo'
  - family-names: 'Kluge'
    given-names: 'Alexandria'
  - family-names: 'Knowles'
    given-names: 'Michael'
  - family-names: 'Koesterer'
    given-names: 'Ryan'
  - family-names: 'Korn'
    given-names: 'Daniel'
  - family-names: 'Koslicki'
    given-names: 'David'
  - family-names: 'Krishnamurthy'
    given-names: 'Ashok'
  - family-names: 'Kvarfordt'
    given-names: 'Lindsey'
  - family-names: 'Lee'
    given-names: 'Jay'
  - family-names: 'Leigh'
    given-names: 'Margaret'
  - family-names: 'Lin'
    given-names: 'Jason'
  - family-names: 'Liu'
    given-names: 'Zheng'
  - family-names: 'Liu'
    given-names: 'Shaopeng'
  - family-names: 'Ma'
    given-names: 'Chunyu'
  - family-names: 'Magis'
    given-names: 'Andrew'
  - family-names: 'Mamidi'
    given-names: 'Tarun'
  - family-names: 'Mandal'
    given-names: 'Meisha'
  - family-names: 'Mantilla'
    given-names: 'Michelle'
  - family-names: 'Massung'
    given-names: 'Jeffrey'
  - family-names: 'Mauldin'
    given-names: 'Denise'
  - family-names: 'McClelland'
    given-names: 'Jason'
  - family-names: 'McMurry'
    given-names: 'Julie'
  - family-names: 'Mease'
    given-names: 'Philip'
  - family-names: 'Mendoza'
    given-names: 'Luis'
  - family-names: 'Mersmann'
    given-names: 'Marian'
  - family-names: 'Mesbah'
    given-names: 'Abrar'
  - family-names: 'Might'
    given-names: 'Matthew'
  - family-names: 'Morton'
    given-names: 'Kenny'
  - family-names: 'Muller'
    given-names: 'Sandrine'
  - family-names: 'Muluka'
    given-names: 'Arun Teja'
  - family-names: 'Osborne'
    given-names: 'John'
  - family-names: 'Owen'
    given-names: 'Phil'
  - family-names: 'Patton'
    given-names: 'Michael'
  - family-names: 'Peden'
    given-names: 'David B.'
  - family-names: 'Peene'
    given-names: 'R. Carter'
  - family-names: 'Persaud'
    given-names: 'Bria'
  - family-names: 'Pfaff'
    given-names: 'Emily'
  - family-names: 'Pico'
    given-names: 'Alexander'
  - family-names: 'Pollard'
    given-names: 'Elizabeth'
  - family-names: 'Price'
    given-names: 'Guthrie'
  - family-names: 'Raj'
    given-names: 'Shruti'
  - family-names: 'Reilly'
    given-names: 'Jason'
  - family-names: 'Riutta'
    given-names: 'Anders'
  - family-names: 'Roach'
    given-names: 'Jared'
  - family-names: 'Roper'
    given-names: 'Ryan T.'
  - family-names: 'Rosenblatt'
    given-names: 'Greg'
  - family-names: 'Rubin'
    given-names: 'Irit'
  - family-names: 'Rucka'
    given-names: 'Sienna'
  - family-names: 'Rudavsky‐Brody'
    given-names: 'Nathaniel'
  - family-names: 'Sakaguchi'
    given-names: 'Rayn'
  - family-names: 'Santos'
    given-names: 'Eugene'
  - family-names: 'Schaper'
    given-names: 'Kevin'
  - family-names: 'Schmitt'
    given-names: 'Charles P.'
  - family-names: 'Schurman'
    given-names: 'Shepherd'
  - family-names: 'Scott'
    given-names: 'Erik'
  - family-names: 'Seitanakis'
    given-names: 'Sarah'
  - family-names: 'Sharma'
    given-names: 'Priya'
  - family-names: 'Shmulevich'
    given-names: 'Ilya'
  - family-names: 'Shrestha'
    given-names: 'Manil'
  - family-names: 'Shrivastava'
    given-names: 'Shalki'
  - family-names: 'Sinha'
    given-names: 'Meghamala'
  - family-names: 'Smith'
    given-names: 'Brett'
  - family-names: 'Southall'
    given-names: 'Noel'
  - family-names: 'Southern'
    given-names: 'Nicholas'
  - family-names: 'Stillwell'
    given-names: 'Lisa'
  - family-names: 'Strasser'
    given-names: 'Michael " Michi"'
  - family-names: 'Su'
    given-names: 'Andrew I.'
  - family-names: 'Ta'
    given-names: 'Casey'
  - family-names: 'Thessen'
    given-names: 'Anne E.'
  - family-names: 'Tinglin'
    given-names: 'Jillian'
  - family-names: 'Tonstad'
    given-names: 'Lucas'
  - family-names: 'Tran‐Nguyen'
    given-names: 'Thi'
  - family-names: 'Tropsha'
    given-names: 'Alexander'
  - family-names: 'Vaidya'
    given-names: 'Gaurav'
  - family-names: 'Veenhuis'
    given-names: 'Luke'
  - family-names: 'Viola'
    given-names: 'Adam'
  - family-names: 'Grotthuss'
    given-names: 'Marcin'
  - family-names: 'Wang'
    given-names: 'Max'
  - family-names: 'Wang'
    given-names: 'Patrick'
  - family-names: 'Watkins'
    given-names: 'Paul B.'
  - family-names: 'Weber'
    given-names: 'Rosina'
  - family-names: 'Wei'
    given-names: 'Qi'
  - family-names: 'Weng'
    given-names: 'Chunhua'
  - family-names: 'Whitlock'
    given-names: 'Jordan'
  - family-names: 'Williams'
    given-names: 'Mark D.'
  - family-names: 'Williams'
    given-names: 'Andrew'
  - family-names: 'Womack'
    given-names: 'Finn'
  - family-names: 'Wood'
    given-names: 'Erica'
  - family-names: 'Wu'
    given-names: 'Chunlei'
  - family-names: 'Xin'
    given-names: 'Jiwen Kevin'
  - family-names: 'Xu'
    given-names: 'Hao'
  - family-names: 'Xu'
    given-names: 'Colleen'
  - family-names: 'Yakaboski'
    given-names: 'Chase'
  - family-names: 'Yao'
    given-names: 'Yao'
  - family-names: 'Yi'
    given-names: 'Hong'
  - family-names: 'Yilmaz'
    given-names: 'Arif'
  - family-names: 'Zheng'
    given-names: 'Marissa'
  - family-names: 'Zhou'
    given-names: 'Xinghua'
  - family-names: 'Zhou'
    given-names: 'Eric'
  - family-names: 'Zhu'
    given-names: 'Qian'
  - family-names: 'Zisk'
    given-names: 'Tom'
doi: '10.1111/cts.13302'
identifiers:
  - type: 'doi'
    value: '10.1111/cts.13302'
  - type: 'url'
    value: 'http://dx.doi.org/10.1111/cts.13302'
  - type: 'other'
    value: 'urn:issn:1752-8054'
title: 'Biolink Model: A universal schema for knowledge graphs in clinical, biomedical, and translational science'
url: 'http://dx.doi.org/10.1111/cts.13302'

GitHub Events

Total
  • Create event: 7
  • Release event: 1
  • Issues event: 3
  • Watch event: 1
  • Delete event: 1
  • Member event: 1
  • Issue comment event: 5
  • Push event: 46
  • Pull request review event: 4
  • Pull request event: 9
Last Year
  • Create event: 7
  • Release event: 1
  • Issues event: 3
  • Watch event: 1
  • Delete event: 1
  • Member event: 1
  • Issue comment event: 5
  • Push event: 46
  • Pull request review event: 4
  • Pull request event: 9

Issues and Pull Requests

Last synced: 4 months ago

All Time
  • Total issues: 3
  • Total pull requests: 7
  • Average time to close issues: N/A
  • Average time to close pull requests: 1 day
  • Total issue authors: 1
  • Total pull request authors: 3
  • Average comments per issue: 0.0
  • Average comments per pull request: 0.57
  • Merged pull requests: 4
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 3
  • Pull requests: 7
  • Average time to close issues: N/A
  • Average time to close pull requests: 1 day
  • Issue authors: 1
  • Pull request authors: 3
  • Average comments per issue: 0.0
  • Average comments per pull request: 0.57
  • Merged pull requests: 4
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • sierra-moxon (3)
Pull Request Authors
  • RichardBruskiewich (4)
  • mbrush (2)
  • colleenXu (1)
Top Labels
Issue Labels
Pull Request Labels

Dependencies

.github/workflows/deploy-docs.yaml actions
  • actions/checkout v4 composite
  • actions/setup-python v4 composite
.github/workflows/main.yaml actions
  • actions/checkout v3 composite
  • actions/setup-python v4 composite
.github/workflows/pypi-publish.yaml actions
  • actions/checkout v3 composite
  • actions/setup-python v4 composite
  • pypa/gh-action-pypi-publish v1.2.2 composite
.github/workflows/test_pages_build.yaml actions
  • actions/checkout v4 composite
  • actions/setup-python v4 composite
  • rossjrw/pr-preview-action v1 composite
pyproject.toml pypi
  • jupyter >=1.0.0 develop
  • linkml >=1.3.5 develop
  • mkdocs-material >=8.2.8 develop
  • mkdocs-mermaid2-plugin >=1.1.1 develop
  • mknotebooks >= 0.8.0 develop
  • schemasheets >=0.1.14 develop
  • linkml-runtime >=1.1.24
uv.lock pypi
  • 222 dependencies