related

Nested Object Models in Python with dictionary, YAML, and JSON transformation support

https://github.com/genomoncology/related

Science Score: 10.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • CITATION.cff file
  • codemeta.json file
  • .zenodo.json file
  • DOI references
  • Academic publication links
  • Committers with academic emails
    1 of 8 committers (12.5%) from academic institutions
  • Institutional organization owner
  • JOSS paper metadata
  • Scientific vocabulary similarity
    Low similarity (12.8%) to scientific vocabulary

Keywords

json object-relational-mapper python yaml
Last synced: 6 months ago · JSON representation

Repository

Nested Object Models in Python with dictionary, YAML, and JSON transformation support

Basic Info
  • Host: GitHub
  • Owner: genomoncology
  • License: mit
  • Language: Python
  • Default Branch: master
  • Size: 329 KB
Statistics
  • Stars: 204
  • Watchers: 14
  • Forks: 16
  • Open Issues: 22
  • Releases: 0
Topics
json object-relational-mapper python yaml
Created over 8 years ago · Last pushed about 3 years ago
Metadata Files
Readme Changelog License

README.md



Related is a Python library for creating nested object models that can be serialized to and de-serialized from nested python dictionaries. When paired with other libraries (e.g. PyYAML), Related object models can be used to convert to and from nested data formats (e.g. JSON, YAML).

Example use cases for related object models include:

  • Configuration file reading and writing
  • REST API message response generation and request processing
  • Object-Document Mapping for a document store (e.g. MongoDB, elasticsearch)
  • Data import parsing or export generation


flow-image


Requirements

  • Python (2.7, 3.5, 3.6)

Installation

Install using pip...

pip install related

First Example

```python import related

@related.immutable class Person(object): firstname = related.StringField() lastname = related.StringField()

@related.immutable class RoleModels(object): scientists = related.SetField(Person)

people = [Person(firstname="Grace", lastname="Hopper"), Person(firstname="Katherine", lastname="Johnson"), Person(firstname="Katherine", lastname="Johnson")]

print(related.to_yaml(RoleModels(scientists=people))) ```

Yields:

yaml scientists: - first_name: Grace last_name: Hopper - first_name: Katherine last_name: Johnson

Second Example

The below example is based off of this Docker Compose example. It shows how a YAML file can be loaded into an object model, tested, and then generated back into a string that matches the original YAML.

yaml version: '2' services: web: build: . ports: - 5000:5000 volumes: - .:/code redis: image: redis

Below is the related object model that represents the above configuration. Notice how the name-based mapping of services (i.e. web, redis) are represented by the model.

```python import related

@related.immutable class Service(object): name = related.StringField() image = related.StringField(required=False) build = related.StringField(required=False) ports = related.SequenceField(str, required=False) volumes = related.SequenceField(str, required=False) command = related.StringField(required=False)

@related.immutable class Compose(object): version = related.StringField(required=False, default=None) services = related.MappingField(Service, "name", required=False) ```

The above yaml can then be loaded by using one of the convenience method and then round-tripped back to yaml to check that the format has been maintained. The related module uses OrderedDict objects in order to maintain sort order by default.

```python from os.path import join, dirname

from model import Compose from related import toyaml, fromyaml, to_model

YMLFILE = join(dirname(file_), "docker-compose.yml")

def testcomposefromyml(): originalyaml = open(YMLFILE).read().strip() ymldict = fromyaml(originalyaml) compose = tomodel(Compose, ymldict)

assert compose.version == '2'
assert compose.services['web'].ports == ["5000:5000"]
assert compose.services['redis'].image == "redis"

generated_yaml = to_yaml(compose,
                         suppress_empty_values=True,
                         suppress_map_key_values=True).strip()

assert original_yaml == generated_yaml

```

More Examples

More examples can be found by reviewing the tests/ folder of this project. Below are links and descriptions of the tests provided so far.

| Example | description | | -------------- | ------------------------------------------------------------------ | | Example 00 | First example above that shows how SetFields work. | | Example 01 | Second example above that demonstrates YAML (de)serialization. | | Example 02 | Compose v3 with long-form ports and singledispatch to_dict | | Example 03 | A single class (Company) with a bunch of value fields. | | Example 04 | A multi-class object model with Enum class value field. | | Example 05 | Handling of renaming of attributes including Python keywords. | | Example 06 | Basic JSON (de)serialization with TimeField, DateTimeField and DecimalField. | | Example 07 | Function decorator that converts inputs to obj and outputs to dict | | Example 08 | Handle self-referencing and out-of-order references using strings. |

Documentation

Below is a quick version of documentation until more time can be dedicated.

Overview

The attrs library is the underlying engine for related. As explained in this article by Glyph, attrs cleanly and cleverly eliminates a lot of the boilerplate required when creating Python classes without using inheritance. Some core functionality provided by attrs:

  • Generated initializer method (__init__)
  • Generated comparison methods (__eq__, __ne__, __lt__, __le__, __gt__, __ge__ )
  • Human-readable representation method (__repr__)
  • Attribute converter and validator framework

The related project is an opinionated layer built on top of the attrs library that provides the following:

  • Value fields that handle both validation and conversion to and from basic data types like str, float, and bool.
  • Nested fields that support relationships such as Child, Sequences, Mappings, and Sets of objects.
  • to_dict function that converts nested object graphs to python dictionaries. Made customizable (without resorting to monkey-patching) by the singledispatch library.
  • to_model function that instantiated classes used by the de-serialization process going from python dictionaries to the related model.
  • Conversion helper functions (to_yaml, from_yaml, to_json, from_json) for easily going between related models and data formats.
  • @mutable and @immutable for decorating classes as related models without the need for inheritance increasing maintainability and flexibility.

Class Decorators

| decorator | description | | -------------- | ---------------------------------------------------------------- | | @mutable | Activate a related class that instantiates changeable objects. | | @immutable | Activate a related class that instantiates unchangeable objects. |

See the decorators.py file to view the source code until proper documentation is generated.

Field Types

| field type | description | | -------------- | ---------------------------------------------------------------- | | BooleanField | bool value field. | | ChildField | Child object of a specified type cls. | | DateField | date field formatted using formatter. | | DateTimeField | datetime field formatted using formatter. | | TimeField | time field formatted using formatter. | | FloatField | float value field. | | IntegerField | int value field. | | MappingField(cls,key) | Dictionary of objects of type cls index by key field values. | | RegexField(regex) | str value field that is validated by re.match(regex). | | SequenceField(cls) | List of objects all of specified type cls. | | SetField | Set of objects all of a specified type cls. | | StringField | str value field. | | URLField | ParseResult object. | | UUIDField | UUID object, will create uuid4 by default if not specified. |

Adding your own field types is fairly straightforward due to the power of the underlying attrs project. See the fields.py file to see how the above are constructed.

Functions

| function | description | | ------------------- | ----------------------------------------------------- | | fromjson(s,cls) | Convert a JSON string or stream into specified class. | | fromyaml(s,cls) | Convert a YAML string or stream into specified class. | | isrelated(obj) | Returns True if object is @mutable or @immutable. | | todict(obj) | Singledispatch function for converting to a dict. | | tojson(obj) | Convert object to a (pretty) JSON string via todict. | | tomodel(cls,value) | Convert a value to a cls instance. | | toyaml(obj) | Convert object to a YAML string via to_dict. |

See the functions.py file to view the source code until proper documentation is generated.

Credits/Prior Art

The related project has been heavily influenced by the following projects that might be worth looking at if related doesn't meet your needs.

  • attrs - The engine that powers related functionality.
  • Django ORM - Object-relational mapping for Django that inspired related's design.
  • cattrs - Alternative take for handling nested-objects using attrs.
  • addict and box - Python dictionary wrappers that do not require a model.
  • Jackson - Java-based technology for serializing and de-serializing objects.

License

The MIT License (MIT) Copyright (c) 2017 Ian Maurer, Genomoncology LLC

Owner

  • Name: GenomOncology, LLC
  • Login: genomoncology
  • Kind: organization
  • Email: justin@genomoncology.com
  • Location: Cleveland, OH

GitHub Events

Total
  • Watch event: 5
  • Fork event: 1
Last Year
  • Watch event: 5
  • Fork event: 1

Committers

Last synced: 9 months ago

All Time
  • Total Commits: 47
  • Total Committers: 8
  • Avg Commits per committer: 5.875
  • Development Distribution Score (DDS): 0.213
Past Year
  • Commits: 0
  • Committers: 0
  • Avg Commits per committer: 0.0
  • Development Distribution Score (DDS): 0.0
Top Committers
Name Email Commits
Ian Maurer i****n@g****m 37
edward e****b@p****u 3
Gabriel Davidian g****v@g****m 2
Nik Khlestov k****a@g****m 1
Christopher Goes g****s@g****m 1
edward e****d@g****m 1
Justin Yeakley j****n@g****m 1
Jake Peacock j****e@g****m 1
Committer Domains (Top 20 + Academic)

Issues and Pull Requests

Last synced: 6 months ago

All Time
  • Total issues: 35
  • Total pull requests: 25
  • Average time to close issues: 4 months
  • Average time to close pull requests: 3 months
  • Total issue authors: 24
  • Total pull request authors: 13
  • Average comments per issue: 2.11
  • Average comments per pull request: 0.8
  • Merged pull requests: 11
  • Bot issues: 0
  • Bot pull requests: 6
Past Year
  • Issues: 0
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
Top Authors
Issue Authors
  • imaurer (6)
  • Avsecz (3)
  • ChristianSauer (2)
  • M0r13n (2)
  • mmmeri (2)
  • GeorgeLubaretsi (2)
  • ambivalentno (1)
  • syonekura (1)
  • ifoukarakis (1)
  • nicoulaj (1)
  • romcab (1)
  • jogo (1)
  • alexxv (1)
  • kapi2289 (1)
  • Roger (1)
Pull Request Authors
  • dependabot[bot] (6)
  • imaurer (5)
  • ewennerb (2)
  • M0r13n (2)
  • GeorgeLubaretsi (2)
  • jamespeacock (1)
  • danielchamberlain (1)
  • GhostofGoes (1)
  • Code0x58 (1)
  • GabrielDav (1)
  • ambivalentno (1)
  • dequis (1)
  • RodrigoDeRosa-zz (1)
Top Labels
Issue Labels
enhancement (4) help wanted (1) question (1) bug (1)
Pull Request Labels
dependencies (6)

Packages

  • Total packages: 2
  • Total downloads:
    • pypi 28,051 last-month
  • Total docker downloads: 757,189,232
  • Total dependent packages: 5
    (may contain duplicates)
  • Total dependent repositories: 27
    (may contain duplicates)
  • Total versions: 20
  • Total maintainers: 2
pypi.org: related

Related: Straightforward nested object models in Python

  • Versions: 18
  • Dependent Packages: 5
  • Dependent Repositories: 27
  • Downloads: 28,051 Last month
  • Docker Downloads: 757,189,232
Rankings
Docker downloads count: 0.2%
Dependent packages count: 1.9%
Downloads: 2.0%
Dependent repos count: 2.8%
Average: 3.6%
Stargazers count: 4.9%
Forks count: 9.6%
Maintainers (2)
Last synced: 6 months ago
conda-forge.org: related

Related is a Python library for creating nested object models that can be serialized to and de-serialized from nested python dictionaries. When paired with other libraries (e.g. PyYAML), Related object models can be used to convert to and from nested data formats (e.g. JSON, YAML).

  • Versions: 2
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Stargazers count: 24.6%
Dependent repos count: 34.0%
Average: 36.6%
Forks count: 36.7%
Dependent packages count: 51.2%
Last synced: 6 months ago

Dependencies

Pipfile pypi
  • black ==18.6b4 develop
  • coverage * develop
  • pipenv * develop
  • pytest == 3.10.1 develop
  • pytest-cov * develop
  • pytest-flake8 * develop
  • pytest-mccabe * develop
  • pytest-pep8 * develop
  • pytest-pythonpath * develop
  • pytest-sugar * develop
  • tox * develop
  • white * develop
  • attrs *
  • future *
  • python-dateutil *
  • pyyaml *
  • singledispatch *
Pipfile.lock pypi
  • apipkg ==1.5 develop
  • appdirs ==1.4.3 develop
  • atomicwrites ==1.3.0 develop
  • attrs ==19.3.0 develop
  • black ==18.6b4 develop
  • certifi ==2019.9.11 develop
  • click ==7.0 develop
  • coverage ==4.5.4 develop
  • entrypoints ==0.3 develop
  • execnet ==1.7.1 develop
  • filelock ==3.0.12 develop
  • flake8 ==3.7.9 develop
  • importlib-metadata ==0.23 develop
  • mccabe ==0.6.1 develop
  • more-itertools ==7.2.0 develop
  • packaging ==19.2 develop
  • pep8 ==1.7.1 develop
  • pipenv ==2018.11.26 develop
  • pluggy ==0.13.0 develop
  • py ==1.8.0 develop
  • pycodestyle ==2.5.0 develop
  • pyflakes ==2.1.1 develop
  • pyparsing ==2.4.5 develop
  • pytest ==3.10.1 develop
  • pytest-cache ==1.0 develop
  • pytest-cov ==2.8.1 develop
  • pytest-flake8 ==1.0.4 develop
  • pytest-mccabe ==1.0 develop
  • pytest-pep8 ==1.0.6 develop
  • pytest-pythonpath ==0.7.3 develop
  • pytest-sugar ==0.9.2 develop
  • six ==1.13.0 develop
  • termcolor ==1.1.0 develop
  • toml ==0.10.0 develop
  • tox ==3.14.1 develop
  • virtualenv ==16.7.7 develop
  • virtualenv-clone ==0.5.3 develop
  • white ==0.1.2 develop
  • zipp ==0.6.0 develop
  • attrs ==19.3.0
  • future ==0.18.2
  • python-dateutil ==2.8.1
  • pyyaml ==5.1.2
  • singledispatch ==3.4.0.3
  • six ==1.13.0
dev-requirements.txt pypi
  • coverage * development
  • pytest ==3.10.1 development
  • pytest-cov * development
  • pytest-flake8 * development
  • pytest-mccabe * development
  • pytest-pep8 * development
  • pytest-pythonpath * development
  • pytest-sugar * development
  • python-dateutil * development
  • tox * development
setup.py pypi
  • PyYAML *
  • attrs >=19.3.0
  • future *
  • python-dateutil *
  • singledispatch *
tests/ex01_compose_v2/docker-compose.yml docker
  • redis latest
tests/ex02_compose_v3_2/docker-compose.yml docker
  • redis latest