Data

Tools

Indexes

Applications

Experiments

Open Source Science

ukaea-multiparser

Multiple File Parser for monitoring and processing updates to a given set of output files.

https://github.com/ukaea/multiparser

Science Score: 44.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ✓
    CITATION.cff file
    Found CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (15.2%) to scientific vocabulary

Keywords

file-parser parallel-parser python
Last synced: 6 months ago · JSON representation ·

Repository

Multiple File Parser for monitoring and processing updates to a given set of output files.

Basic Info
Statistics
  • Stars: 1
  • Watchers: 3
  • Forks: 0
  • Open Issues: 1
  • Releases: 6
Topics
file-parser parallel-parser python
Created almost 2 years ago · Last pushed 9 months ago
Metadata Files
Readme Changelog Contributing License Code of conduct Citation

README.md

Multiparser

PyPI Latest Release PyPI - Python Version multiparser Code style: ruff

Multiparser is module for performing functionality across a set of output files. Given a set of files, and optionally a list of output parameter names or regex filters the module keeps track of changes by monitoring the "last modified" timestamp of each file and executing the assigned callback.

For example, in the case where a set of model outputs were written to a set of files the user is able to specify which files are of importance, the variables to be noted within these files, and a function to execute whenever a change is made to one of these files.

Installation

The module is available in PyPi:

sh pip install ukaea-multiparser

To install optional extras arrow and fortran list them during the install, e.g.:

sh pip install ukaea-multiparser[fortran,arrow]

Example

python with FileMonitor( per_thread_callback=callback_function, lock_callback=True, interval=10.0, flatten_data=True ) as monitor: monitor.track( path_glob_exprs=["file_of_interest.toml", "out_dir/*.other"], tracked_values=[ "list", "of", "interesting", "values", re.compile(r"^list"), re.compile(r"of\s"), re.compile(r"regular"), re.compile(r"Expressions") ], static=True ) monitor.run() ... monitor.terminate()

Documentation

For information on use and functionality please see the documentation.

Licensing

Multiparser is provided under the MIT license allowing reuse within both open source and proprietary software.

Contributing

For contributions and development towards improving Multiparser please see the included CONTRIBUTING.md file.

Copyright (c) 2024 UK Atomic Energy Authority

Owner

  • Name: UK Atomic Energy Authority
  • Login: ukaea
  • Kind: organization

Citation (CITATION.cff) 

cff-version: 1.2.0
title: "Multiparser: A parallel multiple file parse trigger system"
message: >-
  If you use this software, please cite it using the
  metadata from this file.
type: software
authors:
  - given-names: Kristian
    family-names: Zarebski
    orcid: 'https://orcid.org/0000-0002-6773-1049'
    affiliation: United Kingdom Atomic Energy Authority
  - given-names: Aby
    family-names: Abraham
    affiliation: United Kingdom Atomic Energy Authority
  - given-names: Andrew
    family-names: Lahiff
    affiliation: United Kingdom Atomic Energy Authority
  - given-names: Matthew
    family-names: Field
    orcid: 'https://orcid.org/0009-0004-1390-0697'
    affiliation: United Kingdom Atomic Energy Authority
  - given-names: James
    family-names: Panayis
    affiliation: United Kingdom Atomic Energy Authority
abstract: >-
  A parallel parsing module for Python allowing dynamic
  extraction of data from files.
keywords:
  - parsing
license: MIT
repository-code: "https://github.com/ukaea/multiparser"
url: "https://ukaea.github.io/multiparser/"
commit: ee53dca6791daeb07f082bf30f5254427e8d3e31
version: 1.0.4
date-released: '2024-09-24'

GitHub Events

Total
  • Create event: 51
  • Issues event: 1
  • Release event: 1
  • Watch event: 1
  • Delete event: 47
  • Issue comment event: 10
  • Push event: 55
  • Pull request event: 97
Last Year
  • Create event: 51
  • Issues event: 1
  • Release event: 1
  • Watch event: 1
  • Delete event: 47
  • Issue comment event: 10
  • Push event: 55
  • Pull request event: 97

Issues and Pull Requests

Last synced: 6 months ago

All Time
  • Total issues: 0
  • Total pull requests: 19
  • Average time to close issues: N/A
  • Average time to close pull requests: 8 days
  • Total issue authors: 0
  • Total pull request authors: 2
  • Average comments per issue: 0
  • Average comments per pull request: 0.21
  • Merged pull requests: 14
  • Bot issues: 0
  • Bot pull requests: 18
Past Year
  • Issues: 0
  • Pull requests: 19
  • Average time to close issues: N/A
  • Average time to close pull requests: 8 days
  • Issue authors: 0
  • Pull request authors: 2
  • Average comments per issue: 0
  • Average comments per pull request: 0.21
  • Merged pull requests: 14
  • Bot issues: 0
  • Bot pull requests: 18
View more stats
Top Authors
Issue Authors
  • dependabot[bot] (3)
  • wk9874 (1)
Pull Request Authors
  • dependabot[bot] (175)
  • kzscisoft (4)
Top Labels
Issue Labels
dependencies (3)
Pull Request Labels
dependencies (175) bug (2) enhancement (1) python (1)

Packages

  • Total packages: 1
  • Total downloads:
    • pypi 244 last-month
  • Total dependent packages: 0
  • Total dependent repositories: 0
  • Total versions: 6
  • Total maintainers: 1
pypi.org: ukaea-multiparser

Multiple File Parser for monitoring and processing updates to a given set of output files.

  • Versions: 6
  • Dependent Packages: 0
  • Dependent Repositories: 0
  • Downloads: 244 Last month
Rankings
Dependent packages count: 9.7%
Average: 36.7%
Dependent repos count: 63.8%
Maintainers (1)
artemis-beta
Last synced: 6 months ago

Dependencies

.github/workflows/codeql.yaml actions
  • actions/checkout v4 composite
  • github/codeql-action/analyze v3 composite
  • github/codeql-action/autobuild v3 composite
  • github/codeql-action/init v3 composite
.github/workflows/linting.yaml actions
  • actions/checkout v3 composite
  • jpetrucciani/ruff-check 0.1.0 composite
.github/workflows/python-publish.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v3 composite
  • pypa/gh-action-pypi-publish release/v1 composite
.github/workflows/test_run_multiparser.yaml actions
  • actions/checkout v3 composite
  • actions/setup-python v4 composite
poetry.lock pypi
  • 115 dependencies
pyproject.toml pypi
  • interrogate ^1.5.0 develop
  • ipykernel ^6.29.0 develop
  • mypy ^1.9.0 develop
  • pandas-stubs ^2.2.1.240316 develop
  • ruff ^0.3.3 develop
  • types-pyyaml ^6.0.12.20240311 develop
  • types-toml ^0.10.8.20240310 develop
  • mike ^2.0.0 docs
  • mkdocs ^1.5.3 docs
  • mkdocs-git-revision-date-localized-plugin ^1.2.2 docs
  • mkdocs-material ^9.5.5 docs
  • mkdocs-mermaid2-plugin ^1.1.1 docs
  • mkdocstrings ^0.24.1 docs
  • mktestdocs ^0.2.1 docs
  • pymdown-extensions ^10.7 docs
  • bandit ^1.7.5 quality
  • f90nml ^1.4.3
  • flatdict ^4.0.1
  • loguru ^0.7.2
  • pandas ^2.1.1
  • pyarrow >=14,<16
  • python ^3.10,<3.13
  • pyyaml ^6.0.1
  • toml ^0.10.2
  • pandas ^2.1.1 testing
  • pytest ^8.1.1 testing
  • pytest-cov ^4.1.0 testing
  • pytest-mock ^3.11.1 testing
  • pytest-sugar ^1.0.0 testing
  • xeger ^0.4.0 testing