Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/britishgeologicalsurvey/etlhelper

ETL Helper is a Python ETL library to simplify data transfer into and out of databases.

https://github.com/britishgeologicalsurvey/etlhelper

Science Score: 23.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ✓
    Committers with academic emails
    4 of 11 committers (36.4%) from academic institutions
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (8.6%) to scientific vocabulary
Last synced: 10 months ago · JSON representation

Repository

ETL Helper is a Python ETL library to simplify data transfer into and out of databases.

Basic Info
Statistics
  • Stars: 106
  • Watchers: 8
  • Forks: 24
  • Open Issues: 15
  • Releases: 0
Created almost 7 years ago · Last pushed over 1 year ago
Metadata Files
Readme Contributing License Code of conduct

README.md

etlhelper

etlhelper is a Python ETL library to simplify data transfer into and out of databases.

ETL Helper makes it easy to run SQL queries via Python and return the results. It takes care of cursor management, importing drivers and formatting connection strings, while providing memory-efficient functions to read, write and transform data. This reduces the amount of boilerplate code required to manipulate data within relational databases with Python.

  • Documentation: https://britishgeologicalsurvey.github.io/etlhelper/
  • Source code: https://github.com/BritishGeologicalSurvey/etlhelper

Installation

ETL Helper is available on PyPI.

bash pip install etlhelper

Database driver packages are not included by default and should be specified in square brackets. Options are oracle (installs oracledb), mssql (installs pyodbc) and postgres (installs psycopg2). Multiple values can be separated by commas.

bash pip install etlhelper[oracle,postgres]

See the individual database driver configuration pages for any OS-level dependencies.

Development

ETL Helper was created by and is maintained by British Geological Survey Informatics.

See CONTRIBUTING.md for details on how to contribute.

Licence

ETL Helper is distributed under the LGPL v3.0 licence. Copyright: © BGS / UKRI 2019

Owner

  • Name: British Geological Survey
  • Login: BritishGeologicalSurvey
  • Kind: organization
  • Email: enquiries@bgs.ac.uk
  • Location: Keyworth, Nottinghamshire

The British Geological Survey is responsible for advising the UK government on geoscience and providing impartial advice to industry, academia and the public.

GitHub Events

Total
  • Create event: 2
  • Release event: 1
  • Issues event: 4
  • Watch event: 5
  • Delete event: 1
  • Push event: 5
  • Pull request review event: 1
  • Pull request event: 4
Last Year
  • Create event: 2
  • Release event: 1
  • Issues event: 4
  • Watch event: 5
  • Delete event: 1
  • Push event: 5
  • Pull request review event: 1
  • Pull request event: 4

Committers

Last synced: over 3 years ago

All Time
  • Total Commits: 267
  • Total Committers: 11
  • Avg Commits per committer: 24.273
  • Development Distribution Score (DDS): 0.318
Top Committers
Name Email Commits
John A Stevenson j****v@b****k 182
Colin Blackburn c****b@b****k 24
Declan Valters d****l@b****k 23
John A Stevenson v****0@u****m 15
dsut d****t@b****k 7
Declan Valters d****s@g****m 6
Bruno P. Kinoshita k****w@u****m 3
Real-Gecko a****v@g****m 3
Daniel Sutton d****n@g****m 2
Colin Blackburn x****k@u****m 1
Kent Inverarity k****y@h****m 1
Committer Domains (Top 20 + Academic)
bgs.ac.uk: 4

Issues and Pull Requests

Last synced: 11 months ago

All Time
  • Total issues: 64
  • Total pull requests: 94
  • Average time to close issues: 6 months
  • Average time to close pull requests: 15 days
  • Total issue authors: 10
  • Total pull request authors: 12
  • Average comments per issue: 2.11
  • Average comments per pull request: 1.09
  • Merged pull requests: 87
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 4
  • Pull requests: 2
  • Average time to close issues: 18 days
  • Average time to close pull requests: about 1 hour
  • Issue authors: 2
  • Pull request authors: 1
  • Average comments per issue: 0.0
  • Average comments per pull request: 0.0
  • Merged pull requests: 2
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • volcan01010 (40)
  • rbroth (11)
  • leorudczenko (3)
  • rwolniak (2)
  • sophie-taylor (1)
  • metazool (1)
  • KoalaGeo (1)
  • mgierdal (1)
  • LuiguiSaenz (1)
  • ximenesuk (1)
Pull Request Authors
  • volcan01010 (68)
  • leorudczenko (21)
  • ximenesuk (5)
  • jbow0131 (4)
  • KoalaGeo (3)
  • spenny-liam (2)
  • kerberpolis (2)
  • Real-Gecko (2)
  • sophiedorward (2)
  • rbroth (1)
  • rwolniak (1)
  • kinow (1)
Top Labels
Issue Labels
bug (1) documentation (1) enhancement (1)
Pull Request Labels
documentation (1)

Packages

  • Total packages: 1
  • Total downloads:
    • pypi 20,870 last-month
  • Total docker downloads: 372
  • Total dependent packages: 0
  • Total dependent repositories: 2
  • Total versions: 29
  • Total maintainers: 3
pypi.org: etlhelper

etlhelper is Python ETL library to simplify data transfer into and out of databases.

  • Versions: 29
  • Dependent Packages: 0
  • Dependent Repositories: 2
  • Downloads: 20,870 Last month
  • Docker Downloads: 372
Rankings
Downloads: 2.1%
Docker downloads count: 2.2%
Average: 7.0%
Stargazers count: 7.7%
Forks count: 8.6%
Dependent packages count: 10.0%
Dependent repos count: 11.6%
Maintainers (3)
volcan01010 decvalts ximenesuk
Last synced: 11 months ago

Dependencies

Dockerfile docker
  • python 3.6.9-slim build
.github/workflows/test.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v4 composite
  • suo/flake8-github-action releases/v1 composite
.github/workflows/gh-pages.yml actions
  • sphinx-notes/pages v3 composite
pyproject.toml pypi
requirements-dev.txt pypi
  • build * development
  • cffi * development
  • flake8 * development
  • flake8-annotations * development
  • ipdb * development
  • ipython * development
  • mypy * development
  • oracledb * development
  • psycopg2-binary * development
  • pyodbc * development
  • pytest >=4.6 development
  • pytest-cov * development
  • sphinx * development
  • sphinx_rtd_theme * development
  • twine * development
  • types-psycopg2 * development
  • wheel * development