Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/basedosdados/pipelines

🔀 Orquestrador de fluxos de captura, ingestão e tratamento de dados da BD

https://github.com/basedosdados/pipelines

Science Score: 26.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • â—‹
    CITATION.cff file
  • âś“
    codemeta.json file
    Found codemeta.json file
  • âś“
    .zenodo.json file
    Found .zenodo.json file
  • â—‹
    DOI references
  • â—‹
    Academic publication links
  • â—‹
    Academic email domains
  • â—‹
    Institutional organization owner
  • â—‹
    JOSS paper metadata
  • â—‹
    Scientific vocabulary similarity
    Low similarity (3.9%) to scientific vocabulary

Keywords

hacktoberfest hacktoberfest2022 opendata opensource prefect python
Last synced: 5 months ago · JSON representation

Repository

🔀 Orquestrador de fluxos de captura, ingestão e tratamento de dados da BD

Basic Info
  • Host: GitHub
  • Owner: basedosdados
  • Language: Jupyter Notebook
  • Default Branch: main
  • Homepage:
  • Size: 12.3 MB
Statistics
  • Stars: 38
  • Watchers: 6
  • Forks: 16
  • Open Issues: 119
  • Releases: 0
Topics
hacktoberfest hacktoberfest2022 opendata opensource prefect python
Created about 4 years ago · Last pushed 6 months ago
Metadata Files
Readme Contributing

README.md

Base dos Dados

Universalizando o acesso a dados de qualidade.

Pipelines

Esse repositório contém fluxos de captura e subida de dados no datalake da Base dos Dados.

đź‘Ą Como contribuir

Leia nosso guia de contribuição

🗺️ Contatos

Sinta-se livre para entrar em contato com projeto por nossas redes sociais.
Fortemente te incetivamos a participar da nossa comunidade pelo discord, onde você pode ter um gostinho de toda a Base Dos Dados. 🫶

Owner

  • Name: Base dos Dados
  • Login: basedosdados
  • Kind: organization
  • Email: contato@basedosdados.org
  • Location: Brazil

Universalizando o acesso a dados de qualidade

GitHub Events

Total
  • Fork event: 3
  • Create event: 140
  • Commit comment event: 1
  • Issues event: 174
  • Watch event: 4
  • Delete event: 117
  • Member event: 3
  • Issue comment event: 166
  • Push event: 818
  • Gollum event: 8
  • Pull request review comment event: 94
  • Pull request review event: 194
  • Pull request event: 260
Last Year
  • Fork event: 3
  • Create event: 140
  • Commit comment event: 1
  • Issues event: 174
  • Watch event: 4
  • Delete event: 117
  • Member event: 3
  • Issue comment event: 166
  • Push event: 818
  • Gollum event: 8
  • Pull request review comment event: 94
  • Pull request review event: 194
  • Pull request event: 260

Issues and Pull Requests

Last synced: 6 months ago

All Time
  • Total issues: 249
  • Total pull requests: 420
  • Average time to close issues: about 1 month
  • Average time to close pull requests: 6 days
  • Total issue authors: 30
  • Total pull request authors: 20
  • Average comments per issue: 0.22
  • Average comments per pull request: 1.31
  • Merged pull requests: 314
  • Bot issues: 10
  • Bot pull requests: 19
Past Year
  • Issues: 98
  • Pull requests: 176
  • Average time to close issues: 17 days
  • Average time to close pull requests: 5 days
  • Issue authors: 11
  • Pull request authors: 9
  • Average comments per issue: 0.23
  • Average comments per pull request: 0.85
  • Merged pull requests: 113
  • Bot issues: 1
  • Bot pull requests: 2
View more stats
Top Authors
Issue Authors
  • laura-l-amaral (97)
  • folhesgabriel (57)
  • lucascr91 (13)
  • tricktx (11)
  • JoaoCarabetta (11)
  • rdahis (9)
  • github-actions[bot] (9)
  • aspeddro (8)
  • vilelaluiza (5)
  • Winzen (4)
  • arthurfg (3)
  • IngridSRabelo (2)
  • d116626 (2)
  • Italo-Engers (2)
  • gdocesp (1)
Pull Request Authors
  • folhesgabriel (88)
  • tricktx (78)
  • aspeddro (73)
  • lucascr91 (55)
  • Winzen (27)
  • laura-l-amaral (26)
  • arthurfg (20)
  • pre-commit-ci[bot] (19)
  • luizavboas (5)
  • d116626 (5)
  • vncsna (4)
  • gabriel-milan (4)
  • jeantozzi (4)
  • athossampayo (3)
  • laribritto (3)
Top Labels
Issue Labels
bug (18) productivity (9) good first issue (7) milestone-pipelines (4) data (3) documentation (2) enhancement (2) suporte (2) hacktoberfest-accepted (1)
Pull Request Labels
table-approve (19) conflict (15) hacktoberfest-accepted (8) test-dev-model (6) bug (5) data (2) enhancement (1)

Dependencies

poetry.lock pypi
  • 186 dependencies
pyproject.toml pypi
  • pytest_cov ^3.0.0 develop
  • HeapDict 1.0.1
  • Jinja2 3.0.3
  • MarkupSafe 2.0.1
  • PyYAML 6.0
  • Unidecode ^1.3.4
  • basedosdados 1.6.4b4
  • beautifulsoup4 4.11.1
  • cachetools 4.2.4
  • certifi 2021.10.8
  • charset-normalizer 2.0.8
  • ckanapi 4.6
  • click 8.0.3
  • cloudpickle 2.0.0
  • croniter 1.0.15
  • dask 2021.11.2
  • dbt-client ^0.1.3
  • distributed 2021.11.2
  • docker 5.0.3
  • docopt 0.6.2
  • fsspec 2021.11.1
  • google-api-core 1.31.5
  • google-auth 1.35.0
  • google-auth-oauthlib 0.4.6
  • google-cloud-bigquery 2.30.1
  • google-cloud-bigquery-storage 1.1.0
  • google-cloud-core 2.2.1
  • google-cloud-storage 1.42.3
  • google-crc32c 1.3.0
  • google-resumable-media 2.1.0
  • googleapis-common-protos 1.53.0
  • grpcio 1.42.0
  • hvac ^0.11.2
  • idna 3.3
  • ipeadatapy ^0.1.7
  • locket 0.2.1
  • loguru ^0.6.0
  • marshmallow 3.14.1
  • marshmallow-oneofschema 3.0.1
  • matplotlib ^3.5.2
  • msgpack 1.0.3
  • mypy-extensions 0.4.3
  • numpy 1.21.4
  • oauthlib 3.1.1
  • packaging 21.3
  • pandas 1.2.4
  • pandas-gbq 0.17.4
  • partd 1.2.0
  • pendulum 2.1.2
  • prefect 0.15.9
  • proto-plus 1.19.8
  • protobuf 3.19.1
  • psutil 5.8.0
  • pyaml 20.4.0
  • pyarrow 6.0.0
  • pyasn1 0.4.8
  • pyasn1-modules 0.2.8
  • pydata-google-auth 1.2.0
  • pymssql 2.2.3
  • pyparsing 3.0.6
  • python >=3.8,<3.11
  • python-box 5.4.1
  • python-dateutil 2.8.2
  • python-slugify 5.0.2
  • pytz 2021.3
  • pytzdata 2020.1
  • requests 2.26.0
  • requests-oauthlib 1.3.0
  • rsa 4.8
  • ruamel.yaml 0.17.10
  • ruamel.yaml.clib 0.2.6
  • seaborn ^0.11.2
  • six 1.16.0
  • sortedcontainers 2.4.0
  • tabulate 0.8.9
  • tblib 1.7.0
  • text-unidecode 1.3
  • toml 0.10.2
  • tomlkit 0.7.0
  • toolz 0.11.2
  • tornado 6.1
  • tqdm 4.50.2
  • tweepy 4.4.0
  • urllib3 1.26.7
  • websocket-client 1.2.1
  • wget ^3.2
  • zict 2.0.0
requirements-cli.txt pypi
  • cookiecutter <=1.7.99
  • loguru <=0.5.99
  • typer <=0.4.99
requirements-deploy.txt pypi
  • google-cloud-storage ==1.42.1
  • loguru ==0.6.0
  • pandas ==1.3.5
  • poetry ==1.1.11
  • prefect ==0.15.9
  • requests ==2.23.0
  • typer ==0.4.0
requirements-test.txt pypi
  • pytest-cov ==3.0.0
.github/workflows/build-docker.yaml actions
  • actions/checkout v2 composite
  • google-github-actions/setup-gcloud v0.2.1 composite
.github/workflows/cd.yaml actions
  • Ana06/get-changed-files v2.1.0 composite
  • actions/checkout v2 composite
  • actions/setup-python v2 composite
  • google-github-actions/setup-gcloud v0.2.1 composite
  • lewagon/wait-on-check-action v1.1.2 composite
.github/workflows/cd_staging.yaml actions
  • Ana06/get-changed-files v2.1.0 composite
  • actions/checkout v2 composite
  • actions/setup-python v2 composite
  • google-github-actions/setup-gcloud v0.2.1 composite
  • lewagon/wait-on-check-action v1.1.2 composite
.github/workflows/ci.yaml actions
  • Ana06/get-changed-files v2.1.0 composite
  • actions/checkout v2 composite
  • actions/setup-python v2 composite
  • thollander/actions-comment-pull-request v1 composite
.github/workflows/docs.yaml actions
  • JamesIves/github-pages-deploy-action v4.2.3 composite
  • actions/checkout v2 composite
  • actions/setup-python v2 composite
.github/workflows/lint_docker.yaml actions
  • actions/checkout v2 composite
  • reviewdog/action-hadolint v1 composite
.github/workflows/lint_python.yaml actions
  • actions/checkout v2 composite
  • actions/setup-python v2 composite
  • gabriel-milan/action-pylint v1 composite
Dockerfile docker
  • python ${PYTHON_VERSION} build
.github/workflows/productivity.yml actions
  • imjohnbo/issue-bot 3d96848fb5e9a4a473bb81ae62b4b4866a56e93a composite