Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/basedosdados/databasers-utils

Pacote para uso da equipe de dados

https://github.com/basedosdados/databasers-utils

Science Score: 26.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (2.0%) to scientific vocabulary
Last synced: 10 months ago · JSON representation

Repository

Pacote para uso da equipe de dados

Basic Info
  • Host: GitHub
  • Owner: basedosdados
  • License: mit
  • Language: Python
  • Default Branch: main
  • Homepage:
  • Size: 35.2 KB
Statistics
  • Stars: 0
  • Watchers: 2
  • Forks: 0
  • Open Issues: 2
  • Releases: 0
Created about 2 years ago · Last pushed about 1 year ago
Metadata Files
Readme Contributing License

README.md

databasers-utils

Pacote para uso da equipe de dados na Base dos Dados.

Setup

Credenciais

Para fazer o upload de colunas usando a arquitetura você deve configurar suas credenciais.

Linux/WSL

Adicione no seu .bashrc ou .zshrc

sh export BD_DJANGO_EMAIL="seuemail@basedosdados.org" export BD_DJANGO_PASSWORD="password"

Windows

  • Em Pesquisar, procure e selecione: Sistema (Painel de Controle)
  • Clique no link Configurações avançadas do sistema.
  • Clique em Variáveis de Ambiente.
  • Na seção 'Variáveis do Sistema' clique em 'Novo' e insira BD_DJANGO_EMAIL como nome da variável e o seu email como valor. Faça o mesmo para BD_DJANGO_PASSWORD.
  • Clique em OK. Feche todas as janelas restantes clicando em OK.

Uso

```python from databasersutils import ( TableArchitecture, copymodelsfromdevtoprod, getarchitecturetablefromapi, )

arch = TableArchitecture( datasetid="bribgepib", tables={ "uf": "https://docs.google.com/spreadsheets/d/12F5NzhOYlNbi9flLBEdXDWpa5iVakSP4EKm9UoyWuo/edit?usp=drivelink", "gini": "https://docs.google.com/spreadsheets/d/1K1svie4Gyqe6NnRjBgJbapU5sTsLqXWTQUmTRVIRwQc/edit?usp=drivelink", }, )

Retorna um dict[str, pd.DataFrame], cada chave o id da tabela e o dataframe

da arquitetura

arch.tables()

Cria o yaml file

arch.createyamlfile()

Cria os arquivos sql

arch.createsqlfiles()

Atualiza o dbt_project.yml

arch.updatedbtproject()

Faz o upload das colunas para o DJango

arch.upload_columns()

Copia os modelos em dev para prod

copymodelsfromdevtoprod(["bribgeppm", "bribge_pam"])

Retorna um DataFrame da arquitetura obtida na API

Util para gerar arquitetura quando ela não está no Drive

getarchitecturetablefromapi("brmssinasc", "microdados") ```

Owner

  • Name: Base dos Dados
  • Login: basedosdados
  • Kind: organization
  • Email: contato@basedosdados.org
  • Location: Brazil

Universalizando o acesso a dados de qualidade

GitHub Events

Total
  • Delete event: 5
  • Member event: 2
  • Push event: 9
  • Pull request event: 6
  • Create event: 3
Last Year
  • Delete event: 5
  • Member event: 2
  • Push event: 9
  • Pull request event: 6
  • Create event: 3

Issues and Pull Requests

Last synced: 10 months ago

All Time
  • Total issues: 5
  • Total pull requests: 10
  • Average time to close issues: 20 days
  • Average time to close pull requests: 3 days
  • Total issue authors: 2
  • Total pull request authors: 2
  • Average comments per issue: 0.2
  • Average comments per pull request: 0.2
  • Merged pull requests: 10
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 1
  • Pull requests: 7
  • Average time to close issues: 14 days
  • Average time to close pull requests: about 6 hours
  • Issue authors: 1
  • Pull request authors: 2
  • Average comments per issue: 0.0
  • Average comments per pull request: 0.0
  • Merged pull requests: 7
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • aspeddro (5)
  • laura-l-amaral (3)
Pull Request Authors
  • aspeddro (11)
  • Winzen (2)
Top Labels
Issue Labels
Pull Request Labels

Dependencies

.github/workflows/ci.yaml actions
  • actions/checkout v4 composite
  • actions/setup-python v4 composite
poetry.lock pypi
  • anyio 4.4.0
  • backoff 2.2.1
  • basedosdados 2.0.0b15
  • cachetools 5.3.3
  • certifi 2024.6.2
  • charset-normalizer 3.3.2
  • click 8.1.7
  • colorama 0.4.6
  • db-dtypes 1.2.0
  • exceptiongroup 1.2.1
  • fastavro 1.9.4
  • google-api-core 2.19.0
  • google-api-python-client 2.133.0
  • google-auth 2.30.0
  • google-auth-httplib2 0.2.0
  • google-auth-oauthlib 1.2.0
  • google-cloud-bigquery 3.24.0
  • google-cloud-bigquery-connection 1.15.3
  • google-cloud-bigquery-storage 2.25.0
  • google-cloud-core 2.4.1
  • google-cloud-storage 2.17.0
  • google-crc32c 1.5.0
  • google-resumable-media 2.7.1
  • googleapis-common-protos 1.63.1
  • gql 3.5.0
  • graphql-core 3.2.3
  • grpc-google-iam-v1 0.13.0
  • grpcio 1.64.1
  • grpcio-status 1.62.2
  • httplib2 0.22.0
  • idna 3.7
  • iniconfig 2.0.0
  • loguru 0.7.0
  • multidict 6.0.5
  • numpy 1.26.4
  • oauthlib 3.2.2
  • packaging 24.1
  • pandas 2.2.2
  • pandas-gbq 0.19.2
  • pandavro 1.8.0
  • pluggy 1.5.0
  • proto-plus 1.23.0
  • protobuf 4.25.3
  • pyarrow 16.1.0
  • pyasn1 0.6.0
  • pyasn1-modules 0.4.0
  • pydata-google-auth 1.8.2
  • pyparsing 3.1.2
  • pytest 8.2.2
  • python-dateutil 2.9.0.post0
  • pytz 2024.1
  • requests 2.32.3
  • requests-oauthlib 2.0.0
  • requests-toolbelt 1.0.0
  • rsa 4.9
  • ruamel-yaml 0.18.6
  • ruamel-yaml-clib 0.2.8
  • ruff 0.4.8
  • setuptools 70.0.0
  • six 1.16.0
  • sniffio 1.3.1
  • tomli 2.0.1
  • tomlkit 0.11.8
  • tqdm 4.66.4
  • typing-extensions 4.12.2
  • tzdata 2024.1
  • uritemplate 4.1.1
  • urllib3 2.2.1
  • win32-setctime 1.1.0
  • yarl 1.9.4
pyproject.toml pypi
  • pytest ^8.2.2 develop
  • ruff ^0.4.8 develop
  • basedosdados 2.0.0b15
  • numpy ^1.26.4
  • pandas ^2.2.2
  • python ~3.9
  • requests ^2.32.3
  • ruamel-yaml ^0.18.6