airbase

🌬 An easy downloader for the AirBase air quality data.

https://github.com/johnpaton/airbase

Science Score: 26.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • CITATION.cff file
  • codemeta.json file
    Found codemeta.json file
  • .zenodo.json file
    Found .zenodo.json file
  • DOI references
  • Academic publication links
  • Committers with academic emails
  • Institutional organization owner
  • JOSS paper metadata
  • Scientific vocabulary similarity
    Low similarity (12.0%) to scientific vocabulary

Keywords

air-quality airbase data python
Last synced: 6 months ago · JSON representation

Repository

🌬 An easy downloader for the AirBase air quality data.

Basic Info
Statistics
  • Stars: 11
  • Watchers: 2
  • Forks: 4
  • Open Issues: 4
  • Releases: 5
Topics
air-quality airbase data python
Created over 7 years ago · Last pushed 10 months ago
Metadata Files
Readme License

README.md

PyPI version Downloads CI/CD Documentation Status pre-commit Code style: Ruff Checked with mypy

🌬 AirBase

An easy downloader for air quality data provided by the European Environment Agency (EEA).

The data is available for download at the portal, but the interface makes it a bit time consuming to do bulk downloads. Hence, an easy Python-based interface.

Read the full documentation at https://airbase.readthedocs.io.

🔌 Installation

To install airbase, simply run

bash $ pip install airbase

🚀 Getting Started

🗺 Get info about available countries and pollutants:

```pycon

import airbase client = airbase.AirbaseClient() client.countries frozenset({'LI', 'CY', 'IE', 'LV', 'BE', 'EE', ...})

client.pollutants frozenset({'Co', 'sum-PCB', 'PCB-26', 'HNO3', ...})

client.search_pollutant("O3") [{'poll': 'O3', 'id': 7}, {'poll': 'NO3', 'id': 46}, ...] ```

🗂 Request download links from the server and save the resulting Parquet files into a directory:

```pycon

r = client.request("Verified", "NL", "DE", poll=["NO3", "NO3- in PM2.5", "NO3- in PM10"]) r.download(dir="data", skip_existing=True) summary : 100%|█████| 2/2 [00:00<00:00, 4.48requests/s] URLs : 100%|█████| 29.0/29.0 [00:00<00:00, 490URL/s] download: 386kb [00:00, 570kb/s] ```

📦 Download the entire dataset (not for the faint of heart):

```pycon

r = client.request() r.downloadtodirectory("data") summary : 100%|█████| 39/39 [00:15<00:00, 2.54requests/s] URLs : 100%|█████| 47.0k/47.0k [00:00<00:00, 77.7kURL/s] download: 20.6Gb [54:36, 6.74Mb/s] ```

🌡 Don't forget to get the metadata about the measurement stations:

```pycon

client.download_metadata("data/metadata.csv") Writing metadata to data/metadata.csv... ```

🚆 Command line interface

Historical data delivered between 2002 and 2012

``` console $ airbase historical --help Usage: airbase historical [OPTIONS]

Historical Airbase data delivered between 2002 and 2012 before Air Quality Directive 2008/50/EC entered into force.

Use -c/--country and -p/--pollutant to restrict the download specific countries and pollutants, e.g. - download only Norwegian, Danish and Finish sites airbase historical -c NO -c DK -c FI - download only SO2, PM10 and PM2.5 observations airbase historical -p SO2 -p PM10 -p PM2.5

Use -C/--city to further restrict the download to specific cities, e.g. - download only PM10 and PM2.5 from Valletta, the Capital of Malta airbase historical -C Valletta -c MT -p PM10 -p PM2.5

Options: -c, --country [AD|AL|AT|...] -p, --pollutant [k|V|NT|...] -C, --city TEXT only from selected -F, --aggregation-type, --frequency [hourly|daily|other] only hourly data, daily data or other aggregation frequency -M, --metadata download station metadata --path PATH [default: data/historical] -n, --dry-run, --summary Total download files/size, nothing will be downloaded. -O, --overwrite Re-download existing files. -q, --quiet No progress-bar. --help Show this message and exit. ```

Verified data from 2013 to 2023

``` console $ airbase verified --help Usage: airbase verified [OPTIONS]

Verified data (E1a) from 2013 to 2023 reported by countries by 30 September each year for the previous year.

Use -c/--country and -p/--pollutant to restrict the download specific countries and pollutants, e.g. - download only Norwegian, Danish and Finish sites airbase verified -c NO -c DK -c FI - download only SO2, PM10 and PM2.5 observations airbase verified -p SO2 -p PM10 -p PM2.5

Use -C/--city to further restrict the download to specific cities, e.g. - download only PM10 and PM2.5 from Valletta, the Capital of Malta airbase verified -C Valletta -c MT -p PM10 -p PM2.5

Options: -c, --country [AD|AL|AT|...] -p, --pollutant [k|V|NT|...] -C, --city TEXT only from selected -F, --aggregation-type, --frequency [hourly|daily|other] only hourly data, daily data or other aggregation frequency -M, --metadata download station metadata --path PATH [default: data/verified] -n, --dry-run, --summary Total download files/size, nothing will be downloaded. -O, --overwrite Re-download existing files. -q, --quiet No progress-bar. --help Show this message and exit. ```

Unverified data from the beginning of 2024

``` console $ airbase unverified --help Usage: airbase unverified [OPTIONS]

Unverified data transmitted continuously (Up-To-Date/UTD/E2a) data from the beginning of 2024.

Use -c/--country and -p/--pollutant to restrict the download specific countries and pollutants, e.g. - download only Norwegian, Danish and Finish sites airbase unverified -c NO -c DK -c FI - download only SO2, PM10 and PM2.5 observations airbase unverified -p SO2 -p PM10 -p PM2.5

Use -C/--city to further restrict the download to specific cities, e.g. - download only PM10 and PM2.5 from Valletta, the Capital of Malta airbase unverified -C Valletta -c MT -p PM10 -p PM2.5

Options: -c, --country [AD|AL|AT|...] -p, --pollutant [k|V|NT|...] -C, --city TEXT only from selected -F, --aggregation-type, --frequency [hourly|daily|other] only hourly data, daily data or other aggregation frequency -M, --metadata download station metadata --path PATH [default: data/unverified] -n, --dry-run, --summary Total download files/size, nothing will be downloaded. -O, --overwrite Re-download existing files. -q, --quiet No progress-bar. --help Show this message and exit. ```

🛣 Roadmap

  • ~~Parallel CSV downloads~~ Contributed by @avaldebe
  • ~~CLI to avoid using Python all together~~ Contributed by @avaldebe
  • Data wrangling module for AirBase output data

Owner

  • Name: John Paton
  • Login: JohnPaton
  • Kind: user
  • Company: @bookingcom

📊 Machine Learning Engineer / 🎓 Theoretical physics / 🐍 Pythonista / 💼 Booking.com / 📍 Netherlands 🇳🇱

GitHub Events

Total
  • Create event: 11
  • Release event: 2
  • Issues event: 13
  • Watch event: 2
  • Delete event: 3
  • Issue comment event: 15
  • Push event: 65
  • Pull request review comment event: 3
  • Pull request review event: 11
  • Pull request event: 12
Last Year
  • Create event: 11
  • Release event: 2
  • Issues event: 13
  • Watch event: 2
  • Delete event: 3
  • Issue comment event: 15
  • Push event: 65
  • Pull request review comment event: 3
  • Pull request review event: 11
  • Pull request event: 12

Committers

Last synced: 7 months ago

All Time
  • Total Commits: 396
  • Total Committers: 4
  • Avg Commits per committer: 99.0
  • Development Distribution Score (DDS): 0.306
Past Year
  • Commits: 181
  • Committers: 2
  • Avg Commits per committer: 90.5
  • Development Distribution Score (DDS): 0.066
Top Committers
Name Email Commits
Alvaro Valdebenito a****v@m****o 275
John Paton j****n@j****t 88
JohnPaton j****n@c****l 32
Heiko Klein h****n@m****o 1
Committer Domains (Top 20 + Academic)

Issues and Pull Requests

Last synced: 6 months ago

All Time
  • Total issues: 25
  • Total pull requests: 49
  • Average time to close issues: 9 months
  • Average time to close pull requests: about 1 month
  • Total issue authors: 6
  • Total pull request authors: 3
  • Average comments per issue: 2.08
  • Average comments per pull request: 2.08
  • Merged pull requests: 39
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 6
  • Pull requests: 17
  • Average time to close issues: 16 days
  • Average time to close pull requests: 17 days
  • Issue authors: 2
  • Pull request authors: 2
  • Average comments per issue: 2.83
  • Average comments per pull request: 2.18
  • Merged pull requests: 10
  • Bot issues: 0
  • Bot pull requests: 0
Top Authors
Issue Authors
  • JohnPaton (13)
  • avaldebe (9)
  • heikoklein (1)
  • omidnabavi (1)
  • b-kode (1)
  • charlienegri (1)
Pull Request Authors
  • avaldebe (35)
  • JohnPaton (28)
  • heikoklein (2)
Top Labels
Issue Labels
enhancement (9) bug (2) wontfix (1)
Pull Request Labels
enhancement (8) bug (2)

Packages

  • Total packages: 1
  • Total downloads:
    • pypi 161 last-month
  • Total dependent packages: 0
  • Total dependent repositories: 0
  • Total versions: 15
  • Total maintainers: 1
pypi.org: airbase

An easy downloader for the AirBase air quality data.

  • Documentation: https://airbase.readthedocs.io
  • License: MIT License Copyright (c) 2018 John Paton Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
  • Latest release: 1.0.0
    published about 1 year ago
  • Versions: 15
  • Dependent Packages: 0
  • Dependent Repositories: 0
  • Downloads: 161 Last month
Rankings
Dependent packages count: 7.3%
Forks count: 15.4%
Stargazers count: 18.5%
Downloads: 23.8%
Average: 26.7%
Dependent repos count: 68.5%
Maintainers (1)
Last synced: 6 months ago

Dependencies

docs/requirements.txt pypi
  • importlib_metadata *
  • sphinx >=4.2.0
  • sphinx_rtd_theme *
.github/workflows/cicd.yaml actions
  • actions/checkout v2 composite
  • actions/setup-python v2 composite
  • pre-commit/action v2.0.3 composite
  • pypa/gh-action-pypi-publish release/v1 composite
pyproject.toml pypi