Wetterdienst - Open weather data for humans

"What do we want? Climate Justice! When do we want it? Now!" - FFF

[!WARNING] This library is a work in progress! Breaking changes should be expected until a 1.0 release, so version pinning is recommended.

Badges

CI

Meta

Downloads

Citation

Overview

Welcome to Wetterdienst, your friendly weather service library for Python.

We are a group of like-minded people trying to make access to weather data in Python feel like a warm summer breeze, similar to other projects like rdwd for the R language, which originally drew our interest in this project. Our long-term goal is to provide access to multiple weather services as well as other related agencies such as river measurements. With wetterdienst we try to use modern Python technologies all over the place. The library is based on polars (we <3 pandas, it is still part of some IO processes) across the board, uses uv for package administration and GitHub Actions for all things CI. Our users are an important part of the development as we are not currently using the data we are providing and only implement what we think would be the best. Therefore, contributions and feedback whether it be data related or library related are very welcome! Just hand in a PR or Issue if you think we should include a new feature or data source.

Data

For an overview of the data we have currently made available and under which license it is published take a look at the data section. Detailed information on datasets and parameters is given at the coverage subsection. Licenses and usage requirements may differ for each provider so check this out before including the data in your project to be sure that you fulfill copyright requirements!

For a closer look on the DWD data, you can use the interactive map or the table provided by the rdwd package.

Features

• APIs for stations and values
• Get stations nearby a selected location
• Define your request by arguments such as parameters, periods, start date, end date
• Define general settings in Settings context
• Command line interfaced
• Web-API via FastAPI, hosted on wetterdienst.eobs.org
• Rich UI features like explorer, stripes
• Run SQL queries on the results
• Export results to databases and other data sinks
• Public Docker image
• Interpolation and Summary of station values

Setup

Native

Via PyPi (standard):

bash pip install wetterdienst

Via Github (most recent):

bash pip install git+https://github.com/earthobservations/wetterdienst

There are some extras available for wetterdienst. Use them like:

bash pip install wetterdienst[sql]

• cratedb: Install support for CrateDB.
• duckdb: Install support for DuckDB.
• export: Install openpyxl for Excel export and pyarrow for writing files in Feather- and Parquet-format.
• influxdb: Install support for InfluxDB.
• interpolation: Install support for station interpolation.
• mysql: Install support for MySQL.
• matplotlib: Install support for plotting (matplotlib), only used in radar examples.
• plotting: Install support for plotting (plotly).
• postgresql: Install support for PostgreSQL.
• sql: Install DuckDB for querying data using SQL.

To check the installation, invoke:

bash wetterdienst --help

Docker

Docker images for each stable release will get pushed to GitHub Container Registry.

wetterdienst serves a full environment, including all the optional dependencies of Wetterdienst.

Pull the Docker image:

bash docker pull ghcr.io/earthobservations/wetterdienst

Library

Use the latest stable version of wetterdienst:

bash $ docker run -ti ghcr.io/earthobservations/wetterdienst Python 3.8.5 (default, Sep 10 2020, 16:58:22) [GCC 8.3.0] on linux

python import wetterdienst wetterdienst.__version__

Command line script

The wetterdienst command is also available:

```bash

Make an alias to use it conveniently from your shell.

alias wetterdienst='docker run -ti ghcr.io/earthobservations/wetterdienst wetterdienst'

wetterdienst --help wetterdienst --version wetterdienst info ```

Raspberry Pi / LINUX ARM

Running wetterdienst on Raspberry Pi, you need to install numpy and lxml prior to installing wetterdienst by running the following lines:

```bash

not all installations may be required to get lxml running

sudo apt-get install gfortran sudo apt-get install libopenblas-base sudo apt-get install libopenblas-dev sudo apt-get install libatlas-base-dev sudo apt-get install python3-lxml ```

Additionally expanding the Swap to 2048 mb may be required and can be done via swap-file:

bash sudo nano /etc/dphys-swapfile

Thanks chr-sto for reporting back to us!

Example

Task

Get historical climate summary for two German stations between 1990 and 2020

Library

```python from wetterdienst import Settings from wetterdienst.provider.dwd.observation import DwdObservationRequest

settings = Settings( # default tsshape="long", # tidy data tshumanize=True, # humanized parameters tsconvertunits=True # convert values to SI units )

request = DwdObservationRequest( parameters=[ ("daily", "climatesummary", "precipitationheight"), ], startdate="2002-08-11", # if not given timezone defaulted to UTC enddate="2002-08-13", # if not given timezone defaulted to UTC settings=settings ).filterbystationid(stationid=(5779,))

stations = request.df stations.head()

┌────────────┬─────────────────┬────────────┬─────────────────────────┬───┬───────────┬────────┬──────────────────────┬─────────┐

│ resolution ┆ dataset ┆ stationid ┆ startdate ┆ … ┆ longitude ┆ height ┆ name ┆ state │

│ --- ┆ --- ┆ --- ┆ --- ┆ ┆ --- ┆ --- ┆ --- ┆ --- │

│ str ┆ str ┆ str ┆ datetime[μs, UTC] ┆ ┆ f64 ┆ f64 ┆ str ┆ str │

╞════════════╪═════════════════╪════════════╪═════════════════════════╪═══╪═══════════╪════════╪══════════════════════╪═════════╡

│ daily ┆ climate_summary ┆ 05779 ┆ 1971-01-01 00:00:00 UTC ┆ … ┆ 13.7516 ┆ 877.0 ┆ Zinnwald-Georgenfeld ┆ Sachsen │

└────────────┴─────────────────┴────────────┴─────────────────────────┴───┴───────────┴────────┴──────────────────────┴─────────┘

values = request.values.all().df values.head()

┌────────────┬────────────┬─────────────────┬──────────────────────┬─────────────────────────┬───────┬─────────┐

│ station_id ┆ resolution ┆ dataset ┆ parameter ┆ date ┆ value ┆ quality │

│ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- │

│ str ┆ str ┆ str ┆ str ┆ datetime[μs, UTC] ┆ f64 ┆ f64 │

╞════════════╪════════════╪═════════════════╪══════════════════════╪═════════════════════════╪═══════╪═════════╡

│ 05779 ┆ daily ┆ climatesummary ┆ precipitationheight ┆ 2002-08-11 00:00:00 UTC ┆ 67.9 ┆ 10.0 │

│ 05779 ┆ daily ┆ climatesummary ┆ precipitationheight ┆ 2002-08-12 00:00:00 UTC ┆ 312.0 ┆ 10.0 │

│ 05779 ┆ daily ┆ climatesummary ┆ precipitationheight ┆ 2002-08-13 00:00:00 UTC ┆ 26.3 ┆ 10.0 │

└────────────┴────────────┴─────────────────┴──────────────────────┴─────────────────────────┴───────┴─────────┘

to get a pandas DataFrame and e.g. create some matplotlib plots

values.to_pandas() ```

Client

```bash

Get list of all stations for daily climate summary data in JSON format

wetterdienst stations --provider=dwd --network=observation --parameters=daily/kl --all

Get daily climate summary data for specific stations

wetterdienst values --provider=dwd --network=observation --parameters=daily/kl --station=1048,4411 ```

Other

Checkout examples for more examples.

Acknowledgements

We want to acknowledge all environmental agencies which provide their data open and free of charge first and foremost for the sake of endless research possibilities.

We want to acknowledge all contributors for being part of the improvements to this library that make it better and better every day.

Important Links

• Restapi: https://wetterdienst.eobs.org/
• Explorer: https://wetterdienst.streamlit.app/
• Stripes: https://stripes.streamlit.app/

• Documentation: https://wetterdienst.readthedocs.io/

  • Usage: https://wetterdienst.readthedocs.io/en/latest/usage/
  • Contribution: https://wetterdienst.readthedocs.io/en/latest/contribution/
  • Changelog: https://wetterdienst.readthedocs.io/en/latest/changelog.html

• Examples (runnable scripts): https://github.com/earthobservations/wetterdienst/tree/main/examples

• Benchmarks: https://github.com/earthobservations/wetterdienst/tree/main/benchmarks

Supported by

Special thanks to the kind people at JetBrains s.r.o. for supporting us with excellent development tooling.