dask-flood-mapper

Map floods with Sentinel-1 radar images. We replicate in this package the work of Bauer-Marschallinger et al. (2022) on the TU Wien Bayesian-based flood mapping algorithm. This implementation is entirely based on Dask and data access via STAC with odc-stac.

https://github.com/intertwin-eu/dask-flood-mapper

Science Score: 67.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • CITATION.cff file
    Found CITATION.cff file
  • codemeta.json file
    Found codemeta.json file
  • .zenodo.json file
    Found .zenodo.json file
  • DOI references
    Found 6 DOI reference(s) in README
  • Academic publication links
    Links to: zenodo.org
  • Academic email domains
  • Institutional organization owner
  • JOSS paper metadata
  • Scientific vocabulary similarity
    Low similarity (12.9%) to scientific vocabulary

Keywords

dask earth-observation flood flood-monitoring flooding microwave-remote-sensing pangeo remote-sensing sentinel-1
Last synced: 6 months ago · JSON representation ·

Repository

Map floods with Sentinel-1 radar images. We replicate in this package the work of Bauer-Marschallinger et al. (2022) on the TU Wien Bayesian-based flood mapping algorithm. This implementation is entirely based on Dask and data access via STAC with odc-stac.

Basic Info
Statistics
  • Stars: 5
  • Watchers: 3
  • Forks: 4
  • Open Issues: 15
  • Releases: 7
Topics
dask earth-observation flood flood-monitoring flooding microwave-remote-sensing pangeo remote-sensing sentinel-1
Created over 1 year ago · Last pushed 6 months ago
Metadata Files
Readme Changelog Contributing License Code of conduct Citation Codeowners

README.md

Dask based Flood Mapping

CI DOI pypi GitHub Super-Linter GitHub Super-Linter SQAaaS source code

Map floods with Sentinel-1 radar images. We replicate in this package the work of Bauer-Marschallinger et al. (2022)[^1] on the TU Wien Bayesian-based flood mapping algorithm. This implementation is entirely based on dask and data access via STAC with odc-stac. The algorithm requires three pre-processed input datasets stored and accessible via STAC at the Earth Observation Data Centre For Water Resources Monitoring (EODC). It is foreseen that future implementations can also use data from other STAC catalogues. This notebook explains how microwave backscattering can be used to map the extent of a flood. The workflow detailed in this notebook forms the backbone of this package. For a short overview of the Bayesian decision method for flood mapping see this ProjectPythia book.

Installation

To install the package, do the following:

bash pip install dask-flood-mapper

Usage

Storm Babet hit the Denmark and Northern coast of Germany at the 20th of October 2023 Wikipedia. Here an area around Zingst at the Baltic coast of Northern Germany is selected as the study area.

Local Processing

Define the time range and geographic region in which the event occurred.

python time_range = "2022-10-11/2022-10-25" bbox = [12.3, 54.3, 13.1, 54.6]

Use the flood module and calculate the flood extent with the Bayesian decision method applied tp Sentinel-1 radar images. The object returned is a xarray with lazy loaded Dask arrays. To get the data in memory use the compute method on the returned object.

```python from daskfloodmapper import flood

flood.decision(bbox=bbox, datetime=time_range).compute() ```

Distributed Processing

It is also possible to remotely process the data at the EODC Dask Gateway with the added benefit that we can then process close to the data source without requiring rate-limiting file transfers over the internet.

For ease of usage of the Dask Gateway install the eodc package besides the dask-gateway package. Also, see the EODC documentation.

```bash pip install dask-gateway eodc

or use pipenv

git clone https://github.com/interTwin-eu/dask-flood-mapper.git

cd dask-flood-mapper

pipenv sync -d

```

However differences in versions client- and server-side can cause problems. Hence, the most convenient way to successively use the EODC Dask Gateway is Docker. To do this clone the GitHub repository and use the docker-compose.yml.

bash git clone https://github.com/interTwin-eu/dask-flood-mapper.git cd dask-flood-mapper docker compose up

Copy and paste the generated URL to launch Jupyter Lab in your browser. Here one can run the below code snippets or execute the notebook about remote processing.

```python from eodc.dask import EODCDaskGateway from eodc import settings from rich.prompt import Prompt

settings.DASKURL = "http://dask.services.eodc.eu" settings.DASKURL_TCP = "tcp://dask.services.eodc.eu:10000/" ```

Connect to the gateway (this requires an EODC account).

python your_username = Prompt.ask(prompt="Enter your Username") gateway = EODCDaskGateway(username=your_username)

Create a cluster.

[!CAUTION] Per default no worker is spawned, therefore please use the widget to add/scale Dask workers in order to enable computations on the cluster.

python cluster_options = gateway.cluster_options() cluster_options.image = "ghcr.io/eodcgmbh/cluster_image:2025.4.1" cluster = gateway.new_cluster(cluster_options) client = cluster.get_client() cluster

Map the flood the same way as we have done when processing locally.

python flood.decision(bbox=bbox, datetime=time_range).compute()

User Interface

It is also possible to run the workflow in an user-friendly interface, as shown below:

screenshot

Firstly, install the extra packages with:

bash pip install dask-flood-mapper[app]

Then, to access it, simplify run the in terminal the command:

bash floodmap

It will open the GUI in the web browser.

Authors

Martin Schobben, Thais Beham, Clay Harrison

Contributors

https://github.com/interTwin-eu/dask-flood-mapper/graphs/contributors

Contributing Guidelines

Please find the contributing guidelines in the specific file CONTRIBUTING.md.

Automated Delivery

This repository holds a container image to be used for running Dask based flood mapping on the EODC Dask Gateway. Use the URL ghcr.io/intertwin-eu/dask-flood-mapper:latest to specify the image.

bash docker pull ghcr.io/intertwin-eu/dask-flood-mapper:latest

Credits

Credits go to EODC (https://eodc.eu) for developing the infrastructure and the management of the data required for this workflow. This work has been supported as part of the interTwin project (https://www.intertwin.eu). The interTwin project is funded by the European Union Horizon Europe Programme - Grant Agreement number 101058386.

Views and opinions expressed are however those of the authors only and do not necessarily reflect those of the European Union Horizon Europe/Horizon 2020 Programmes. Neither the European Union nor the granting authorities can be held responsible for them.

License

This repository is covered under the MIT License.

Literature

[^1]: Bauer-Marschallinger, Bernhard, Senmao Cao, Mark Edwin Tupas, Florian Roth, Claudio Navacchi, Thomas Melzer, Vahid Freeman, and Wolfgang Wagner. Satellite-Based Flood Mapping through Bayesian Inference from a Sentinel-1 SAR Datacube. Remote Sensing 14, no. 15 (January 2022): 3673. https://doi.org/10.3390/rs14153673.

Owner

  • Name: interTwin Community
  • Login: interTwin-eu
  • Kind: organization
  • Email: info@intertwin.eu

Co-designing and prototyping an interdisciplinary Digital Twin Engine.

Citation (CITATION.cff)

cff-version: 1.2.0
message: "If you use the Dask flood mapper, please cite it as below."
authors:
  - family-names: Beham
    given-names: Thais
    website: https://github.com/thaisbeham
    affiliation: Technische Universität Wien, Vienna, Austria
  - family-names: Harrison
    given-names: Clay
    orcid: https://orcid.org/0009-0004-0057-612X
    website: https://github.com/claytharrison
    affiliation: Technische Universität Wien, Vienna, Austria
  - family-names: Schobben
    given-names: Martin
    orcid: https://orcid.org/0000-0001-8560-0037
    website: https://github.com/martinschobben
    affiliation: Technische Universität Wien, Vienna, Austria
title: "Dask Flood Mapper"
abstract: "Map floods with Sentinel-1 radar images. We replicate in this \
  package the work of Bauer-Marschallinger et al. (2022) on the TU Wien \
  Bayesian-based flood mapping algorithm. This implementation is entirely \
  based on Dask and data access via STAC with odc-stac. The algorithm requires \
  three pre-processed input datasets stored and accessible via STAC at the \
  Earth Observation Data Centre For Water Resources Monitoring (EODC)."

GitHub Events

Total
  • Create event: 17
  • Issues event: 5
  • Release event: 8
  • Delete event: 11
  • Issue comment event: 15
  • Push event: 80
  • Pull request review comment event: 4
  • Pull request review event: 12
  • Pull request event: 20
Last Year
  • Create event: 17
  • Issues event: 5
  • Release event: 8
  • Delete event: 11
  • Issue comment event: 15
  • Push event: 80
  • Pull request review comment event: 4
  • Pull request review event: 12
  • Pull request event: 20

Issues and Pull Requests

Last synced: 6 months ago

All Time
  • Total issues: 8
  • Total pull requests: 73
  • Average time to close issues: 4 days
  • Average time to close pull requests: 2 days
  • Total issue authors: 2
  • Total pull request authors: 5
  • Average comments per issue: 1.38
  • Average comments per pull request: 0.81
  • Merged pull requests: 61
  • Bot issues: 0
  • Bot pull requests: 7
Past Year
  • Issues: 8
  • Pull requests: 73
  • Average time to close issues: 4 days
  • Average time to close pull requests: 2 days
  • Issue authors: 2
  • Pull request authors: 5
  • Average comments per issue: 1.38
  • Average comments per pull request: 0.81
  • Merged pull requests: 61
  • Bot issues: 0
  • Bot pull requests: 7
Top Authors
Issue Authors
  • thaisbeham (7)
  • MartinSchobben (1)
Pull Request Authors
  • MartinSchobben (34)
  • thaisbeham (26)
  • dependabot[bot] (7)
  • claytharrison (4)
  • npikall (2)
Top Labels
Issue Labels
documentation (1) enhancement (1)
Pull Request Labels
dependencies (7) github_actions (7) documentation (1) enhancement (1)

Packages

  • Total packages: 1
  • Total downloads:
    • pypi 131 last-month
  • Total dependent packages: 0
  • Total dependent repositories: 0
  • Total versions: 6
  • Total maintainers: 1
pypi.org: dask-flood-mapper

Map floods with Sentinel-1 radar images. We replicate in this package the work

  • Versions: 6
  • Dependent Packages: 0
  • Dependent Repositories: 0
  • Downloads: 131 Last month
Rankings
Dependent packages count: 9.3%
Average: 30.8%
Dependent repos count: 52.3%
Maintainers (1)
Last synced: 6 months ago

Dependencies

pyproject.toml pypi
requirements.txt pypi
  • Cartopy ==0.23.0
  • Jinja2 ==3.1.4
  • Markdown ==3.7
  • MarkupSafe ==2.1.5
  • PyYAML ==6.0.2
  • Pygments ==2.18.0
  • Send2Trash ==1.8.3
  • affine ==2.4.0
  • aiohappyeyeballs ==2.4.0
  • aiohttp ==3.10.5
  • aiosignal ==1.3.1
  • anyio ==4.4.0
  • argon2-cffi ==23.1.0
  • argon2-cffi-bindings ==21.2.0
  • arrow ==1.3.0
  • asttokens ==2.4.1
  • async-lru ==2.0.4
  • async-timeout ==4.0.3
  • attrs ==24.2.0
  • babel ==2.16.0
  • beautifulsoup4 ==4.12.3
  • black ==24.8.0
  • bleach ==6.1.0
  • bokeh ==3.4.3
  • cachetools ==5.5.0
  • certifi ==2024.8.30
  • cffi ==1.17.1
  • charset-normalizer ==3.3.2
  • click ==8.1.7
  • click-plugins ==1.1.1
  • cligj ==0.7.2
  • cloudpickle ==3.0.0
  • colorcet ==3.1.0
  • comm ==0.2.2
  • contourpy ==1.3.0
  • cycler ==0.12.1
  • dask ==2024.9.0
  • dask_labextension ==7.0.0
  • datashader ==0.16.3
  • debugpy ==1.8.5
  • decorator ==5.1.1
  • defusedxml ==0.7.1
  • distributed ==2024.9.0
  • exceptiongroup ==1.2.2
  • executing ==2.1.0
  • fastjsonschema ==2.20.0
  • fonttools ==4.53.1
  • fqdn ==1.5.1
  • frozenlist ==1.4.1
  • fsspec ==2024.9.0
  • geoviews ==1.12.0
  • h11 ==0.14.0
  • holoviews ==1.19.1
  • httpcore ==1.0.5
  • httpx ==0.27.2
  • hvplot ==0.10.0
  • idna ==3.10
  • importlib_metadata ==8.5.0
  • ipykernel ==6.29.5
  • ipython ==8.27.0
  • ipywidgets ==8.1.5
  • isoduration ==20.11.0
  • jedi ==0.19.1
  • json5 ==0.9.25
  • jsonpointer ==3.0.0
  • jsonschema ==4.23.0
  • jsonschema-specifications ==2023.12.1
  • jupyter-events ==0.10.0
  • jupyter-lsp ==2.2.5
  • jupyter_client ==8.6.2
  • jupyter_core ==5.7.2
  • jupyter_server ==2.14.2
  • jupyter_server_proxy ==4.4.0
  • jupyter_server_terminals ==0.5.3
  • jupyterlab ==4.2.5
  • jupyterlab_pygments ==0.3.0
  • jupyterlab_server ==2.27.3
  • jupyterlab_widgets ==3.0.13
  • kiwisolver ==1.4.7
  • linkify-it-py ==2.0.3
  • llvmlite ==0.43.0
  • locket ==1.0.0
  • markdown-it-py ==3.0.0
  • matplotlib ==3.9.2
  • matplotlib-inline ==0.1.7
  • mdit-py-plugins ==0.4.2
  • mdurl ==0.1.2
  • mistune ==3.0.2
  • msgpack ==1.1.0
  • multidict ==6.1.0
  • multipledispatch ==1.0.0
  • mypy-extensions ==1.0.0
  • nbclient ==0.10.0
  • nbconvert ==7.16.4
  • nbformat ==5.10.4
  • nest-asyncio ==1.6.0
  • nodejs ==0.1.1
  • notebook_shim ==0.2.4
  • numba ==0.60.0
  • numpy ==2.0.2
  • odc-geo ==0.4.8
  • odc-stac ==0.3.10
  • optional-django ==0.1.0
  • overrides ==7.7.0
  • packaging ==24.1
  • pandas ==2.2.2
  • pandocfilters ==1.5.1
  • panel ==1.4.5
  • param ==2.1.1
  • parso ==0.8.4
  • partd ==1.4.2
  • pathspec ==0.12.1
  • pexpect ==4.9.0
  • pillow ==10.4.0
  • platformdirs ==4.3.3
  • prometheus_client ==0.20.0
  • prompt_toolkit ==3.0.47
  • psutil ==6.0.0
  • ptyprocess ==0.7.0
  • pure_eval ==0.2.3
  • pycparser ==2.22
  • pyct ==0.5.0
  • pyparsing ==3.1.4
  • pyproj ==3.6.1
  • pyshp ==2.3.1
  • pystac ==1.10.1
  • pystac-client ==0.8.3
  • python-dateutil ==2.9.0.post0
  • python-json-logger ==2.0.7
  • pytz ==2024.2
  • pyviz_comms ==3.0.3
  • pyzmq ==26.2.0
  • rasterio ==1.3.11
  • referencing ==0.35.1
  • requests ==2.32.3
  • rfc3339-validator ==0.1.4
  • rfc3986-validator ==0.1.1
  • rioxarray ==0.17.0
  • rpds-py ==0.20.0
  • scipy ==1.14.1
  • shapely ==2.0.6
  • simpervisor ==1.0.0
  • six ==1.16.0
  • sniffio ==1.3.1
  • snuggs ==1.4.7
  • sortedcontainers ==2.4.0
  • soupsieve ==2.6
  • stack-data ==0.6.3
  • tblib ==3.0.0
  • terminado ==0.18.1
  • tinycss2 ==1.3.0
  • tomli ==2.0.1
  • toolz ==0.12.1
  • tornado ==6.4.1
  • tqdm ==4.66.5
  • traitlets ==5.14.3
  • types-python-dateutil ==2.9.0.20240906
  • typing_extensions ==4.12.2
  • tzdata ==2024.1
  • uc-micro-py ==1.0.3
  • uri-template ==1.3.0
  • urllib3 ==2.2.3
  • wcwidth ==0.2.13
  • webcolors ==24.8.0
  • webencodings ==0.5.1
  • websocket-client ==1.8.0
  • widgetsnbextension ==4.0.13
  • xarray ==2024.9.0
  • xyzservices ==2024.9.0
  • yarl ==1.11.1
  • zict ==3.0.0
  • zipp ==3.20.2
setup.py pypi