MagicBathyNet: A Multimodal Remote Sensing Dataset for Benchmarking Learning-based Bathymetry and Pixel-based Classification in Shallow Waters

MagicBathyNet is a benchmark dataset made up of image patches of Sentinel-2, SPOT-6 and aerial imagery, bathymetry in raster format and seabed classes annotations. Dataset also facilitates unsupervised learning for model pre-training in shallow coastal areas. It is developed in the context of MagicBathy project.


DOI of GitHub Repository

Package for benchmarking MagicBathyNet dataset in learning-based bathymetry and pixel-based classification.

This repository contains the code of the paper "P. Agrafiotis, Ł. Janowski, D. Skarlatos and B. Demir, "MAGICBATHYNET: A Multimodal Remote Sensing Dataset for Bathymetry Prediction and Pixel-Based Classification in Shallow Waters," IGARSS 2024 - 2024 IEEE International Geoscience and Remote Sensing Symposium, Athens, Greece, 2024, pp. 249-253, doi: 10.1109/IGARSS53475.2024.10641355."

Citation

If you find this repository useful, please consider giving a star ⭐.
If you use the code in this repository or the dataset please cite:

P. Agrafiotis, Ł. Janowski, D. Skarlatos and B. Demir, "MAGICBATHYNET: A Multimodal Remote Sensing Dataset for Bathymetry Prediction and Pixel-Based Classification in Shallow Waters," IGARSS 2024 - 2024 IEEE International Geoscience and Remote Sensing Symposium, Athens, Greece, 2024, pp. 249-253, doi: 10.1109/IGARSS53475.2024.10641355. @INPROCEEDINGS{10641355, author={Agrafiotis, Panagiotis and Janowski, Łukasz and Skarlatos, Dimitrios and Demir, Begüm}, booktitle={IGARSS 2024 - 2024 IEEE International Geoscience and Remote Sensing Symposium}, title={MAGICBATHYNET: A Multimodal Remote Sensing Dataset for Bathymetry Prediction and Pixel-Based Classification in Shallow Waters}, year={2024}, volume={}, number={}, pages={249-253}, doi={10.1109/IGARSS53475.2024.10641355}}

Getting started

Downloading the dataset

For downloading the dataset and a detailed explanation of it, please visit the MagicBathy Project website at https://www.magicbathy.eu/magicbathynet.html

Dataset structure

The folder structure should be as follows: ┗ 📂 magicbathynet/ ┣ 📂 agia_napa/ ┃ ┣ 📂 img/ ┃ ┃ ┣ 📂 aerial/ ┃ ┃ ┃ ┣ 📜 img_339.tif ┃ ┃ ┃ ┣ 📜 ... ┃ ┃ ┣ 📂 s2/ ┃ ┃ ┃ ┣ 📜 img_339.tif ┃ ┃ ┃ ┣ 📜 ... ┃ ┃ ┣ 📂 spot6/ ┃ ┃ ┃ ┣ 📜 img_339.tif ┃ ┃ ┃ ┣ 📜 ... ┃ ┣ 📂 depth/ ┃ ┃ ┣ 📂 aerial/ ┃ ┃ ┃ ┣ 📜 depth_339.tif ┃ ┃ ┃ ┣ 📜 ... ┃ ┃ ┣ 📂 s2/ ┃ ┃ ┃ ┣ 📜 depth_339.tif ┃ ┃ ┃ ┣ 📜 ... ┃ ┃ ┣ 📂 spot6/ ┃ ┃ ┃ ┣ 📜 depth_339.tif ┃ ┃ ┃ ┣ 📜 ... ┃ ┣ 📂 gts/ ┃ ┃ ┣ 📂 aerial/ ┃ ┃ ┃ ┣ 📜 gts_339.tif ┃ ┃ ┃ ┣ 📜 ... ┃ ┃ ┣ 📂 s2/ ┃ ┃ ┃ ┣ 📜 gts_339.tif ┃ ┃ ┃ ┣ 📜 ... ┃ ┃ ┣ 📂 spot6/ ┃ ┃ ┃ ┣ 📜 gts_339.tif ┃ ┃ ┃ ┣ 📜 ... ┃ ┣ 📜 [modality]_split_bathymetry.txt ┃ ┣ 📜 [modality]_split_pixel_class.txt ┃ ┣ 📜 norm_param_[modality]_an.txt ┃ ┣ 📂 puck_lagoon/ ┃ ┣ 📂 img/ ┃ ┃ ┣ 📜 ... ┃ ┣ 📂 depth/ ┃ ┃ ┣ 📜 ... ┃ ┣ 📂 gts/ ┃ ┃ ┣ 📜 ... ┃ ┣ 📜 [modality]_split_bathymetry.txt ┃ ┣ 📜 [modality]_split_pixel_class.txt ┃ ┣ 📜 norm_param_[modality]_pl.txt The mapping between RGB color values and classes is:

``` For the Agia Napa area: 0 : (0, 128, 0), #seagrass 1 : (0, 0, 255), #rock 2 : (255, 0, 0), #macroalgae 3 : (255, 128, 0), #sand 4 : (0, 0, 0)} #Undefined (black)

For the Puck Lagoon area: 0 : (255, 128, 0), #sand 1 : (0, 128, 0) , #eelgrass/pondweed 2 : (0, 0, 0)} #Undefined (black) ```

Clone the repo

git clone https://github.com/pagraf/MagicBathyNet.git

Installation Guide

The requirements are easily installed via Anaconda (recommended):

conda env create -f environment.yml

After the installation is completed, activate the environment:

conda activate magicbathynet

Open Jupyter Notebook:

jupyter notebook

Train and Test the models

To train and test the bathymetry models use MagicBathyNet_bathymetry.ipynb.

To train and test the pixel-based classification models use MagicBathyNet_pixelclass.ipynb.

Pre-trained Deep Learning Models

We provide code and model weights for the following deep learning models that have been pre-trained on MagicBathyNet for pixel-based classification and bathymetry tasks:

Pixel-based classification

| Model Names | Modality | Area | Pre-Trained PyTorch Models | | ----------- |----------| ---- |----------------------------------------------------------------------------------------------------------------------------------------------| | U-Net | Aerial | Agia Napa | unetaerialan.zip | | SegFormer | Aerial | Agia Napa | segformeraerialan.zip | | U-Net | Aerial | Puck Lagoon | unetaerialpl.zip | | SegFormer | Aerial | Puck Lagoon | segformeraerialpl.zip | | U-Net | SPOT-6 | Agia Napa | unetspot6an.zip | | SegFormer | SPOT-6 | Agia Napa | segformerspot6an.zip | | U-Net | SPOT-6 | Puck Lagoon | unetspot6pl.zip | | SegFormer | SPOT-6 | Puck Lagoon | segformerspot6pl.zip | | U-Net | Sentinel-2 | Agia Napa | unets2an.zip | | SegFormer | Sentinel-2 | Agia Napa | segformers2an.zip | | U-Net | Sentinel-2 | Puck Lagoon | unets2pl.zip | | SegFormer | Sentinel-2 | Puck Lagoon | segformers2pl.zip |

Learning-based Bathymetry

| Model Name | Modality | Area | Pre-Trained PyTorch Models | | ----------- |----------| ---- |----------------------------------------------------------------------------------------------------------------------------------------------| | Modified U-Net for bathymetry | Aerial | Agia Napa | bathymetryaerialan.zip | | Modified U-Net for bathymetry | Aerial | Puck Lagoon | bathymetryaerialpl.zip | | Modified U-Net for bathymetry | SPOT-6 | Agia Napa | bathymetryspot6an.zip | | Modified U-Net for bathymetry | SPOT-6 | Puck Lagoon | bathymetryspot6pl.zip | | Modified U-Net for bathymetry | Sentinel-2 | Agia Napa | bathymetrys2an.zip | | Modified U-Net for bathymetry | Sentinel-2 | Puck Lagoon | bathymetrys2pl.zip |

To achieve the results presented in the paper, use the parameters and the specific train-evaluation splits provided in the dataset. Parameters can be found here while train-evaluation splits are included in the dataset.

Example testing results

Example patch of the Agia Napa area (left), pixel classification results obtained by U-Net (middle) and predicted bathymetry obtained by MagicBathy-U-Net (right). For more information on the results and accuracy achieved read our paper.

Authors

Panagiotis Agrafiotis https://www.user.tu-berlin.de/pagraf/

Feedback

Feel free to give feedback, by sending an email to: agrafiotis@tu-berlin.de

Funding

This work is part of MagicBathy project funded by the European Union’s HORIZON Europe research and innovation programme under the Marie Skłodowska-Curie GA 101063294. Work has been carried out at the Remote Sensing Image Analysis group. For more information about the project visit https://www.magicbathy.eu/.