Marine Detect 🌊🐟

This repository provides access to two YOLOv8 object detection models for identifying species of interest in underwater environments.

These models were developed in the context of the Let's Revive project in partnership with Tēnaka. Tēnaka emphasizes impact measurement through Tēnaka Science, sharing monthly coral ecosystem data. To automate data collection, Orange Business and Tēnaka partner to deploy an underwater research station with cameras. The system uses AI to identify and quantify 17 species in coral areas, streamlining monitoring and allowing marine biologists more time for restoration.

Images: @copyright Tēnaka

🐟 Species Scope

The Fish and Invertebrates Object Detection Model detects the Fish and Invertebrates Species and the MegaFauna Object Detection Model detects MegaFauna and Rare Species.

• MegaFauna and Rare Species: Sharks, Sea Turtles, Rays.
• Fish Species: Butterfly Fish (Chaetodontidae), Grouper (Serranidae), Parrotfish (Scaridae), Snapper (Lutjanidae), Moray Eel (Muraenidae), Sweet Lips (Haemulidae), Barramundi Cod (Cromileptes altivelis), Humphead (Napoleon) Wrasse (Cheilinus undulatus), Bumphead Parrotfish (Bolbometopon muricatum), Fish (other than above or unrecognizable).
• Invertebrates Species: Giant Clam, Urchin, Sea Cucumber, Lobster, Crown of Thorns.

These species are "bio-indicating" species, which serve as indicators of the ecosystem health. These bio-indicating species are of course dependent on each region - here the focus is for Malaysia/Indo-Pacific region.

📊 Datasets Details

The models utilize a combination of publicly available datasets (~ 90%) and Tēnaka-based datasets (~ 10%). Some datasets were already annotated, and others undergo manual labeling.

References to the public datasets used can be found in the 'References' section of this README.

The images used with annotations (YOLO format) can be downloaded using the following links: FishInv dataset, MegaFauna dataset.

Datasets split details

| Model | Training + Validation Sets | Test Set | | -------------- | -------------------------- | -------- | | FishInv | 12,243 images (80%, 20%) | 499 images | | MegaFauna | 8,130 images (80%, 20%) | 253 images |

[!NOTE] The rationale behind the development of two distinct models lies in the utilization of already annotated images available in public datasets. By having separate models, we sidestep the necessity of reannotating images that already encompass annotations for specific species with every Fish, Invertebrates and MegaFauna species. For example, we found a lot of images of turtles already annotated. If we were to adopt a single, all-encompassing model for both Fish and Invertebrates Species 🐟 and MegaFauna 🦈, it would necessitate the reannotation of all those turtle images to include species like urchins, fishes, ...

🤖 Model Details

You can download the trained models using the following links: FishInv model, MegaFauna model.

Performances on test sets

🚗 Usage

🏁 Environment Setup

There are 3 options to install the development environment.

Option 1 - Developing Inside a Docker Container with Visual Studio Code's Dev Containers Extension (recommended):

• If you are using Windows, make sure that Windows Subsytem for Linux is installed and working on your machine (to do so, follow the instructions here).
• Make sure Docker is installed on your machine.
• Install the Dev Containers Extension in Visual Studio Code (ms-vscode-remote.remote-containers).
• In VS Code, open the command palette (CTRL + SHIFT + P) and select Dev Containers: Rebuild and Reopen in Container (make sure Docker is running before executing this step). If the build seems to freeze, read the "Common Errors and Fixes" section below.

Note that the Dockerfile was created for CPU machines. If you wish to use GPU for inference, you can change the base image to nvidia/cuda:12.0.0-runtime-ubuntu22.04.

Option 2 - Developing on Your Host OS with Anaconda:

• Make sure Conda is installed and working on your machine (to do so, click here).
• Then, run the following commands in the project directory: shell conda create --name your_env_name python=3.10 conda activate your_env_name pip install -r requirements.txt

Option 3 - Developing on Your Host OS with PIP:

• Make sure pyenv is installed and working
• Then, run the following commands in the project directory: shell pyenv install 3.10 pyenv local 3.10 python -m venv venv source venv/bin/activate pip install -r requirements.txt

🚀 Inference

To make predictions on images or videos using the marine detection models, follow these simple steps:

1. Models Download: Refer to the 'Model Details' section for information on the models.
2. Prediction Functions: Utilize the following Python functions to generate predictions with bounding box annotations.

```python from src.marinedetect.predict import predictonimages, predicton_video

Predict on a set of images using FishInv and MegaFauna models

predictonimages( modelpaths=["path/to/FishInv/model", "path/to/MegaFauna/model"], confsthreshold=[0.523, 0.546], imagesinputfolderpath="path/to/input/images", imagesoutputfolderpath="path/to/output/folder", )

Predict on a video using FishInv and MegaFauna models

predictonvideo( modelpaths=["path/to/FishInv/model", "path/to/MegaFauna/model"], confsthreshold=[0.523, 0.546], inputvideopath="path/to/input/video.mp4", outputvideopath="path/to/output/video.mp4", ) ```

[!NOTE] The optimal confidence thresholds for the FishInv model and the MegaFauna model are 0.523 and 0.546, respectively.

The resulting images or video files will have bounding boxes annotations, visually indicating the location and extent of the detected marine species within the original data.

For example:

📚 References

Datasets

• Ticon Dataset. (2023). Shark Dataset [ Open Source Dataset ]. In Roboflow Universe . Roboflow . https://universe.roboflow.com/ticon-dataset/shark-ibmby
• Minhajul Arefin. (2021). zebrashark Dataset [ Open Source Dataset ]. In Roboflow Universe . Roboflow . https://universe.roboflow.com/minhajul-arefin/zebrashark
• Rizal Fadia Al Fikri. (2022). sharkspecies Dataset [ Open Source Dataset ]. In Roboflow Universe . Roboflow . https://universe.roboflow.com/rizal-fadia-al-fikri/sharkspecies
• Aya Abd-Elnaser. (2022). SHARK Dataset [ Open Source Dataset ]. In Roboflow Universe . Roboflow . https://universe.roboflow.com/aya-abd-elnaser/shark-jatfb
• Nomi. (2023). seaturtle Dataset [ Open Source Dataset ]. In Roboflow Universe . Roboflow . https://universe.roboflow.com/nomi/seaturtle
• Parvej Hosen. (2022). Turtle Dataset [ Open Source Dataset ]. In Roboflow Universe . Roboflow . https://universe.roboflow.com/parvej-hosen/turtle-f9xgw
• Seami New 5 Fishes. (2023). EagleRay New Dataset [ Open Source Dataset ]. In Roboflow Universe . Roboflow . https://universe.roboflow.com/seami-new-5-fishes/eagleray-new
• Le Wagon. (2023). count-a-manta Dataset [ Open Source Dataset ]. In Roboflow Universe . Roboflow . https://universe.roboflow.com/le-wagon-w02yl/count-a-manta
• Renaldo Rasfuldi. (2022). fishid2 Dataset [ Open Source Dataset ]. In Roboflow Universe . Roboflow . https://universe.roboflow.com/renaldo-rasfuldi/fishid2
• Universiti Teknologi Malaysia. (2023). Giant Clam Dataset [ Open Source Dataset ]. In Roboflow Universe . Roboflow . https://universe.roboflow.com/universiti-teknologi-malaysia-juyvx/giant-clam
• Universiti Teknologi Malaysia. (2023). Tioman Giant Clams Dataset [ Open Source Dataset ]. In Roboflow Universe . Roboflow . https://universe.roboflow.com/universiti-teknologi-malaysia-juyvx/tioman-giant-clams
• Jacob Solawetz. (2023). Fish Dataset [ Open Source Dataset ]. In Roboflow Universe . Roboflow . https://universe.roboflow.com/roboflow-gw7yv/fish-yzfml
• Dataset. (2022). Dataset Dataset [ Open Source Dataset ]. In Roboflow Universe . Roboflow . https://universe.roboflow.com/dataset-gdypo/dataset-axhm3
• Addison Howard, W. K., Eunbyung Park. (2018). ImageNet Object Localization Challenge. Kaggle. https://kaggle.com/competitions/imagenet-object-localization-challenge
• Australian Institute of Marine Science (AIMS), University of Western Australia (UWA) and Curtin University. (2019), OzFish Dataset - Machine learning dataset for Baited Remote Underwater Video Stations, https://doi.org/10.25845/5e28f062c5097
• GBIF.org (09 January 2024) GBIF Occurrence Download https://doi.org/10.15468/dl.w5xy62
• GBIF.org (09 January 2024) GBIF Occurrence Download https://doi.org/10.15468/dl.a5uwzp
• GBIF.org (09 January 2024) GBIF Occurrence Download https://doi.org/10.15468/dl.r5xqkc
• GBIF.org (09 January 2024) GBIF Occurrence Download https://doi.org/10.15468/dl.ug7n62
• GBIF.org (19 December 2023) GBIF Occurrence Download https://doi.org/10.15468/dl.32mwtb

Model

• Jocher, G., Chaurasia, A., & Qiu, J. (2023). Ultralytics YOLO (Version 8.0.0) [Computer software]. https://github.com/ultralytics/ultralytics