Insect Detect ML - Model training and data processing

This repository contains Jupyter notebooks that can be used to train custom YOLOv5, YOLOv6, YOLOv7 and YOLOv8 object detection models or a custom YOLOv5 image classification model. All notebooks can be run in Google Colab, where you will have access to a free cloud GPU for fast training without special hardware requirements.

The Python script for classification of the captured insect images is available in the custom yolov5 fork and can be used together with the provided insect classification model.

Use the process_metadata.py script for post-processing of metadata .csv files with classification results.

Model training

You can find more information about detection model training at the Insect Detect Docs 📑.

• YOLOv5 detection model training  
• YOLOv6 detection model training  
• YOLOv7 detection model training  
• YOLOv8 detection model training  

The PyTorch model weights can be converted to .blob format at tools.luxonis.com for on-device inference with the Luxonis OAK devices.

You can find more information about classification model training at the Insect Detect Docs 📑.

• YOLOv5 classification model training  

The notebook for classification model training includes export to ONNX format for faster CPU inference.

Classification

The modified classification script in the custom yolov5 fork includes the following added options:

• --sort-top1 sort the classified images to folders with the predicted top1 class as folder name
• --sort-prob sort images first by probability and then by top1 class (requires --sort-top1)
• --concat-csv concatenate all metadata .csv files and append classification results to new columns

More information about deployment of the classification script can be found at the Insect Detect Docs 📑.

Classification model

| Model
^{(.onnx) | size
(pixels) | Top1 Accuracytest
| Precisiontest
| Recalltest
| F1 scoretest
| | --------------------- | --------------------- | -------------------------- | ---------------------- | ------------------- | --------------------- | | EfficientNet-B0 | 128 | 0.972 | 0.971 | 0.967 | 0.969 |}

Table Notes

• The model was trained to 20 epochs with image size 128, batch size 64 and default settings and hyperparameters. Reproduce the model training with the provided Google Colab notebook.
• Trained on Insect Detect - insect classification dataset v2 with 27 classes. To reproduce the dataset split, keep the default settings in the Colab notebook (train/val/test ratio = 0.7/0.2/0.1, random seed = 1).
• Dataset can be explored at Roboflow Universe. Export from Roboflow compresses the images and can lead to a decreased model accuracy. It is recommended to use the uncompressed dataset from Zenodo.

Full model metrics on dataset test split (click to expand)

| Class | Images | Top1 Accuracy^{test
| Precisiontest
| Recalltest
| F1 scoretest
| | ------------ | ------ | -------------------------- | ---------------------- | ------------------- | --------------------- | | all | 2125 | 0.972 | 0.971 | 0.967 | 0.969 | | ant | 111 | 1.0 | 0.991 | 1.0 | 0.996 | | bee | 107 | 0.963 | 0.972 | 0.963 | 0.967 | | bee_apis | 31 | 1.0 | 0.969 | 1.0 | 0.984 | | bee_bombus | 127 | 1.0 | 0.992 | 1.0 | 0.996 | | beetle | 52 | 0.885 | 0.92 | 0.885 | 0.902 | | beetle_cocci | 78 | 0.987 | 1.0 | 0.987 | 0.994 | | beetle_oedem | 21 | 0.905 | 0.905 | 0.905 | 0.905 | | bug | 39 | 0.846 | 1.0 | 0.846 | 0.917 | | bug_grapho | 19 | 1.0 | 1.0 | 1.0 | 1.0 | | fly | 173 | 0.971 | 0.944 | 0.971 | 0.957 | | fly_empi | 19 | 1.0 | 1.0 | 1.0 | 1.0 | | fly_sarco | 33 | 0.909 | 0.938 | 0.909 | 0.923 | | fly_small | 167 | 0.958 | 0.952 | 0.958 | 0.955 | | hfly_episyr | 253 | 0.996 | 0.996 | 0.996 | 0.996 | | hfly_eristal | 197 | 0.99 | 0.995 | 0.99 | 0.992 | | hfly_eupeo | 137 | 0.985 | 0.993 | 0.985 | 0.989 | | hfly_myathr | 60 | 1.0 | 1.0 | 1.0 | 1.0 | | hfly_sphaero | 39 | 0.974 | 1.0 | 0.974 | 0.987 | | hfly_syrphus | 50 | 0.98 | 1.0 | 0.98 | 0.99 | | lepi | 24 | 1.0 | 0.96 | 1.0 | 0.98 | | none_bg | 86 | 0.988 | 0.966 | 0.988 | 0.977 | | none_bird | 8 | 1.0 | 1.0 | 1.0 | 1.0 | | none_dirt | 85 | 0.976 | 0.902 | 0.976 | 0.938 | | none_shadow | 66 | 0.924 | 0.953 | 0.924 | 0.938 | | other | 79 | 0.861 | 0.883 | 0.861 | 0.872 | | scorpionfly | 12 | 1.0 | 1.0 | 1.0 | 1.0 | | wasp | 52 | 1.0 | 1.0 | 1.0 | 1.0 |}

Metadata post-processing

Install the required packages by running:

python.exe -m pip install -r requirements.txt

Or use the Python Launcher for Windows with:

py -m pip install -r requirements.txt

The process_metadata.py script can be used to automatically post-process the concatenated metadata .csv file after the classification step, as it will still contain multiple rows for each tracked insect.

The output of the script includes a *top1_final.csv file in which each row corresponds to an individual tracked insect and its classification result with the highest weighted probability. Additionally, several plots are generated that can give a first overview of the processed metadata.

More information about deployment of the post-processing script can be found at the Insect Detect Docs 📑.

Image processing

The process_images.py script can be used to calculate different metrics of the captured images (e.g. mean/median/min/max width/height) and remove corrupted .jpg images from the data folder (camera trap output). These can be rarely generated by the automated monitoring script (e.g. in power outage situations) and will cause an error while running the classification script.

License

This repository is licensed under the terms of the GNU Affero General Public License v3.0 (GNU AGPLv3).

Citation

If you use resources from this repository, please cite our paper:

Sittinger M, Uhler J, Pink M, Herz A (2024) Insect detect: An open-source DIY camera trap for automated insect monitoring. PLoS ONE 19(4): e0295474. https://doi.org/10.1371/journal.pone.0295474