Zenodo DOI:

QuPath Spotiflow extension

This repo adds some support to use Spotiflow within QuPath through a Python virtual environment.

[!WARNING] This extension is developed for QuPath 0.6.x

Citing

Please cite this extension by linking it to this GitHub or to the release you used, and feel free to give us a star ⭐️

As this code is neither novel nor entirely original, there is no linked publication, but you can use the following Zenodo entry:

Code authorship

Author: Rémy Dornier (1)

Contributors: Albert Dominguez Mantes (2), Olivier Burri (1), Nicolas Chiaruttini (1), Romain Guiet (1)

This code heavily mirrors functions and design implemented in QuPath Cellpose.

Affiliations

(1) EPFL BioImaging and Optics Platform (BIOP) (2) EPFL Weigert Lab

If you use this extension, you should cite the following publications

Dominguez Mantes, A., Herrera, A., Khven, I., Schlaeppi, A., Kyriacou, E., Tsissios, G., Skoufa, E., Santangeli, L., Buglakova, E., Berna Durmus, E., Manley, S., Kreshuk, A., Arendt, D., Aztekin, C., Lingner, J., La Manno, G., Weigert, M. Spotiflow: accurate and efficient spot detection for fluorescence microscopy with deep stereographic flow regression. bioRxiv 2024.02.01.578426. https://doi.org/10.1101/2024.02.01.578426

Bankhead, P. et al. QuPath: Open source software for digital pathology image analysis. Scientific Reports (2017). https://doi.org/10.1038/s41598-017-17204-5

Installation

Step 1: Install Spotiflow

Installation from the official Spotiflow documentation

You will need a Python virtual environment with Spotiflow working.
Please follow the instructions to install Spotiflow from the main Spotiflow repository.

Get the path to the Python executable

This extension will then need to know the path (the location in your file system) to (at least) your Spotiflow environment. You will need to enter this information in the QuPath Preferences when configuring the QuPath Spotiflow extension.

[!IMPORTANT] Make sure you activate your Spotiflow environment and then check the path!

For example, if you installed spotiflow into a conda environment named spotiflow-biop then you can get the path to the Python executable using the command line or terminal using the following two commands to activate the conda environment and locate the Python binary (the 3rd line is example output).
On Windows: ```

mamba activate spotiflow-biop where python F:\conda-envs\spotiflow-biop\python.exe ```

On macOS or Linux: ```

mamba activate spotiflow-biop which python {HOME}/conda/envs/spotiflow-biop/bin/python `` where{HOME}will be the location of your home directory, typically/Users/user-name/on macOS or/home/user-name/` on Linux.

Step 2: Install the QuPath Spotiflow extension

On QuPath 0.6.x

• Install and setup the qupath-biop-catalog first.
• Open the Extension Manager
• Follow steps described in the official documention to browse and install/update the latest version of QuPath Spotiflow extension from the QuPath BIOP catalog.

[!NOTE] If your extensions directory is unset, create a new empty folder somewhere on your computer. On QuPath, go under Edit -> Preferences -> Extensions and browse the empty folder you've just created. Extensions will then be added inside this folder.

• You might then need to restart QuPath (but not your computer).

On QuPath 0.5.x

Download the latest qupath-extension-spotiflow-[version].jar file from releases and copy it into your extensions directory.

If your extensions directory is unset, drag & drop qupath-extension-spotiflow-[version].jar onto the main QuPath window. You'll be prompted to select a QuPath user directory. The extension will then be copied to a location inside that directory. You finally need to restart QuPath.

QuPath Extension Spotiflow: First time setup

Go to Edit > Preferences > Spotiflow Complete the fields with the requested information by pasting the path(s) you obtained above. Based on the mamba installation above, this is what it should look like on Windows:

[!NOTE] Unless stated otherwise, this extension should work with the latest releases of both Spotiflow. If this is not the case, please open an issue on our GitHub or write a post on the Image.sc forum detailing the problem and the steps necessary to reproduce it.

Running Spotiflow the first time in standalone

Spotiflow needs to download the pretrained models the first time it is run. On some OSes, this does not work from within QuPath due to permission issues.

One trick is to run Spotiflow from the command line once with the model you want to use. The download should work from there, and you can then use it within the QuPath Extension Spotiflow.

spotiflow-predict path/to/your/image

or, to download a specific model, different from the general one

spotiflow-predict path/to/your/image --pretrained-model hybiss

Using the Spotiflow QuPath Extension

Prediction

Running Spotiflow is done via a script and is very similar to the excellent QuPath Cellpose Extension

You can find a template in QuPath in

Extensions > Spotiflow > Spotiflow detection template

Or you can download the Spotiflowdetectiontemplate.groovy script from this repo and open it in the QuPath script editor.

[!IMPORTANT] Lines starting with // are commented out and are not used. To enable those parameters, delete the //. To disable a parameter, add // at the start of the line. Some exmaples as provided in the announcement forum post

Getting all available arguments

All builder options that are implemented are documented in the Javadoc. You can pass additional options to spotiflow by adding .addParameter() before the .build() line.

To get the available additional arguments, call the help from the spotiflow object using spotiflow.helpPredict() def spotiflow = Spotiflow.builder()...build() spotiflow.helpPredict()

Prediction using custom models

To use a custom model or your own trained model, provide the path to the directory containing the model to the Spotiflow.builder using .setModelDir() def pathModel = 'C:/spotiflow-custom-models/my-nice-spotiflow-model' def spotiflow = Spotiflow.builder() .setModelDir(new File(pathModel))

Prediction using other pre-trained models

To use a pre-trained model, different from the default one (general), provide the name of this model to the Spotiflow.builder using .setPretrainedModelName() def pretrainedModel = 'hybiss' def spotiflow = Spotiflow.builder() .setPretrainedModelName(pretrainedModel) <!--

Training custom models

Requirements: A QuPath project with rectangles of class "Training" and "Validation" inside which the ground truth objects have been painted as annotations with no class.

[!IMPORTANT] Training requires dense annotations. This means you cannot just annotate a few objects per Training and Valudation rectanble. You must annotate ALL relevant objects within each of those regions!

We typically create a standalone QuPath project for training only. This project will contain the training images along with the ground truth annotations drawn in QuPath. Here are some reasons we do it this way:

1. Separating training and prediction/analysis makes for clean project structures and easier sharing of the different steps of your workflow.
2. In case we need to get more ground truth, we can simply fire up the relevant QuPath project and rerun the training, and then import the newly trained model into any other project that might need it.

Protocol

1. In your QuPath project create at least 2 rectangle annotations.
2. In the Annotations tab, add new classes name "Training" and "Validation" and assign your rectangles to each of them. You do not need an equal number of Training and Validation rectangles.
3. Lock the rectangles (right click > Annotations > Lock).
4. Draw your ground truth annotations within all of the rectangles. You can also select each rectangle and run the Cellpose detection script template with the .createAnnotations() line not commented out in the builder (see Prediction instructions above) to use a pre-trained cellpose model as a start, but make sure you manually correct it to get proper ground truth!

[!IMPORTANT] Any ground truth annotations must have no classes assigned.

1. Repeat this for as many images/regions of interest as you would like.

[!WARNING] All images with Training or Validation annotations in the project will be used for the training.

Once you have your labeled Training and Validation rectangles with ground truth annotations, make sure you save your project! Then you can run the Cellpose training template script in Extensions > Cellpose > Cellpose training script template

Or you can download Cellposetrainingtemplate.groovy from this repo and run it from the script editor.

[!NOTE] In the line def cellpose = Cellpose2D.builder("cyto") you can choose to fine-tune a pre-trained model (e.g. cyto), train from scratch (enter "None"), or start with a custom model (see below). Please see the Prediction instructions above for information regarding the other builder parameters.

The first thing the script will do is create a sub-folder in your project called cellpose-training containing sub-folders test and train, followed by exporting the image(s) that will be processed by cellpose. The train folder will contain images of your training rectangles and your annotations converted to masks. The test folder will contain the Validation data, which is also used by the QC script. If your Validation is not what you expect, you can check that the exported image(s) represent what you intended for cellpose to train on.

Once the script successfully completes training, you will have a models sub-folder within your Project folder, which will contain your custom model, as well as a QC sub-folder with the output of the QC script.

Once you are happy with your training script, you should save the edited copy to your Project (or another scripts folder) for re-use!

Training a custom model

To train using your custom model, you need to provide the path to the model to the Cellpose2D.builder. Just replace the name of the pre-trained model (e.g. cyto) with the path to your model, for example: // Specify the model name (cyto, nuclei, cyto2, omni_bact or a path to your custom model as a string) def pathModel = 'C:/cellpose-custom-models/cellpose_residual_on_style_on_concatenation_off_train_2023_07_26_11_31_47.433625' def cellpose = Cellpose2D.builder( pathModel )

[!NOTE]
If you decide that your model needs more training, you can add more images to the Project and provide more annotated Training rectangles.
You can use your custom model to segment additional Training rectangles, as described in the Prediction section above. Just be sure to perform careful manual correction!
Then save the Project and re-run the training script with the path of the custom model from the previous training step.
If you save your edited Prediction and Training scripts, you can repeat this process as needed, by adding more training data and training the model obtained from the previous run--just edit the path to the model.
This is analogous to the cellpose 2.0 GUI human-in-the-loop process.

More training options

All options in Cellpose have not been transferred.

In case that this might be of use to you, please open an issue.

Training validation

You can find a run-cellpose-qc.py python script in the QC folder of this repository. This is an adaptation of the Quality Control part of a ZeroCostDL4Mic Notebook that was made for cellpose.

Basically, when you train using this extension: 1. It will first train your model as expected 2. It will then run your newly trained cellpose model on your "Validation" images 3. At the end, it will run the run-cellpose-qc.py python script to output validation metrics. 4. The validation metrics will be saved into a folder called QC in your QuPath Project

Saving training results for publication purposes

In order to be as reproducible and sure of your results as possible, especially when it comes to publishing, these are our current guidelines: 1. Use saveBuilder() which saves a JSON file of your CellposeBuilder, which can be reused with CellposeBuilder(File builderFile). That way you will not lose the setting your did 2. Save the cellpose-training, QC and models folders at the end of your training somewhere. This will contain everything that was made during training. 3. Save the training script as well.

-->

Building

You can build the QuPath Spotiflow extension from source with

bash gradlew clean build

The output will be under build/libs.

• clean removes anything old
• build builds the QuPath extension as a .jar file and adds it to libs