Rework the Cellpose 3 detector integration for TrackMate v8.

Author: tinevez

_pages/plugins/trackmate/detectors/trackmate-cellpose.md

@@ -50,45 +50,41 @@ This is documented [on this page](/plugins/trackmate/trackmate-conda-path).
 {% include img src="/media/plugins/trackmate/detectors/cellpose/trackmate-cellpose-ui-01.png" align='center' width='300' %}
 We document these parameters from top to bottom in the GUI.
 
-<!---
+
 ##### `Conda environment`
 
-Specify in what conda environment you installed Cellpose-SAM. 
-If you get an error at this stage, it is likely because the conda path for TrackMate was not configured properly. Check [this page]((/plugins/trackmate/trackmate-conda-path).
--->
+Specify in what conda environment you installed Cellpose (the v3.1.1 in the case of this detector). 
+If you get an error at this stage, it is likely because the conda path for TrackMate was not configured properly. 
+Again, check [this page]((/plugins/trackmate/trackmate-conda-path) if this happens.
 
-##### `Path to cellpose / Python executable`
 
-We must specify where is the cellpose executable, as it was installed outside of Fiji. Because there are several ways of installing cellpose (with Conda or using the precompiled executables), we have to accommodate several cases we document [later in this document](#installing-cellpose). Briefly:
+##### `Pretrained model`
 
-- If you installed cellpose via the precompiled executables, enter the _path to the executable_. For instance:  `C:\Users\tinevez\Applications\cellpose.exe`
-- If you installed cellpose via Conda, enter the _path to the Python executable of the Conda environment you created for cellpose_. For instance: `C:\Users\tinevez\anaconda3\envs\cellpose\python.exe`
+This list lets you select the pretrained models that were shipped with Cellpose 3.
+They are dully documented in details on the [Cellpose doc website](https://cellpose.readthedocs.io/en/v3.1.1.1/models.html).
+The 'main' ones are:
+
+- the `cyto3` model, that excels at segmenting cells stained for their cytoplasm. This model supports the specification of a second channel (see below) where nuclei are stained, to guide cell sgmentation.
+- the `nuclei` model, trained to segment cell nuclei.
 
-##### `Pretrained model`
-Right now we only support three pretrained models of cellpose:
-- the `cyto` model ("Cytoplasm"), to segment cells stained for their cytoplasm;
-- the `nuclei` model ("Nuclei") to segment cell nuclei;
-- the `cyto2` model ("Cytoplasm 2.0"), which is the `cyto` model augmented with user-submitted images.
-- the `live cell` model from Cellpose was trained on the `LIVECELL` [dataset](https://sartorius-research.github.io/LIVECell/) of label free images of cells. 
-- the `TissueNet` model from Cellpose was trained on the `TissueNet` [dataset](https://datasets.deepcell.org/data) available from deepcell.
-- "Custom" lets you specify a custom model you would have trained or downloaded, which is documented below.
 
 ##### `Path to custom model`
 
-If in the `Pretrained model` selection you pick `Custom`, a text field will appear above, allowing for entering the path to a custom cellpose model. You need to browse to the actually model file generated by the training. For instance: `D:\Projects\Brightfield\Cellpose model 171121-20211118T111402Z-001\171121\cellpose_residual_on_style_on_concatenation_off_train_folder_2021_11_17_19_45_23.398850`
+There are radio buttons before the `Pretrained model` and `Path to custom model` that lets you select whether you want to use a pretrained model, or a custom one.
+When `Path to custom model` is selected, you need to browse to the actual model file.
 
 Note that it must be the an `absolute path` to the file.
 
-##### `Channel to segment`
+##### `Target channel`
 
 If your image has several color channels, choose here the channel to segment. The number of the channel corresponds to the _imagej channel_ in your image. 
 It should be the channel where the cytoplam staining is for cell segmentation, or the nuclei staining for nuclei segmentation, e.g. with the `nuclei` model.
 
 Note that TrackMate doesn't handle RGB stacks so you might need to convert your image if it is not already a composite image (see below for [tip on how to pass RGB images to TrackMate-Cellpose](#tip-passing-rgb-images-to-trackmate-for-cellpose))
 
-##### `Optional second channel`
+##### `Second optional channel`
 
-The `cyto` and `cyto2` pretrained models have been trained on images with a second channels in which the cell nuclei were labeled. It is used as a seed to make the detection of single cells more robust.
+The `cyto3` (and `cyto2` and `cyto`) pretrained models have been trained on images with a second channels in which the cell nuclei were labeled. It is used as a seed to make the detection of single cells more robust.
 It is optional and this parameter specifies in which channel are the nuclei (the number of the imagej channel of the nuclei). Use '0' to skip using the second optional channel.
 For the `nuclei` model, this parameter is ignored.
 
@@ -97,9 +93,19 @@ For the `nuclei` model, this parameter is ignored.
 cellpose can exploit an _a priori_ knowledge on the size of cells. If you have a rough estimate of the size of the cell, enter it here. In TrackMate this parameter must be specified in physical units (for instance µm if the source image has a pixel size expressed in µm).
 Enter the value '0' to have cellpose automatically determine the cell size estimate.
 
+_Careful!_ I insist a bit on this point.
+If you are used to running Cellpose from the Cellpose UI or a Python script, it expects the diameter to be in pixels. 
+But here in TrackMate, _the diameter is in the spatial units of your image_.
+
 ##### `Use GPU`
 
-If this box is checked, the GPU will be used _if cellpose was installed with required librairies and hardware to use the GPU_. If the GPU support is absent or incorrect, this setting will be safely ignored and the computation will rely on CPU only. Unchecking the box will force cellpose to use the CPU even if GPU support is available.
+If this box is checked, the GPU will be used _if cellpose was installed with required librairies and hardware to use the GPU_. 
+If the GPU support is absent or incorrect, this setting will be safely ignored and the computation will rely on CPU only. 
+Unchecking the box will force cellpose to use the CPU even if GPU support is available.
+
+We took special care to implement multi-threaded processing if you use CPU.
+With this setting, there will be one Cellpose process running per thread you set in the _Edit > Options > Memory and Threads..._ configuration ("Parallel Threads for stacks").
+And then each image corresponding to the time-points will be dispatched automatically to these multiple processes, running concurrently.
 
 ##### `Simplify contours`
 
@@ -108,11 +114,11 @@ If this box is checked, the contour outlines generated by the masks returned by
 
 ### (Re)Training of a cellpose model
 
-To optimize CellPose on your data, you can retrain one its model and use it in TrackMate-CellPose. You can do the retraining easily through [CellPose 2.0 GUI](https://www.nature.com/articles/s41592-022-01663-4). When the model gives acceptable results, select it in TrackMate interface with the `custom_model` parameter. 
+To optimize CellPose on your data, you can retrain one its model and use it in TrackMate-CellPose. You can do the retraining easily through [CellPose 2.0 GUI](https://www.nature.com/articles/s41592-022-01663-4). When the model gives acceptable results, select it in TrackMate interface with the `Path to custom model` parameter. 
 
+<!---
 Note that in 3D stacks, it can be worth it to create xy, yz and xz images of the stack corrected for anisotropy and to draw annotations on it to train the model to make it more performant for the segmentation with the `3D mode`.
-
-
+--->
 
 
 ## Tutorials
@@ -136,15 +142,21 @@ In the second panel, select the **cellpose detector**:
 
 then click the `Next` button. You should see the configuration panel for cellpose. 
 
-{% include img src="/media/plugins/trackmate/detectors/cellpose/trackmate-cellpose-tutorial-03" align='center' width='300' %}
+{% include img src="/media/plugins/trackmate/detectors/cellpose/trackmate-cellpose-ui-01.png" align='center' width='300' %}
 
-We want to segment the touching cells and obtain their contour. So we will use the **Cytoplasm** model. The cytoplasm is imaged in the second channel (it happens to use the green LUT, but it would not matter) so we enter **2** for the `Channel to segment` parameter. The channel **1** contains the nuclei so we can specify it in the `Optional second channel` parameter. Finally, we can measure a rough estimate of the cell size using ImageJ ROI tools, which is about **40 µm**, and enter this value in the `Cell diameter` field.
+On the `Conda environment` select the name of the environment Cellpose 3 is installed (**cellpose-3** in the example above).
+We want to segment the touching cells and obtain their contour. So we will use the **cyto3** model, to be selected in the `Pretrained model` list. 
+The cytoplasm is imaged in the second channel (it happens to use the green LUT, but it would not matter) so we enter **2** for the `Target channel` parameter. 
+The channel **1** contains the nuclei so we can specify it in the `Second optional channel` parameter. 
+Finally, we can measure a rough estimate of the cell size using ImageJ ROI tools, which is about **40 µm**, and enter this value in the `Cell diameter` field.
 
-The movie is quite big, and will take several minutes to process using only the CPU (on my Mac it takes about 20 minutes without multithreading). Having a cellpose installation with GPU support really accelerate segmentation, but this is not available on Mac. Instead, for  we developed TrackMate-cellpose so that on Mac, it can accelerate processing using multithreading. CPU multithreading is used only on Mac and if the `Use GPU` box is unchecked, which is why it is the case on the image above.
+The movie is quite big, and will take several minutes to process using only the CPU (on my Mac it takes about 20 minutes without multithreading). 
+Having a cellpose installation with GPU support really accelerate segmentation.
 
 Finally, for this movie we want the cell contour to follow the pixels of the masks generated by cellpose, so we unchecked the `Simplify contour` box.
 
-Then we are ready to launch segmentation. Click the `Next` button. The log window will echo the messages from cellpose. On a windows machine with GPU support, the detection step takes about 2 minutes. On a Mac with multithreading using 8 cores, it takes about 4 minutes.
+Then we are ready to launch segmentation. Click the `Next` button. The log window will echo the messages from cellpose. On a windows machine with GPU support, the detection step takes about 2 minutes. 
+On a 2022 MacBook Pro using GPU, it takes about 4 minutes.
 
 The **Initial thresholding** reports the quality histogram for all detections. Here, the quality is just the area of detections in pixel units. We see that there are some detections with very little size, probably for spurious objects. We can filter them out by setting a threshold value around 220.
 
@@ -174,7 +186,11 @@ Another key advantage of cellpose is that it is relatively easy and fast to trai
 
 {% include img src="/media/plugins/trackmate/detectors/cellpose/trackmate-cellpose-tutorial-b-01" align='center' width='400' %}
 
-For instance the cells above were imaged in bright-field at high resolution. They are Glioblastoma-astrocytoma U373 cells migrating on a polyacrylamide gel. They have a rather complex shape and a low contrast. The segmentation of cells in such images is normally difficult with classical image processing techniques. In the following we will use the trackMate-Cellpose intragration to segment and track them. Our goal is to see if we discriminate between two types of cell locomotion based on dynamics and morphological features of single cells. We will see how to give more robustness to the analysis by filtering out some detections close to image border.
+For instance the cells above were imaged in bright-field at high resolution. 
+They are Glioblastoma-astrocytoma U373 cells migrating on a polyacrylamide gel. 
+They have a rather complex shape and a low contrast. 
+The segmentation of cells in such images is normally difficult with classical image processing techniques. In the following we will use the TrackMate-Cellpose intragration to segment and track them. 
+Our goal is to see if we discriminate between two types of cell locomotion based on dynamics and morphological features of single cells. We will see how to give more robustness to the analysis by filtering out some detections close to image border.
 
 The data can be obtained from Zenodo:
 
@@ -183,18 +199,17 @@ The data can be obtained from Zenodo:
 The dataset contains a custom cellpose model. We have trained a it using the [ZeroCostDL4Mic platform](https://github.com/HenriquesLab/ZeroCostDL4Mic/wiki). This cellpose model was trained for 500 epochs on 214 paired image patches (image dimensions: 520x 696), with a batch size of 8, using the [Cellpose ZeroCostDL4Mic notebook (v 1.13)](https://colab.research.google.com/github/HenriquesLab/ZeroCostDL4Mic/blob/master/Colab_notebooks/Beta%20notebooks/Cellpose_2D_ZeroCostDL4Mic.ipynb). The cellpose `cyto2` model was used as a training starting point. The training was accelerated using a Tesla K80 GPU.
 
 To use the model you need to unzip the `Cellpose model 171121-20211118T111402Z-001.zip` file. The cellpose model file itself is the `cellpose_residual_on_style_on_concatenation_off_train_folder_2021_11_17_19_45_23.398850` file in the `171121` folder.
-We can use it in TrackMate by specifying "Custom" as a model in the interface:
+We can use it in TrackMate by checking the radio button in front of the `Path to a custom model` parameter:
 {% include img src="/media/plugins/trackmate/detectors/cellpose/trackmate-cellpose-tutorial-b-02" align='center' width='350' %}
 
-Here I am running TrackMate on a Windows machine, and I used the cellpose installation recommended by BIOP people ([see below](#biop-conda-installation-for-gpu-support-on-windows)). So in the `Path to cellpose / Python executable` I have: `C:\Users\tinevez\anaconda3\envs\cellpose_biop_gpu\python.exe` 
-
-As `Pretrained model` I picked **Custom** and in the `Path to custom model` text field I entered the path to the model file (in this case, ending in `.398850`).
+For the `Path to a custom model` parameter, I browsed to the model file (in this case, ending in `.398850`).
 
 There is only one channel, so I used **0** for both the target and optional channels. The cell size measured with ImageJ is about 60 µm.
 
-On this Windows machine and with this cellpose installation I have GPU support, so I left the corresponding box checked. Later we will measure shape descriptors for the cell, so it is important to simplify their contours, and the corresponding box is also checked. 
+On a 2022 MacBook Pro used in this demo, the Cellpose installation have GPU support, so I left the corresponding box checked. 
+Later we will measure shape descriptors for the cell, so it is important to simplify their contours, and the corresponding box is also checked. 
 
-Clicking `Next` starts the segmentation. In my case it completes in about 1 minute. In the **Initial thresholding** panel, choose a threshold of 0 to include all detections in the next step. We get the following results displayed in the **Spot filter** panel:
+Clicking `Next` starts the segmentation. In my case it completes in about 30 seconds. In the **Initial thresholding** panel, choose a threshold of 0 to include all detections in the next step. We get the following results displayed in the **Spot filter** panel:
 
 {% include img src="/media/plugins/trackmate/detectors/cellpose/trackmate-cellpose-tutorial-b-03" width='300' %}
 {% include img src="/media/plugins/trackmate/detectors/cellpose/trackmate-cellpose-tutorial-b-04" width='300' %}
@@ -236,17 +251,17 @@ Then proceed as before for the tracking step and the plot generation. The area a
 
 {% include img src="/media/plugins/trackmate/detectors/cellpose/trackmate-cellpose-tutorial-b-11" width='600'  align='center' %}
 
-
+<!-----------
 ### 3D segmentation with cellpose
 
 A tutorial in the documentation for the [advanced version of the cellpose detector](trackmate-cellpose-advanced) demonstrates how to use cellpose for 3D cell segmentation and tracking.
-
+-------------->
 
 ## Additional informations
 
-### Tip: Passing RGB images to TrackMate for cellpose
+### Tip: Passing RGB images to TrackMate for Cellpose
 
-On the cellpose webiste you can find a collection of test images to test with cellpose. They will work with the TrackMate integration as well, but they are RGB images. TrackMate does not support RGB images. So we give here a short optional procedure on how to feed RGB images to TrackMate and have them still segmented by cellpose as expected. Again, if you don't have RGB images as input, you can skip this section.
+On the Cellpose website you can find a collection of test images to test with cellpose. They will work with the TrackMate integration as well, but they are RGB images. TrackMate does not support RGB images. So we give here a short optional procedure on how to feed RGB images to TrackMate and have them still segmented by cellpose as expected. Again, if you don't have RGB images as input, you can skip this section.
 
 cellpose can and does work with RGB images. They are single-channel but encode red, green and blue components of each pixel in one value, effectively behaving as a 3-channels 8-bit image. However, TrackMate cannot deal with RGB images. If you launch TrackMate with an RGB image you will receive an error message. The workaround is to convert them to a proper 3-channels 8-bit image before running TrackMate. cellpose will be able to run with them anyway. Here is how to do it. 
 
@@ -264,13 +279,13 @@ cellpose can and does work with RGB images. They are single-channel but encode r
 {% include img src="/media/plugins/trackmate/detectors/cellpose/trackmate-cellpose-convert-rgb-04.png" %}
 {% include img src="/media/plugins/trackmate/detectors/cellpose/trackmate-cellpose-convert-rgb-05.png" %}
 
-### Installing cellpose
+### Installing Cellpose
 
 {% include notice icon="tech"
   content="This is the recommended way to install Python tools to be used with TrackMate." %}
 
   {% include notice icon="tech"
-  content="From now on we support **cellpose 3** in TrackMate. It is great, simple to install, and has GPU acceleration on all modern platforms." %}
+  content="This detector supports **Cellpose 3** in TrackMate. It is great, simple to install, and has GPU acceleration on all modern platforms. There is a separate detector for **Cellpose SAM**, linked above." %}
 
 You need to have conda (or mamba) installed on your system. 
 Any flavor (Anaconda, miniconda, miniforge, mamba, micromamba, ... ) works, but if you have to install one, we recommend [miniforge](https://github.com/conda-forge/miniforge).
@@ -324,4 +339,4 @@ torch version:          2.8.0+cu126
 
 _____
 
-*Jean-Yves Tinevez - August 2025*
+*Jean-Yves Tinevez - December 2025*