Segmentation¶

Segmentation describes the process of finding cells in an image. This can be done either on the nucleus or on whole cell level. Spatialproteomics provides wrappers for StarDist, mesmer and cellpose.

[1]:

%reload_ext autoreload
%autoreload 2

[2]:

import spatialproteomics
import pandas as pd
import matplotlib.pyplot as plt
import xarray as xr

[3]:

ds = xr.load_dataset('../../data/segmentation_example.zarr', engine='zarr').pp[1000:1300, 1000:1300].pp.drop_layers(keep='_image')

[4]:

ds

Whole-Cell Segmentation with Mesmer¶

You can also do the same thing with mesmer. Note however that mesmer expects the channels to be in the order [nuclear, membrane].

[10]:

# here, we add the segmentation directly to ds_mesmer instead of ds
# note that whenever we want to make a segmentation the "default" segmentation of an object, we need to call 'pp.add_segmentation()'
ds_mesmer = ds.pp[['DAPI', 'Na/K ATPase']].tl.mesmer().pp.add_segmentation('_mesmer_segmentation')
# plotting the result
_ = ds_mesmer.pl.colorize(['gold', 'deepskyblue']).pl.show(render_segmentation=True)

2024-09-17 08:03:16.779560: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /g/huber/users/meyerben/.conda/envs/tmp_env_2/lib/python3.10/site-packages/cv2/../../lib64:
2024-09-17 08:03:16.779592: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
Checking for cached data
Checking MultiplexSegmentation-9.tar.gz against provided file_hash...
MultiplexSegmentation-9.tar.gz with hash a1dfbce2594f927b9112f23a0a1739e0 already available.
Extracting /home/meyerben/.deepcell/models/MultiplexSegmentation-9.tar.gz
Successfully extracted /home/meyerben/.deepcell/models/MultiplexSegmentation-9.tar.gz into /home/meyerben/.deepcell/models
2024-09-17 08:04:04.578971: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /g/huber/users/meyerben/.conda/envs/tmp_env_2/lib/python3.10/site-packages/cv2/../../lib64:
2024-09-17 08:04:04.579002: W tensorflow/stream_executor/cuda/cuda_driver.cc:269] failed call to cuInit: UNKNOWN ERROR (303)
2024-09-17 08:04:04.579038: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (jupyter-meyerben): /proc/driver/nvidia/version does not exist
2024-09-17 08:04:04.619456: I tensorflow/core/platform/cpu_feature_guard.cc:151] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
No training configuration found in save file, so the model was *not* compiled. Compile it manually.
Converting image dtype to float

../_images/notebooks_Segmentation_12_1.png

Nuclear Segmentation with StarDist¶

Sometimes, you may not have a membrane stain available. In that case, you can perform nuclear segmentation. This is possible with cellpose (simply call tl.cellpose(channel='DAPI')) and StarDist. The following shows and example of how StarDist performs on such a task.

[12]:

# running the segmentation
ds_stardist = ds.tl.stardist(channel='DAPI', key_added='_stardist_segmentation')

Found model '2D_versatile_fluo' for 'StarDist2D'.
Loading network weights from 'weights_best.h5'.
Loading thresholds from 'thresholds.json'.
Using default values: prob_thresh=0.479071, nms_thresh=0.3.

100%|██████████| 144/144 [00:08<00:00, 17.28it/s]

Since it is possible to have multiple segmentations in a spatialproteomics object, we need to tell the package which segmentation to treat as the default one. This is done with pp.add_segmentation().

[13]:

# adding the segmentation as default segmentation
ds_stardist = ds_stardist.pp.add_segmentation('_stardist_segmentation')

[14]:

# plotting the segmentation
fix, ax = plt.subplots(1, 2, figsize=(16, 8))
_ = ds_stardist.pp['DAPI'].pl.colorize('orange').pl.show(ax=ax[0])
_ = ds_stardist.pp['DAPI'].pl.colorize('orange').pl.show(render_segmentation=True, ax=ax[1])

../_images/notebooks_Segmentation_17_0.png

You might have noticed that StarDist tends to segment a lot of small cells. Let’s look at the distribution of cell sizes, and then remove all nuclei under a certain size.

[15]:

# getting a histogram of the areas
_ = plt.hist(ds_stardist.pp.add_observations('area').pp.get_layer_as_df()['area'], bins=100)

Found _obs in image container. Concatenating.

../_images/notebooks_Segmentation_19_1.png

[16]:

# removing cells which have an area below 50 pixels
ds_stardist = ds_stardist.pp.add_observations('area').pp.filter_by_obs('area', lambda x: x > 50)

# plotting the segmentation
fix, ax = plt.subplots(1, 2, figsize=(16, 8))
_ = ds_stardist.pp['DAPI'].pl.colorize('orange').pl.show(ax=ax[0])
_ = ds_stardist.pp['DAPI'].pl.colorize('orange').pl.show(render_segmentation=True, ax=ax[1])

Found _obs in image container. Concatenating.

../_images/notebooks_Segmentation_20_1.png

To try to capture full cells, we can now grow the segmentation masks, e. g. by two pixels into each direction. If two masks collide, the masks will stop growing into that direction.

[17]:

ds_stardist_grown = ds_stardist.pp.grow_cells(2)

Mask growing requires recalculation of the observations. All features other than the centroids will be removed and should be recalculated with pp.add_observations().

[18]:

# plotting the segmentation
fix, ax = plt.subplots(1, 2, figsize=(16, 8))
_ = ds_stardist.pp['DAPI'].pl.colorize('orange').pl.show(render_segmentation=True, ax=ax[0])
_ = ds_stardist_grown.pp['DAPI'].pl.colorize('orange').pl.show(render_segmentation=True, ax=ax[1])
ax[0].set_title('StarDist segmentation')
ax[1].set_title('Grown masks')
plt.tight_layout()

../_images/notebooks_Segmentation_23_0.png

Subselecting Data Interactive Removal of Artifacts

spatialproteomics

Segmentation¶

Whole-Cell Segmentation with Cellpose¶

Whole-Cell Segmentation with Mesmer¶

Nuclear Segmentation with StarDist¶

Segmenting Multiple Channels with Cellpose¶