How to get SLIC visualisation using SimpleITK SLICImageFilter?

aliabidi · May 16, 2022, 6:59pm

Sorry if this is a basic question, I was just wondering how I could get a visualisation to show the result of a SLIC segmentation, such as:

I would like to generate a few visualisations like this using different SLIC parameters. My code for SLIC segmentation is simply as follows:

slic = sitk.SLICImageFilter()
slic.SetMaximumNumberOfIterations(self.slic_max_num_of_iterations)
slic.SetSuperGridSize(self.slic_super_grid_size)
slic.SetSpatialProximityWeight(self.slic_spatial_prox_weight)
slic.SetEnforceConnectivity(self.slic_enforce_connectivity)
slic.SetInitializationPerturbation(self.slic_initialization_perturbation)
seg = slic.Execute(denoised_img)

zivy · May 16, 2022, 7:11pm

Hello @aliabidi,

This is done via LabelMapContourOverlayImageFilter. For various overlays see this Jupyter notebook, specifically the section titled Overlay segmentation boundaries onto original image.

blowekamp · May 16, 2022, 7:16pm

The code used for the Insight Journal paper is here:

github.com

blowekamp/itkSuperPixel/blob/master/doc/scripts/evaluation_ski.py#L10-L16

      
        
            def mask_label_contour(image, seg):
               """Combine an image and segmentation by masking the segmentation contour.
            
            
   For an input image (scalar or vector), and a multi-label
               segmentation image, creates an output image where the countour of
               each label masks the input image to black."""
               return sitk.Mask(image, sitk.LabelContour(seg+1)==0)

aliabidi · June 5, 2022, 10:01pm

Unfortunately, I am unable to get the visualisations using this method. Please note that my images are 3D, but I do not mind just using one slice for this visualisation. I get the following error:

RuntimeError                              Traceback (most recent call last)
Input In [72], in <module>
      4 seg  = slic.Execute(denoised_img)
      6 overlay_filter = sitk.LabelMapContourOverlayImageFilter()
----> 7 overlay = overlay_filter.Execute(seg, image)

File /env/lib/python3.8/site-packages/SimpleITK/SimpleITK.py:34934, in LabelMapContourOverlayImageFilter.Execute(self, labelMapImage, featureImage)
  34925 def Execute(self, labelMapImage, featureImage):
  34926     r"""
  34927     Execute(LabelMapContourOverlayImageFilter self, Image labelMapImage, Image featureImage) -> Image
  34928 
   (...)
  34932 
  34933     """
> 34934     return _SimpleITK.LabelMapContourOverlayImageFilter_Execute(self, labelMapImage, featureImage)

RuntimeError: Exception thrown in SimpleITK LabelMapContourOverlayImageFilter_Execute: /tmp/SimpleITK/Code/Common/include/sitkDualMemberFunctionFactory.hxx:193:
sitk::ERROR: Pixel type: 32-bit unsigned integer is not supported in 3D byN3itk6simple33LabelMapContourOverlayImageFilterE

My code is as follows:

image = sitk.ReadImage("3d_t2_mri_scan.nii.gz")

slic = sitk.SLICImageFilter()
seg  = slic.Execute(denoised_img)

overlay_filter = sitk.LabelMapContourOverlayImageFilter()
overlay = overlay_filter.Execute(seg, image)

mauigna06 · June 6, 2022, 8:33am

You just need to use the cast filter first, as your new input for your intended filter
https://simpleitk.org/doxygen/latest/html/classitk_1_1simple_1_1CastImageFilter.html

aliabidi · June 6, 2022, 11:38am

Hi, I have tried this but I can’t find out which type of image I should cast to?

dzenanz · June 6, 2022, 11:54am

Probably unsigned char, or possibly unsigned short. If cast clips intensities, you should consider rescale intensity.

mauigna06 · June 6, 2022, 12:17pm

You should cast to float

Get Outlook para Android

blowekamp · June 6, 2022, 2:05pm

The LabelMapContourOverlayImageFilter works with the “LabelMap” Pixel types. The seg image would need to be cast with something like sitk.Cast(seg, sitk.sitkLabelUInt16).

aliabidi · June 6, 2022, 3:57pm

Unfortunately I get the same error with casting to any of: sitkLabelUInt8, sitkLabelUInt16, sitkLabelUInt32, sitkLabelUInt64. Depending on which one:

sitk::ERROR: Pixel type: label of 64-bit unsigned integer is not supported in 3D byN3itk6simple33LabelMapContourOverlayImageFilterE

I am doing:

image = sitk.Cast(image, sitk.sitkLabelUInt64)
seg = sitk.Cast(seg, sitk.sitkLabelUInt64)

aliabidi · June 6, 2022, 3:58pm

I get the same error for sitkFloat32 and sitkFloat64:

RuntimeError: Exception thrown in SimpleITK LabelMapContourOverlayImageFilter_Execute: /tmp/SimpleITK/Code/Common/include/sitkDualMemberFunctionFactory.hxx:193:
sitk::ERROR: Pixel type: 64-bit float is not supported in 3D byN3itk6simple33LabelMapContourOverlayImageFilterE

aliabidi · June 6, 2022, 4:07pm

Unfortunately, I have tried all of the pixel types on the list for SimpleITK (see here), and none of them work. I get the same error:

RuntimeError: Exception thrown in SimpleITK LabelMapContourOverlayImageFilter_Execute: /tmp/SimpleITK/Code/Common/include/sitkDualMemberFunctionFactory.hxx:193:
sitk::ERROR: Pixel type: [the pixel type] is not supported in 3D byN3itk6simple33LabelMapContourOverlayImageFilterE

What could I be doing wrong? My full code is simply, for sitkLabelUInt64, for example:

image = sitk.ReadImage("3D_T2_MRI_scan.nii.gz")

slic = sitk.SLICImageFilter()
seg  = slic.Execute(denoised_img)

image = sitk.Cast(image, sitk.sitkLabelUInt64)
seg = sitk.Cast(seg, sitk.sitkLabelUInt64)

overlay_filter = sitk.LabelMapContourOverlayImageFilter()
overlay = overlay_filter.Execute(seg, image)

The properties for image are (before casting):

Image (0x18f50190)
  RTTI typeinfo:   itk::Image<short, 3u>
  Reference Count: 1
  Modified Time: 12110844
  Debug: Off
  Object Name: 
  Observers: 
    none
  Source: (none)
  Source output name: (none)
  Release Data: Off
  Data Released: False
  Global Release Data: Off
  PipelineMTime: 12110821
  UpdateMTime: 12110843
  RealTimeStamp: 0 seconds 
  LargestPossibleRegion: 
    Dimension: 3
    Index: [0, 0, 0]
    Size: [250, 138, 30]
  BufferedRegion: 
    Dimension: 3
    Index: [0, 0, 0]
    Size: [250, 138, 30]
  RequestedRegion: 
    Dimension: 3
    Index: [0, 0, 0]
    Size: [250, 138, 30]
  Spacing: [0.791016, 0.791016, 3.85]
  Origin: [-85.1901, -66.4408, 31.3689]
  Direction: 
0.99998 5.66689e-06 0.00628384
-0.00628174 -0.0249818 0.999668
0.000162647 -0.999688 -0.0249813

  IndexToPointMatrix: 
0.791 4.4826e-06 0.0241928
-0.00496895 -0.019761 3.84872
0.000128656 -0.790769 -0.0961781

  PointToIndexMatrix: 
1.26417 -0.00794136 0.000205618
7.16407e-06 -0.031582 -1.2638
0.00163217 0.259654 -0.00648865

  Inverse Direction: 
0.99998 -0.00628174 0.000162647
5.66689e-06 -0.0249818 -0.999688
0.00628384 0.999668 -0.0249813

  PixelContainer: 
    ImportImageContainer (0xa468a20)
      RTTI typeinfo:   itk::ImportImageContainer<unsigned long, short>
      Reference Count: 1
      Modified Time: 12110840
      Debug: Off
      Object Name: 
      Observers: 
        none
      Pointer: 0x2059ebb0
      Container manages memory: true
      Size: 1035000
      Capacity: 1035000

And for seg (before casting):

Image (0xbb15780)
  RTTI typeinfo:   itk::Image<unsigned int, 3u>
  Reference Count: 2
  Modified Time: 12111150
  Debug: Off
  Object Name: 
  Observers: 
    none
  Source: (0x6010380) 
  Source output name: Primary
  Release Data: Off
  Data Released: False
  Global Release Data: Off
  PipelineMTime: 12111083
  UpdateMTime: 12111151
  RealTimeStamp: 0 seconds 
  LargestPossibleRegion: 
    Dimension: 3
    Index: [0, 0, 0]
    Size: [250, 138, 30]
  BufferedRegion: 
    Dimension: 3
    Index: [0, 0, 0]
    Size: [250, 138, 30]
  RequestedRegion: 
    Dimension: 3
    Index: [0, 0, 0]
    Size: [250, 138, 30]
  Spacing: [0.791016, 0.791016, 3.85]
  Origin: [-85.1901, -66.4408, 31.3689]
  Direction: 
0.99998 5.66689e-06 0.00628384
-0.00628174 -0.0249818 0.999668
0.000162647 -0.999688 -0.0249813

  IndexToPointMatrix: 
0.791 4.4826e-06 0.0241928
-0.00496895 -0.019761 3.84872
0.000128656 -0.790769 -0.0961781

  PointToIndexMatrix: 
1.26417 -0.00794136 0.000205618
7.16407e-06 -0.031582 -1.2638
0.00163217 0.259654 -0.00648865

  Inverse Direction: 
0.99998 -0.00628174 0.000162647
5.66689e-06 -0.0249818 -0.999688
0.00628384 0.999668 -0.0249813

  PixelContainer: 
    ImportImageContainer (0x17114ea0)
      RTTI typeinfo:   itk::ImportImageContainer<unsigned long, unsigned int>
      Reference Count: 1
      Modified Time: 12111092
      Debug: Off
      Object Name: 
      Observers: 
        none
      Pointer: 0xd5164f0
      Container manages memory: true
      Size: 1035000
      Capacity: 1035000

I am very confused. The image I’m using is simply a cropped 3D T2 MRI file. It is not corrupted, as I am able to open and view it clearly, and apply filters on it.

blowekamp · June 6, 2022, 4:38pm

Only Cast seg to a label map type.

aliabidi · June 6, 2022, 5:43pm

This works, thank you. @blowekamp

image = sitk.ReadImage("3D_T2_MRI_scan.nii.gz")

slic = sitk.SLICImageFilter()
seg  = slic.Execute(image)

seg = sitk.Cast(seg, sitk.sitkLabelUInt8)

overlay_filter = sitk.LabelMapContourOverlayImageFilter()
overlay = overlay_filter.Execute(seg, image)

aliabidi · June 6, 2022, 5:50pm

I would also like to ask how to save the colours and the overlayed image locally. In Jupyter Notebook, I see the following, using this code:

overlay = sitk.LabelMapContourOverlay(
    seg,
    image,
    opacity=0.75,
    contourThickness=[1]*3,
    dilationRadius=[3]*3,
    colormap=red+blue+green,
)

gui.MultiImageDisplay(
    image_list=[overlay]
)

However, when I save the file by doing:

sitk.WriteImage(overlay, "overlay.nii.gz")

I see the following:

Where the overlayed MRI scan, and the colours, are not present. This is using ITK-SNAP. How could I retain the overlayed MRI scan and colours I see in Jupyter Notebook?

zivy · June 6, 2022, 5:53pm

Hello @aliabidi,

I believe that’s an ITK-SNAP constraint, doesn’t do color, open the image in Slicer.