Help displaying a 3D Dicom and the corresponding segmentation Slicewise using itkwidgets

Kashyap · October 2, 2020, 9:32pm

I am able to display the 3D Dicom read using simple ITK and also overlay the corresponding segmentation mesh in vtkpolydata format.

Now I am trying to display the same image slice wise (kind of like an extension of the example in the examples/2DImage.ipynb). As a bonus, it would be great if I could visualize the corresponding mesh on those slices.

Is it possible to do something like this? And would itkwidgets be the right tool? I really like the convenience of quickly visualizing these images within the notebook rather than writing out both the image and segmentation meshes.

Another issue perhaps related is that whenever I use the view() as part of the loop, it does not display anything. I have to use view() in a separate cell. Is that by design or is it a bug?

Any guidance would be appreciated.

dzenanz · October 2, 2020, 9:46pm

@matt.mccormick can best answer this.

zivy · October 2, 2020, 10:09pm

Hello @Kashyap,

If you are working in SimpleITK and have the segmentation as a label map there are several ways of overlaying it onto the original image. For the various options see this Jupyter notebook.

lassoan · October 3, 2020, 3:45pm

@zivy Could you add a “Launch binder” button to each notebook in SimpleITK-Notebooks. Now it takes several clicks to go back to the main page, find the binder link, launch it, wait for it to start, and find the page that you originally wanted to run - and probably many people don’t know that you can quickly try things in binder.

@Kashyap You may also try 3D Slicer’s Jupyter kernel for more advanced interactive visualization of medical images in notebooks. Latest version of SimpleITK, VTK, DICOM toolkits, etc. are already bundled. You can do the usual volume slicing but also volume rendering, visualization of surface meshes, and can interactively add/edit landmarks, curves, and other markups, do interactive segmentation, registration, etc. (all features of 3D Slicer application and its extensions are available). You can give it a quick try on binder or you can use it on your computer.

zivy · October 3, 2020, 8:07pm

@lassoan,
Good suggestion. Will add to all notebooks.

Kashyap · October 5, 2020, 2:16pm

@zivy and @lassoan Thanks for the resources. Both look great. I will try out both these options.

matt.mccormick · December 15, 2020, 10:01pm

@Kashyap you can display 3D images with an overlay of a segmentation mesh with itkwidgets. Pass the vtk.PolyData or itk.Mesh as the geometries keyword argument. Here is an example:

github.com

InsightSoftwareConsortium/itkwidgets/blob/master/examples/pyvista.PolyDataAndImageData.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Note: This example currently does not work in MyBinder.\n",
    "\n",
    "# Install dependencies for this example\n",
    "# Note: This does not include itkwidgets, itself\n",
    "import sys\n",
    "!{sys.executable} -m pip install pyvista"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},

This file has been truncated. show original

Another issue perhaps related is that whenever I use the view() as part of the loop, it does not display anything. I have to use view() in a separate cell. Is that by design or is it a bug?

When view(image) is called at the end of a cell, Jupyter is displaying the returned value. You can capture that value and tell Jupyter to display it:

from IPython.display import display

viewer = view(image)
display(viewer)

# [...] other code