How to `itk.resample_image_filter` with a `BSplineTransform` with 3D images?

Table of Contents

1. How to itk.resample_image_filter with a BSplineTransform with 3D images?

Hello, everyone!

I am trying to use parts of the DeformableRegistration15.cxx in Python, but I don’t know what to do when ResampleImageFilter is combined with GetCoefficientImages (from a BSplineTransform).

1.1. What I have tried

1.1.1. Inputs (data)

Listing 1: Block to print the input to ResampleImageFilter and resample_image_filter.
print("moving_image_type:\t", moving_image_type)
print("coord_type:\t", moving_image_type)
print("identity_transform:\t", identity_transform)
print("bspline_transform_coarse\t", bspline_transform_coarse)
print("bspline_transform_fine\t", bspline_transform_fine)
interpolator_fine = itk.BSplineResampleImageFunction[
    moving_image_type, coord_type].New()
moving_image_type:	 <class 'itk.itkImagePython.itkImageF3'>
coord_type:	 <class 'itk.itkImagePython.itkImageF3'>
identity_transform:	 IdentityTransform (0x55a08d43e540)
  RTTI typeinfo:   itk::IdentityTransform<double, 3u>
  Reference Count: 1
  Modified Time: 127
  Debug: Off
  Object Name:
  Observers:
    none

bspline_transform_coarse	 BSplineTransform (0x55a08c2b42b0)
  RTTI typeinfo:   itk::BSplineTransform<double, 3u, 3u>
  Reference Count: 3
  Modified Time: 14797
  Debug: Off
  Object Name:
  Observers:
    none
  CoefficientImage: [ 0x55a08cd32630, 0x55a08cd328f0, 0x55a08cade1e0 ]
  TransformDomainOrigin: [0, 0, 0]
  TransformDomainPhysicalDimensions: [99, 99, 9]
  TransformDomainDirection: 1 0 0
0 1 0
0 0 1

  TransformDomainMeshSize: [2, 2, 2]
  GridSize: [5, 5, 5]
  GridOrigin: [-49.5, -49.5, -4.5]
  GridSpacing: [49.5, 49.5, 4.5]
  GridDirection: 1 0 0
0 1 0
0 0 1


bspline_transform_fine	 BSplineTransform (0x55a08f8fa480)
  RTTI typeinfo:   itk::BSplineTransform<double, 3u, 3u>
  Reference Count: 1
  Modified Time: 14848
  Debug: Off
  Object Name:
  Observers:
    none
  CoefficientImage: [ 0x55a08f198a00, 0x55a08f198cc0, 0x55a08f198f80 ]
  TransformDomainOrigin: [0, 0, 0]
  TransformDomainPhysicalDimensions: [99, 99, 9]
  TransformDomainDirection: 1 0 0
0 1 0
0 0 1

  TransformDomainMeshSize: [2, 2, 2]
  GridSize: [5, 5, 5]
  GridOrigin: [-49.5, -49.5, -4.5]
  GridSpacing: [49.5, 49.5, 4.5]
  GridDirection: 1 0 0
0 1 0
0 0 1

1.1.2. Failed attempt with resample_image_filter

Listing 2: This block fails probably because I don’t know how to provide the GetCoefficientImages.
# https://examples.itk.org/src/filtering/imagegrid/resampleavectorimage/documentation
upsampler = itk.resample_image_filter(
    bspline_transform_coarse.GetCoefficientImages(),
    interpolator=interpolator_fine,
    transform=identity_transform,
    size=bspline_transform_fine.GetCoefficientImages())
swig/python detected a memory leak of type 'itk::FixedArray< itk::SmartPointer< itk::Image< double,3 > >,3 > *', no destructor found.
swig/python detected a memory leak of type 'itk::FixedArray< itk::SmartPointer< itk::Image< double,3 > >,3 > *', no destructor found.
Traceback (most recent call last):
  File "<string>", line 17, in __PYTHON_EL_eval
  File "<string>", line 2, in <module>
  File "/usr/lib/python3.10/site-packages/itk/support/helpers.py", line 176, in image_filter_wrapper
    return image_filter(*args, **kwargs)
  File "/usr/lib/python3.10/site-packages/itk/itkResampleImageFilterPython.py", line 5542, in resample_image_filter
    instance = itk.ResampleImageFilter.New(*args, **kwargs)
  File "/usr/lib/python3.10/site-packages/itk/support/template_class.py", line 734, in New
    raise itk.TemplateTypeError(self, input_type)
itk.support.extras.TemplateTypeError: itk.ResampleImageFilter is not wrapped for input type `None`.

To limit the size of the package, only a limited number of
types are available in ITK Python. To print the supported
types, run the following command in your python environment:

    itk.ResampleImageFilter.GetTypes()

Possible solutions:
* If you are an application user:
** Convert your input image into a supported format (see below).
** Contact developer to report the issue.
* If you are an application developer, force input images to be
loaded in a supported pixel type.

    e.g.: instance = itk.ResampleImageFilter[itk.Image[itk.SC,2], itk.Image[itk.SC,2]].New(my_input)

* (Advanced) If you are an application developer, build ITK Python yourself and
turned to `ON` the corresponding CMake option to wrap the pixel type or image
dimension you need. When configuring ITK with CMake, you can set
`ITK_WRAP_${type}` (replace ${type} with appropriate pixel type such as
`double`). If you need to support images with 4 or 5 dimensions, you can add
these dimensions to the list of dimensions in the CMake variable
`ITK_WRAP_IMAGE_DIMS`.

Supported input types:

itk.Image[itk.SC,2]
itk.Image[itk.SC,3]
itk.Image[itk.SC,4]
itk.Image[itk.SS,2]
itk.Image[itk.SS,3]
itk.Image[itk.SS,4]
itk.Image[itk.UC,2]
itk.Image[itk.UC,3]
itk.Image[itk.UC,4]
itk.Image[itk.US,2]
itk.Image[itk.US,3]
itk.Image[itk.US,4]
itk.Image[itk.F,2]
itk.Image[itk.F,3]
itk.Image[itk.F,4]
itk.Image[itk.D,2]
itk.Image[itk.D,3]
itk.Image[itk.D,4]
itk.Image[itk.Vector[itk.D,2],2]
itk.Image[itk.Vector[itk.D,2],3]
itk.Image[itk.Vector[itk.D,2],4]
itk.Image[itk.Vector[itk.D,3],2]
itk.Image[itk.Vector[itk.D,3],3]
itk.Image[itk.Vector[itk.D,3],4]
itk.Image[itk.Vector[itk.D,4],2]
itk.Image[itk.Vector[itk.D,4],3]
itk.Image[itk.Vector[itk.D,4],4]
itk.Image[itk.Vector[itk.F,2],2]
itk.Image[itk.Vector[itk.F,2],3]
itk.Image[itk.Vector[itk.F,2],4]
itk.Image[itk.Vector[itk.F,3],2]
itk.Image[itk.Vector[itk.F,3],3]
itk.Image[itk.Vector[itk.F,3],4]
itk.Image[itk.Vector[itk.F,4],2]
itk.Image[itk.Vector[itk.F,4],3]
itk.Image[itk.Vector[itk.F,4],4]
itk.Image[itk.RGBPixel[itk.UC],2]
itk.Image[itk.RGBPixel[itk.UC],3]
itk.Image[itk.RGBPixel[itk.UC],4]
itk.Image[itk.RGBPixel[itk.US],2]
itk.Image[itk.RGBPixel[itk.US],3]
itk.Image[itk.RGBPixel[itk.US],4]
itk.Image[itk.RGBAPixel[itk.UC],2]
itk.Image[itk.RGBAPixel[itk.UC],3]
itk.Image[itk.RGBAPixel[itk.UC],4]
itk.Image[itk.RGBAPixel[itk.US],2]
itk.Image[itk.RGBAPixel[itk.US],3]
itk.Image[itk.RGBAPixel[itk.US],4]
itk.VectorImage[itk.UC,2]
itk.VectorImage[itk.SC,2]
itk.VectorImage[itk.SS,2]
itk.VectorImage[itk.US,2]
itk.VectorImage[itk.F,2]
itk.VectorImage[itk.D,2]
itk.VectorImage[itk.UC,3]
itk.VectorImage[itk.SC,3]
itk.VectorImage[itk.SS,3]
itk.VectorImage[itk.US,3]
itk.VectorImage[itk.F,3]
itk.VectorImage[itk.D,3]
itk.VectorImage[itk.UC,4]
itk.VectorImage[itk.SC,4]
itk.VectorImage[itk.SS,4]
itk.VectorImage[itk.US,4]
itk.VectorImage[itk.F,4]
itk.VectorImage[itk.D,4]

1.1.3. Useless result with ResampleImageFilter

Listing 3: This block works.
# https://examples.itk.org/src/filtering/imagegrid/resampleavectorimage/documentation
upsampler = itk.ResampleImageFilter[moving_image_type, moving_image_type].New()
print(upsampler)
ResampleImageFilter (0x55a08bde5270)
  RTTI typeinfo:   itk::ResampleImageFilter<itk::Image<float, 3u>, itk::Image<float, 3u>, double, double>
  Reference Count: 1
  Modified Time: 15126
  Debug: Off
  Object Name:
  Observers:
    none
  Inputs:
    Primary: (0) *
    ReferenceImage: (0)
    Transform: (0x55a08ffa49c0) *
  Indexed Inputs:
    0: Primary (0)
    1: ReferenceImage (0)
  Required Input Names: Primary, Transform
  NumberOfRequiredInputs: 1
  Outputs:
    Primary: (0x55a08ee2f840)
  Indexed Outputs:
    0: Primary (0x55a08ee2f840)
  NumberOfRequiredOutputs: 1
  Number Of Work Units: 32
  ReleaseDataFlag: Off
  ReleaseDataBeforeUpdateFlag: Off
  AbortGenerateData: Off
  Progress: 0
  Multithreader:
    RTTI typeinfo:   itk::PoolMultiThreader
    Reference Count: 1
    Modified Time: 15106
    Debug: Off
    Object Name:
    Observers:
      none
    Number of Work Units: 32
    Number of Threads: 8
    Global Maximum Number Of Threads: 128
    Global Default Number Of Threads: 8
    Global Default Threader Type: itk::MultiThreaderBaseEnums::Threader::Pool
    SingleMethod: 0
    SingleData: 0
  DynamicMultiThreading: On
  CoordinateTolerance: 1e-06
  DirectionTolerance: 1e-06
  DefaultPixelValue: 0
  Size: [0, 0, 0]
  OutputStartIndex: [0, 0, 0]
  OutputSpacing: [1, 1, 1]
  OutputOrigin: [0, 0, 0]
  OutputDirection: 1 0 0
0 1 0
0 0 1

  Transform: 0x55a08f6d3800
  Interpolator: 0x55a08fd5e4b0
  Extrapolator: 0
  UseReferenceImage: Off

1.1.4. Error with ResampleImageFilter and GetCoefficientImages()

Listing 4: This block will not work, because of my wrong call to GetCoefficientImages.
# https://examples.itk.org/src/filtering/imagegrid/resampleavectorimage/documentation
upsampler = itk.ResampleImageFilter[moving_image_type, moving_image_type].New(
    Input=bspline_transform_coarse.GetCoefficientImages(),
    Interpolator=interpolator_fine,
    Transform=identity_transform,
    Size=bspline_transform_fine.GetCoefficientImages())
Traceback (most recent call last):
  File "<string>", line 17, in __PYTHON_EL_eval
  File "<string>", line 2, in <module>
  File "/usr/lib/python3.10/site-packages/itk/itkResampleImageFilterPython.py", line 516, in New
    template_class.New(obj, *args, **kargs)
  File "/usr/lib/python3.10/site-packages/itk/support/template_class.py", line 800, in New
    itk.set_inputs(self, args, kargs)
  File "/usr/lib/python3.10/site-packages/itk/support/extras.py", line 1543, in set_inputs
    attrib(itk.output(value))
TypeError: Expecting argument of type itkImageF3 or itkImageSourceIF3.
Additional information:
Wrong number or type of arguments for overloaded function 'itkImageToImageFilterIF3IF3_SetInput'.
  Possible C/C++ prototypes are:
    itkImageToImageFilterIF3IF3::SetInput(itkImageF3 const *)
    itkImageToImageFilterIF3IF3::SetInput(unsigned int,itkImageF3 const *)

1.1.5. Error with ResampleImageFilter and GetCoefficientImages()[0]

Listing 5: This block will not work, because of my wrong call to GetCoefficientImages (or trying to access an inexisting element).
# https://examples.itk.org/src/filtering/imagegrid/resampleavectorimage/documentation
upsampler = itk.ResampleImageFilter[moving_image_type, moving_image_type].New(
    Input=bspline_transform_coarse.GetCoefficientImages()[0],
    Interpolator=interpolator_fine,
    Transform=identity_transform,
    Size=bspline_transform_fine.GetCoefficientImages())
swig/python detected a memory leak of type 'itk::FixedArray< itk::SmartPointer< itk::Image< double,3 > >,3 > *', no destructor found.
Traceback (most recent call last):
  File "<string>", line 17, in __PYTHON_EL_eval
  File "<string>", line 3, in <module>
TypeError: 'SwigPyObject' object is not subscriptable

1.2. Software

import itk
import sys
print("Self-compiled version of ITK:\t", itk.__version__)
print("Python version:\t", sys.version)
Self-compiled version of ITK:	 5.3.0
Python version:	 3.10.4 (main, Mar 23 2022, 23:05:40) [GCC 11.2.0]

Hi @edgar ,

To resample an image with itk.resample_image_filter and a b-spline transform, pass the transform in as the transform argument.

For the input, use the moving image. The sampling grid can be specified with a reference image, e.g. the fixed image, or by the parameters of the sampling gride, including the ‘size’.

HTH,
Matt

Hi and thanks, @matt.mccormick. Sorry for the wall of text. In DeformableRegistration15.cxx, GetCoefficientImages()[k] is used within a loop to provide the input image. How can I access that image in Python? I get an error if I try (see above). Thanks!