How to use ITKMontage to stitch segmentation label images?

Hi,

I’m trying to use ITKMontage’s TileMergeImageFilter to stitch segmentation label images (uint8) using the transformations obtained from grayscale images.

The file below shows at its header that the default interpolator is LinearInterpolateImageFunction.
itkTileMergeImageFilter.h
When using the default linear interpolator, the intermediate values between different classes is mistakenly interpolated to other label values.
image

Is it possible to set it to NearestNeighborInterpolateImageFunction without rebuilding itk from source?

Below is a code with the issued error when trying to set the interpolator at the constructor:
Code snippet:

label = itk.imread(filename, itk.US)  # read as grayscale
label.SetOrigin(origin)
labels.append(label)
...
    resampleF = itk.TileMergeImageFilter[type(labels[0]),\
                itk.NearestNeighborInterpolateImageFunction[type(labels[0])].New(labels[0])].New()
    resampleF.SetMontageSize(stage_tiles.GetAxisSizes())
    for t in range(stage_tiles.LinearSize()):
        resampleF.SetInputTile(t, labels[t])
        index = stage_tiles.LinearIndexToNDIndex(t)
        resampleF.SetTileTransform(index, montage.GetOutputTransform(index))
    resampleF.Update()
    itk.imwrite(resampleF.GetOutput(), os.path.join(tgt, f'stitched_{k}_label.mhd'))

Error log:

>    itk.NearestNeighborInterpolateImageFunction[type(labels[0])].New(labels[0])].New()
  File "py\lib\site-packages\itk\support\template_class.py", line 529, in __getitem__
    raise itk.TemplateTypeError(self, key)
itk.support.extras.TemplateTypeError: itk.NearestNeighborInterpolateImageFunction is not wrapped for input type `itk.Image[itk.UC,3]`.

To limit the size of the package, only a limited number of
types are available in ITK Python. To print the supported
types, run the following command in your python environment:

    itk.NearestNeighborInterpolateImageFunction.GetTypes()

Possible solutions:
* If you are an application user:
** Convert your input image into a supported format (see below).
** Contact developer to report the issue.
* If you are an application developer, force input images to be
loaded in a supported pixel type.

    e.g.: instance = itk.NearestNeighborInterpolateImageFunction[itk.Image[itk.SS,2], itk.D].New(my_input)

* (Advanced) If you are an application developer, build ITK Python yourself and
turned to `ON` the corresponding CMake option to wrap the pixel type or image
dimension you need. When configuring ITK with CMake, you can set
`ITK_WRAP_${type}` (replace ${type} with appropriate pixel type such as
`double`). If you need to support images with 4 or 5 dimensions, you can add
these dimensions to the list of dimensions in the CMake variable
`ITK_WRAP_IMAGE_DIMS`.

Supported input types:

itk.Image[itk.SS,2]
itk.Image[itk.UC,2]
itk.Image[itk.US,2]
itk.Image[itk.F,2]
itk.Image[itk.D,2]
itk.Image[itk.SS,3]
itk.Image[itk.UC,3]
itk.Image[itk.US,3]
itk.Image[itk.F,3]
itk.Image[itk.D,3]
itk.Image[itk.SS,4]
itk.Image[itk.UC,4]
itk.Image[itk.US,4]
itk.Image[itk.F,4]
itk.Image[itk.D,4]

Thankful for your help in advance,

Try replacing

itk.NearestNeighborInterpolateImageFunction[type(labels[0])]

by:

itk.NearestNeighborInterpolateImageFunction[type(labels[0]), itk.D]

@dzenanz thanks a lot for reply.

I replaced it as:

   resampleF = itk.TileMergeImageFilter[type(labels[0]),\
                itk.NearestNeighborInterpolateImageFunction[type(labels[0]), itk.D].New(labels[0])].New()

But it gave the error:

  File "\py\lib\site-packages\itk\support\template_class.py", line 529, in __getitem__
    raise itk.TemplateTypeError(self, key)
itk.support.extras.TemplateTypeError: itk.TileMergeImageFilter is not wrapped for input type `itk.Image[itk.US,3], itk.NearestNeighborInterpolateImageFunction[itk.Image[itk.US,3],itk.D]`.

To limit the size of the package, only a limited number of
types are available in ITK Python. To print the supported
types, run the following command in your python environment:

    itk.TileMergeImageFilter.GetTypes()

Possible solutions:
* If you are an application user:
** Convert your input image into a supported format (see below).
** Contact developer to report the issue.
* If you are an application developer, force input images to be
loaded in a supported pixel type.

    e.g.: instance = itk.TileMergeImageFilter[itk.Image[itk.SS,2]].New(my_input)

* (Advanced) If you are an application developer, build ITK Python yourself and
turned to `ON` the corresponding CMake option to wrap the pixel type or image
dimension you need. When configuring ITK with CMake, you can set
`ITK_WRAP_${type}` (replace ${type} with appropriate pixel type such as
`double`). If you need to support images with 4 or 5 dimensions, you can add
these dimensions to the list of dimensions in the CMake variable
`ITK_WRAP_IMAGE_DIMS`.

Supported input types:

itk.Image[itk.SS,2]
itk.Image[itk.UC,2]
itk.Image[itk.US,2]
itk.Image[itk.F,2]
itk.Image[itk.D,2]
itk.Image[itk.RGBPixel[itk.UC],2]
itk.Image[itk.RGBAPixel[itk.UC],2]
itk.Image[itk.SS,3]
itk.Image[itk.UC,3]
itk.Image[itk.US,3]
itk.Image[itk.F,3]
itk.Image[itk.D,3]
itk.Image[itk.RGBPixel[itk.UC],3]
itk.Image[itk.RGBAPixel[itk.UC],3]
itk.Image[itk.SS,4]
itk.Image[itk.UC,4]
itk.Image[itk.US,4]
itk.Image[itk.F,4]
itk.Image[itk.D,4]
itk.Image[itk.RGBPixel[itk.UC],4]
itk.Image[itk.RGBAPixel[itk.UC],4]

Any ideas to solve it?

Thanks again for yur help,

Looking at the wrapping specification, only the default (linear) interpolator is wrapped. Can you add scalar+nearest instantiations there, and propose a PR?

Thanks for the proposal. Sure, I will be glad to contribute, but sorry I’m new to ITK development.
Could you please explain in more detail which lines I need to add/modify?

It would be something similar to this. I don’t have time to fix/test this. Feel free to use as a starting point for your own experiments.

Thanks a lot! Will try to use it to fix the issue.

If you make any progress, do share it via a PR.

I updated PR 227 and it now compiles locally, and passes tests:

100% tests passed, 0 tests failed out of 68

Label Time Summary:
Montage    = 289.73 sec*proc (68 tests)