Plot TRE per iteration for 2D registration

Algorithms
, , ,
1

Hello, I’m trying to plot target registration error (TRE) values at each iteration as shown in this notebook in the Registration cell. The input images are 3D and 2D, where the 3D image is sliced into 2D. They are transformed using the affine transformation (specified in centered transform initialization as sitk.AffineTransform(2)). The registration method has the following Command callbacks added to get the TRE at each iteration, specifically metric_and_reference_plot_values and its start and stop variants.

I’m more than happy with the registration result, however, I can’t get the TREs due to the following error:

sitk::ERROR: Transform  argument has dimension 3 does not match this dimension of 2

and occurs at this part of the above mention callbacks:

# Compute and store TRE statistics (mean, min, max).
    current_transform = sitk.CompositeTransform(registration_method.GetInitialTransform())
    current_transform.SetParameters(registration_method.GetOptimizerPosition())
    current_transform.AddTransform(registration_method.GetMovingInitialTransform()) <--- error at this line
    current_transform.AddTransform(registration_method.GetFixedInitialTransform().GetInverse())

It is caused by the different dimensions of InitialTransform (dim = 2), MovingInitialTransform (dim = 3) and is also likely to occur when adding the inverse of FixedInitialTransform (also dim = 3).

Is there a way to change the above lines, in the initialization transform, add a dimension to the 2D fixed image or change the dimension of transforms in order to get the TREs values and plot them based on iteration?

So far I’ve tried various ways of reshaping the fixed 2D image into a 3D array, but that throws this error when executing the registration:

ITK ERROR: SmoothingRecursiveGaussianImageFilter(000001D511A21330): The number of pixels along dimension 2 is less than 4. This filter requires a minimum of four pixels along the dimension to be processed.

The expand filter also doesn’t work since the input image is doesn’t have a third dimension. Either the reshaping is done wrong or I’m missing something. I think a more likely reason, is that the newly created image doesn’t have the relevant pixel information or something.

Thanks in advance

2

Hello @Vojtech_Benda,

It is unclear how there are 2D and 3D transformations mixed in the same registration. ITK/SimpleITK only supports registration of images with the same spatial dimensionality, 2D/2D or 3D/3D registration. You can “cheat” and do 2D/3D registration by making the 2D image a 3D one with a single slice but the transformations are then all 3D. Not sure what you mean by “3D image is sliced into 2D”, please clarify the registration configuration that is being used so that we can help resolve the issue.

3

Yes, I should’ve clarified the setup for sitk.ImageRegistrationMethod() and the input images. The fixed image is 2D and I can’t change as it is created from using gVirtualXray package. The moving image is 3D and created using the ITK’s ray casting interpolator. In order to do a 2D affine transformation, I’m extracting the moving image into 2D using slicing:

moving_image_2d = moving_image_3d[:, :, 0]

The first error occurs when trying to register these images and plot TRE at the same time, using the above mentioned notebook. It is caused by the registration’s MovingInitialTransform and FixedInitialTransform which all have dimension of 3:

registration_method.GetMovingInitialTransform().GetDimension() == 3
registration_method.GetFixedInitialTransform().GetDimension() == 3

I don’t know why it is like this. I believe the most straightforward solution is to add depth to the fixed image: 2D → 3D. However, I wasn’t successful. I tried making a numpy array from the fixed image, reshape it into a 3D array and create sitk image from that array, but that didn’t work though. Is it possible to make the fixed 2D image into a 3D image using SimpleITK’s functionality? I’m certain I’ve missed something, but I’m not sure what.

4

Hello @Vojtech_Benda,

The notebook you are referencing deals with 3D images and thus uses 3D transformations, Euler3DTransform. As you want to do a 2D/2D affine registration you need to switch to AffineTransform and create it in 2D, something like:

initial_transform = sitk.CenteredTransformInitializer(
sitk.Cast(fixed_image, moving_image.GetPixelID()),
moving_image,
sitk.AffineTransform(2),
sitk.CenteredTransformInitializerFilter.GEOMETRY,
)

or possibly sitk.Euler2DTransform if you only need a rigid transformation.

5

Hey there @zivy, thanks for the answer. 2D affine registration was working fine. However, I was getting errors when I tried to compute and plot TRE values per iteration due to dimension mismatch between the registration method’s MovingInitialTransform, FixedInitialTransform and InitialTransform.

Anyway, I solved it by making both moving and fixed images as three dimensional. Registration and TRE plotting works great, so more than I’m happy with the results.

2 Likes