TransformToDisplacementField Affine outputting similar but different transformation

Beginner Questions
1

Hi all,

I’m trying to convert an affine transformation into a displacement field using the code below:

sitk.WriteTransform(aft1, 'aft1.txt')
df = sitk.TransformToDisplacementFieldFilter()
df.SetReferenceImage(moving_image) #tried fixed_image too but still same result

AffDF = df.Execute(aft1)

The affine works well in that it places the moving image in the same size,origin,direction,etc as the fixed image (along with manipulating the actual image) if I were to just apply the transform ‘aft1.txt’. It is invertible and so on.

However, when I get the transform from the df (to check if it does the same transformation as the aft1 transform):

sitk.WriteTransform(sitk.DisplacementFieldTransform(AffDF),'affinedf.h5')

I get something that is close but different from aft1 in every aspect. As an example, the size should stay at 512,512,394 but moves to 540,540,400. It is no longer invertible and such.

My question is, is there a way to get the affine transformation aft1 in a displacement field that outputs exactly how it should if using the txt file (i.e. aft1.txt should be the same as affinedf.h5) ?

2

Hello @CrownMella,

Converting an AffineTransform to a DisplacementFieldTransform is generally something you do not want to do.

What do you mean by: “the size should stay at 512,512,394 but moves to 540,540,400.”?

The affine transform has no size, it has a global domain. A displacement field has a local domain. Consequentially, it is not equivalent to the affine transformation. Outside of the displacement field’s grid domain the transformation is the identity (displacement of zero).

When you provide the image to the TransformToDisplacementFieldFilter the semantics of it is “create a displacement field using the same grid as the image grid: size, spacing, direction cosine and origin”.

Additionally, the affine transformation is represented using very few parameters. A displacement field is represented by many parameters as a function of the grid size.

For additional details, including how to invert displacement fields, see this jupyter notebook.

3

Hi Ziv,

Thanks for your response. I do agree it is not ideal but it might be useful to me if it can be done.

I mean to ask really why the affine transformation (in the form of a DFT) changes something that shouldn’t be affected. Both of my images have dimensions (512,512,394) so what causes the transform to change to arbitrary dimensions?

Am I correct in assuming that when I place moving_image into the filter, the output image should have those same parameters (ex moving_image.GetSize(), GetDirection(), etc)?

4

Hello @CrownMella,

There must be some bug in your code. The displacement field image (it is not a transform object) has the same size, spacing, origin and direction cosine as the image set for the filter. See code below:

import SimpleITK as sitk


def print_metadata(image):
    print(f"size: {image.GetSize()}")
    print(f"origin: {image.GetOrigin()}")
    print(f"spacing: {image.GetSpacing()}")
    print(f"direction: {image.GetDirection()}")


aft1 = sitk.AffineTransform(3)

image = sitk.Image([512, 512, 394], sitk.sitkUInt8)
image.SetSpacing([0.5, 0.5, 3])
image.SetOrigin([-100, 200, -3])

df = sitk.TransformToDisplacementFieldFilter()
df.SetReferenceImage(image)
AffDF = df.Execute(aft1)

print_metadata(image)
print_metadata(AffDF)
5

Hi Ziv,

Yes, those parts work fine for me too. It’s when you convert the AffDF image into a DF transform that gives me trouble. If you apply this transform to the image, it does not give the same result as the txt

sitk.WriteTransform(aft1, ‘aft1.txt’) vs sitk.WriteTransform(sitk.DisplacementFieldTransform(AffDF),‘affinedf.h5’)
are not the same.

6

Hello @CrownMella,

That just means that you are not creating the displacement field correctly.

Assuming the affine transformation was obtained from registration then it maps points from the fixed image coordinate system to the moving image coordinate system.

To resample the moving image onto the fixed image grid, the displacement field needs to be created using the fixed image, df.SetReferenceImage(fixed_image) and then save to file.

Then you read the transform, affine or displacement field, and resample the moving image onto the fixed image grid, resampled_moving = sitk.Resample(moving_image, fixed_image, tx).

7

Hi Ziv,

As I said before, I tried to set the reference image as fixed image also and it still did not work. Also, I tried this case where the two are using the same coordinate system.

Resampling in sitk did produce the same results so now I’m wondering why 3DSlicer’s output is different (for the DF transform) when it usually agrees with SITK output.

The problem still stands though. I am trying to produce the DVF field to use out of ITK/SITK and having it only work within ITK does not solve my problem.