Registering with Rigid and a tiny bit of Affine transform

Beginner Questions
, ,
1

Hi, everyone,
My first post here, please be merciful. I am using simpleITK for 3D microscopy image registration, and it generally works well for rigid transformations. My moving image is a bit skewed, so I’d like to add a small amount of affine transformation to register better. Doing it simply with Affine transform squishes the image too much! Is there a way to add a little bit of affine deformation in the registration? To constrain it somehow?
Thanks!

2

Hello @nvladimus,

Welcome to SimpleITK!

For a first question, this is pretty advanced (even if you didn’t intend it to be :wink:).
What you describe is constrained optimization. Generally speaking, ITK/SimpleITK perform registration using un-constrained optimization, so we cannot place bounds on the transformation parameters.

There still might be a solution. There is one optimizer that supports simple bound constraints, the Limited memory Broyden Fletcher Goldfarb Shannon optimizer, which you can set with the SetOptimizerAsLBFGSB method. It will allow you to set a lower and upper bound for all transformation parameters. If you initialize the affine registration using the rigid transformation which accounts for coarse alignment, then the affine transformation parameters may all be inside the same bounds. It may just work, but it is likely that this is not flexible enough due to using the same bounds for all parameters.

If you can share data / code we might be able to see if there is some other issue that causes the registration to overshoot with the affine transformation parameters.

3

Hi, @zivy,
Thanks for your response. Also, many thanks for the collection of notebooks on github - without them I wouldn’t be able to start with simpleITK!
In my code I may be doing something dumb to begin with. My main code looks like shown below.
Any tips are much appreciated!

final_transform1 = sitk.AffineTransform(3)
registration_method = sitk.ImageRegistrationMethod()
registration_method.SetMetricAsCorrelation()
registration_method.SetInterpolator(sitk.sitkLinear)
registration_method.SetOptimizerAsPowell(stepLength=0.1, numberOfIterations=25)
registration_method.SetOptimizerScalesFromPhysicalShift()
registration_method.SetInitialTransform(final_transform1)
registration_method.SetShrinkFactorsPerLevel(shrinkFactors = [16, 8, 4])
registration_method.SetSmoothingSigmasPerLevel(smoothingSigmas = [4, 2, 0])
registration_method.SmoothingSigmasAreSpecifiedInPhysicalUnitsOn()
registration_method.Execute(fixed, moving)
print(final_transform1)

Output (looking at the matrix, Y-dimension is elongated by 1.39x):
itk::simple::AffineTransform
…
Matrix:
1.01454 0.0669623 -0.254868
-0.029979 1.39409 0.0173503
0.0337193 -0.185563 1.08004
Offset: [-3.50399, -4.87317, 1.04575]
Center: [0, 0, 0]
Translation: [-3.50399, -4.87317, 1.04575]
Inverse:
0.977517 -0.0162139 0.230936
0.0213551 0.715431 -0.00645369
-0.0268495 0.123426 0.917576
Singular: 0

4

Hi @nvladimus,

You’re welcome.

OK, so some observations with respect to the code, from most likely culprit to least:

  1. Registration initialization is done with the identity transformation final_transform1 = sitk.AffineTransform(3). As you are working with an affine transformation the rotation portion will be about the fixed image’s origin and not it’s center. This makes it hard to converge because a tiny rotation around the origin results in a huge shift at the other edge of the image. I suspect that the registration diverges immediately and you don’t see a smooth reduction in the similarity metric value. Please use the CenteredTransformInitializer.

  2. Very tight bound on numberOfIterations, so likely terminated before convergence. This is a hard termination criterion for the optimizer, usually set to several hundred (let’s say 300). You should check why the optimizer terminated print('Optimizer\'s stopping condition, {0}'.format(registration_method.GetOptimizerStopConditionDescription()))

  3. Aggressive shrinking of the data, 16, possibly too much? Depends on your image size.

  4. Speculative, based on my experience with microscopy. What is your image spacing? If it is very small, you may encounter numerical instabilities in registration. When reading images into ITK/SimpleITK the default unit is mm and readers are expected to convert to mm, lets say your microscopy image unit is nanometers which is converted to mm, so a spacing of 5nm becomes 1.0E-5mm. If you use the original units for spacing, and origin then the registration will be stable (5 vs. 1.0E-5). Once you read an image, ITK/SimpleITK do not have a concept of units, sizes and locations are unit-less numbers.

Hopefully one of these resolves the issue.

1 Like
5

Thanks for the tips, @zivy! My units are microns, but I set spacing as if they were mm (pixel size 0.14 mm). I didn’t know about the origin of rotation, will change that, along with other things you suggested!