Hi all,
I’m getting bad affine registration results for certain combinations of moving and fixed images.
To rule out the possibility of poor initial alignment, I considered affine and bspline CTtoMR registrations using sitk.LandmarkBasedTransformInitializer
(for affine) or sitk.BSplineTransformInitializer
. While I get good bspline registrations, I get very poor affine registrations, despite using the same fiducial points for both.
Following is the method used to perform the affine registration:
affineTx = sitk.AffineTransform(fixIm.GetDimension())
initialTx= sitk.LandmarkBasedTransformInitializer(
transform=affineTx, fixedLandmarks=fixPts,
movingLandmarks=movPts, referenceImage=fixIm
)
initialTx= sitk.AffineTransform(initialTx)
regMethod = sitk.ImageRegistrationMethod()
regMethod.SetInitialTransform(initialTx, inPlace=False)
# Similarity metric settings:
samplingPercentage = 0.5
regMethod.SetMetricAsMattesMutualInformation(numberOfHistogramBins=50)
regMethod.SetMetricSamplingStrategy(regMethod.RANDOM) # 08/09
regMethod.SetMetricSamplingPercentage(samplingPercentage)
# Setup for the multiresolution framework:
shrinkFactors = [4,2,1]
smoothingSigmas = [2,1,1]
regMethod.SetShrinkFactorsPerLevel(shrinkFactors=shrinkFactors)
regMethod.SetSmoothingSigmasPerLevel(smoothingSigmas=smoothingSigmas)
regMethod.SmoothingSigmasAreSpecifiedInPhysicalUnitsOn()
# Interpolator settings:
regMethod.SetInterpolator(sitk.sitkLinear)
# Optimizer settings:
regMethod.SetOptimizerAsGradientDescent(
learningRate=learningRate,
numberOfIterations=numIters,
estimateLearningRate=regMethod.Once
)
regMethod.SetOptimizerScalesFromPhysicalShift()
finalTx = regMethod.Execute(fixIm, movIm)
regIm = sitk.Resample(
movIm, fixIm, finalTx, sitk.sitkLinear, 0.0, movIm.GetPixelID()
)
I’ve left out the method I use for the deformable registration since that one isn’t causing me problems, but I would be happy to share if it helped.
Plots of Metric v Iterations and the difference between fixed and registered image at a midstack slice for 7 repeated bspline runs can be found here, and for 5 repeated affine runs here.
When comparing the Metric v Iteration curves of the bspline v affine results, it’s evident that they look very different. Interestingly 1 out of 5 repeated affine runs produced a decent result, and although the metric plot looks very different from the bspline plots, it also looks very different from the other 4 (bad) affine plots.
Interestingly changing the MetricSamplingStrategy
to regMethod.NONE
(in the affine method above) produced a much worse result than the one reasonably good result using regMethod.RANDOM
with samplingPercentage=0.5
.
Note that for the bspline registration I only need samplingPercentage=0.05
to get quite decent results.
My questions are:

Why would the affine registrations fail using the same fiducials that were used for bspline registrations, especially given that I use 50% sampling for affine v 5% for bspline?

Why would the affine registration result using all voxels produce a worse result than one obtained using 50% samples?

Are there any suggested changes to the affine registration method to help improve the results?
Following are links to HDF files of the moving and fixed images.
Thank you.
Update 08/09/21:
Since I used sitk.BSplineTransformInitializer
for the bspline registration and sitk.LandmarkBasedTransformInitializer
for the affine, I wondered if something went wrong with the latter, resulting in a poor initial alignment for the affine registration.
I checked the resampled moving image to fixed image using initialTx
and can confirm that the aligned image looks good  in fact much better than the affine registered results.
So it seems that initial alignment can be ruled out as a possible culprit. I guess the metric settings should be fine (MattesMutualInformation is needed for multimodal images)  besides, those metrics work well for the bspline registration.
For the bspline registration I use the LBFGS2 optimizer with scale factors, and while the shrinkFactors
were as above, smoothingSigmas = [4,2,1]
(I tried this value for smoothingSigmas
for the affine registration but it didn’t improve the result).
I also tried playing around with SetOptimizerAsRegularStepGradientDescent
and SetOptimizerAsGradientDescentLineSearch
for the affine registration, but was unable to improve it.
I’m out of ideas and would greately appreciate some advice.