Thanks for the reply Dženan,
When I take a single image and translate/rotate a copy, then run registration, the mean square reported goes from a high at the start of 650,000 to a fitted value of 300,000. You can see this in the images below, showing the metric as well as the fitted image overlaid the “original”.
So these images are identical - except for translation/rotation. So I would expect that a mean squares metric would be essentially zero. (assuming its a mean squared difference that’s being reported).
I tried setting the translation to only (2,2) and rotation to only 2 degrees, and even in that case the metric returned is still ~300,000 when fitted.
These images are 512x512 originally int32’s with a value range ~ -3000-3000. In the example below the 2nd image is a translation by (10,10) and rotation of 25 degrees.
The only reason I can think it isn’t, is that the metric is being calculated without a mask to make sure to only to take values from where the images overlap. But even then, I made sure I moved the minimum of each image to 0 prior to registration, so that if it was doing that, and using a default value of 0 for any undefined points, it wouldn’t affect the result