the images come from different modalities and do not have labels/annotations. Though visually the center elliptical anatomy overlap, is there a way I can come up with a metric that only considers the center elliptical match/overlap performance?
I head about mutual histogram matching or mutual info. optimization but dont know how to implement that with ITK platform.
If you have a region of interest and you want to use that in the registration you will need to explicitly create a mask representing it. Given this mask, the registration will only use voxels/pixels that are inside it for computing similarity.
The ImageRegistrationMethod has SetMetricFixedMask and SetMetricMovingMask methods. Use those to set the mask(s) and then work with the ImageRegistrtaionMethod as usual, using the MetricEvaluate or SetOptimizerAsExhaustive to compute the metric values, see this notebook for code.