# 3D multi-modal medical image registration using user defined function for similarity metric computation

**URL:** <https://discourse.itk.org/t/3d-multi-modal-medical-image-registration-using-user-defined-function-for-similarity-metric-computation/5064>\
**Category:** Beginner Questions\
**Tags:** simpleitk, registration, python\
**Created:** [June 7, 2022, 10:09am UTC](https://discourse.itk.org/t/3d-multi-modal-medical-image-registration-using-user-defined-function-for-similarity-metric-computation/5064 "2022-06-07T10:09:04Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![debapriya](https://discourse.itk.org/letter_avatar_proxy/v4/letter/d/bc79bd/32.png) [@debapriya](https://discourse.itk.org/u/debapriya)\
**Post date:** [June 7, 2022, 10:09am UTC](https://discourse.itk.org/t/3d-multi-modal-medical-image-registration-using-user-defined-function-for-similarity-metric-computation/5064/1 "2022-06-07T10:09:04Z")

</div>

Hello,  
I want to register two 3D multi-modal medical images using mutual information. However instead of using the inbuilt `MattesMutualInformation` or `JointHistogramMutualInformation`, I want to use my own mutual information computation function. The rest of the code, as given in SITK, will remain unchanged. Is this possible? If yes, how can I do that?

---

<div class="post-metadata">

**Author:** ![dzenanz](https://discourse.itk.org/user_avatar/discourse.itk.org/dzenanz/32/1093_2.png) [@dzenanz](https://discourse.itk.org/u/dzenanz)\
**Post date:** [June 7, 2022, 12:16pm UTC](https://discourse.itk.org/t/3d-multi-modal-medical-image-registration-using-user-defined-function-for-similarity-metric-computation/5064/2 "2022-06-07T12:16:44Z")

</div>

To define your own metric, you might have to implement it in ITK and expose it in SimpleITK. @machadoL wanted to [write his own variant of mutual information](https://discourse.itk.org/t/changes-in-itk-mattes-mutual-information-metric/2353), he might update us on progress and maybe share his experience.

---

<div class="post-metadata">

**Author:** ![debapriya](https://discourse.itk.org/letter_avatar_proxy/v4/letter/d/bc79bd/32.png) [@debapriya](https://discourse.itk.org/u/debapriya)\
**Post date:** [June 9, 2022, 5:54am UTC](https://discourse.itk.org/t/3d-multi-modal-medical-image-registration-using-user-defined-function-for-similarity-metric-computation/5064/3 "2022-06-09T05:54:01Z")

</div>

Thank you @dzenanz !! Following your advice and the [chain of conversation you mentioned](https://discourse.itk.org/t/changes-in-itk-mattes-mutual-information-metric/2353), I have found that the computation of mutual information is happening [here](https://github.com/InsightSoftwareConsortium/ITK/blob/b5fdb855a01cd1b223e4b8ecac050fb5e3e2f46c/Modules/Registration/Metricsv4/include/itkJointHistogramMutualInformationImageToImageMetricv4.hxx#L332-L384). Please correct me if I am wrong.

Now I want to run this code, preferably in debugger mode, and modify it. How can I do that? My knowledge in C++ is limited to high school level.

---

<div class="post-metadata">

**Author:** ![dzenanz](https://discourse.itk.org/user_avatar/discourse.itk.org/dzenanz/32/1093_2.png) [@dzenanz](https://discourse.itk.org/u/dzenanz)\
**Post date:** [June 9, 2022, 6:10am UTC](https://discourse.itk.org/t/3d-multi-modal-medical-image-registration-using-user-defined-function-for-similarity-metric-computation/5064/4 "2022-06-09T06:10:08Z")

</div>

The easiest way is to [configure ITK](https://itk.org/Wiki/ITK/Getting_Started) to build with examples (`BUILD_EXAMPLES=ON`), compile it in debug mode, and run one of the registration examples which uses mutual information (such as [ImageRegistration13](https://github.com/InsightSoftwareConsortium/ITK/blob/master/Examples/RegistrationITKv4/ImageRegistration13.cxx)). Put a breakpoint somewhere in the code you highlighted, and see what happens. If you modify that code, most IDEs (integrated development environments) should automatically recompile the example you are using.

---

<div class="post-metadata">

**Author:** ![debapriya](https://discourse.itk.org/letter_avatar_proxy/v4/letter/d/bc79bd/32.png) [@debapriya](https://discourse.itk.org/u/debapriya)\
**Post date:** [June 15, 2022, 12:40pm UTC](https://discourse.itk.org/t/3d-multi-modal-medical-image-registration-using-user-defined-function-for-similarity-metric-computation/5064/5 "2022-06-15T12:40:40Z")

</div>

Thank you @dzenanz . I have configured ITK.

I have run Example codes `ImageRegistration1.cxx` to `ImageRegistration20.cxx` putting breakpoints in `itkJointHistogramMutualInformationImageToImageMetricv4.hxx`. However, none of the codes stopped at the breakpoints.  
Please tell me which code calls `itkJointHistogramMutualInformationImageToImageMetricv4.hxx`.

Also, I could not run the codes which require 3D images as argument (`ImageRegistration8.cxx` and `ImageRegistration20.cxx`). The following error is showing.  
`Could not create IO object for reading file: C:\Users\...BrainWeb\brainweb1e1a10f20.mha. Tried to create one of the following: BMPImageIO BioRadImageIO Bruk..`  
How can I resolve this?

---

<div class="post-metadata">

**Author:** ![dzenanz](https://discourse.itk.org/user_avatar/discourse.itk.org/dzenanz/32/1093_2.png) [@dzenanz](https://discourse.itk.org/u/dzenanz)\
**Post date:** [June 15, 2022, 8:06pm UTC](https://discourse.itk.org/t/3d-multi-modal-medical-image-registration-using-user-defined-function-for-similarity-metric-computation/5064/6 "2022-06-15T20:06:13Z")

</div>

Not all examples use mutual information. Example1 uses [mean squares](https://github.com/InsightSoftwareConsortium/ITK/blob/v5.2.1/Examples/RegistrationITKv4/ImageRegistration1.cxx#L50), and so does [example 20](https://github.com/InsightSoftwareConsortium/ITK/blob/v5.2.1/Examples/RegistrationITKv4/ImageRegistration20.cxx#L35). Make sure that the example you try uses mutual information, and not another metric type.

> BMPImageIO BioRadImageIO Bruk…

What is the complete list of ImageIOs tried? Is `MetaIO` or `MetaImageIO` on that list? That’s what `.mha` file should be read by.

---

<div class="post-metadata">

**Author:** ![debapriya](https://discourse.itk.org/letter_avatar_proxy/v4/letter/d/bc79bd/32.png) [@debapriya](https://discourse.itk.org/u/debapriya)\
**Post date:** [June 17, 2022, 3:48pm UTC](https://discourse.itk.org/t/3d-multi-modal-medical-image-registration-using-user-defined-function-for-similarity-metric-computation/5064/8 "2022-06-17T15:48:33Z")

</div>

`msg | 0x0000021c9d765d40 Could not create IO object for reading file C:\\Users\\ …\\ITK1\\Examples\\Data\\BrainWeb1\\brainweb1e1a10f20.mha\n Tried to create one of the following:\n BMPImageIO\n BioRadImageIO\n Bruk... | std::basic_ostringstream<char,std::char_traits<char>,std::allocator<char>>`

This is the entire error message. How can I find out the complete list of ImageIOs? I am using Visual Studio 2019.

I tried to set the ImageIO with the following command, but it did not work.  
`fixedImageReader->SetImageIO("MetaImageIO");`

Apparently, `SetImageIO` expects `itk::ImageIOBase*` as the datatype of the input parameter. I do not know how to convert `"MetaImageIO"` to `itk::ImageIOBase*`.

---

<div class="post-metadata">

**Author:** ![zivy](https://discourse.itk.org/user_avatar/discourse.itk.org/zivy/32/1726_2.png) [@zivy](https://discourse.itk.org/u/zivy)\
**Post date:** [June 17, 2022, 4:02pm UTC](https://discourse.itk.org/t/3d-multi-modal-medical-image-registration-using-user-defined-function-for-similarity-metric-computation/5064/9 "2022-06-17T16:02:45Z")

</div>

```auto
itk::MetaImageIO::Pointer metaIO = itk::MetaImageIO::New();
fixedImageReader->SetImageIO(metaIO);

```

---

<div class="post-metadata">

**Author:** ![debapriya](https://discourse.itk.org/letter_avatar_proxy/v4/letter/d/bc79bd/32.png) [@debapriya](https://discourse.itk.org/u/debapriya)\
**Post date:** [June 20, 2022, 4:16pm UTC](https://discourse.itk.org/t/3d-multi-modal-medical-image-registration-using-user-defined-function-for-similarity-metric-computation/5064/10 "2022-06-20T16:16:21Z")

</div>

Thank you @zivy . I am able to read 3D images with these commands.

I have modified `itkJointHistogramMutualInformationImagetoImageMetricv4.cxx`. The code is now computing mutual information according to the modified formula. I have used the modified formula to register two 3D images (CT and MR), and it is giving acceptable result.

Now I want to expose this modification to SITK. Please tell me the process.

---

<div class="post-metadata">

**Author:** ![zivy](https://discourse.itk.org/user_avatar/discourse.itk.org/zivy/32/1726_2.png) [@zivy](https://discourse.itk.org/u/zivy)\
**Post date:** [June 20, 2022, 11:05pm UTC](https://discourse.itk.org/t/3d-multi-modal-medical-image-registration-using-user-defined-function-for-similarity-metric-computation/5064/11 "2022-06-20T23:05:39Z")

</div>

Hello @debapriya,

To expose the new ITK metric in SimpleITK, add to the existing metrics by following the existing pattern. The relevant locations: [CreateMetric](https://github.com/SimpleITK/SimpleITK/blob/12c5683f02e5bd90688f0105fed0cf9b84432c43/Code/Registration/src/sitkImageRegistrationMethod_CreateMetric.hxx#L54) and the [SetMetricAsX](https://github.com/SimpleITK/SimpleITK/blob/12c5683f02e5bd90688f0105fed0cf9b84432c43/Code/Registration/src/sitkImageRegistrationMethod.cxx#L213-L260).

---

<div class="post-metadata">

**Author:** ![debapriya](https://discourse.itk.org/letter_avatar_proxy/v4/letter/d/bc79bd/32.png) [@debapriya](https://discourse.itk.org/u/debapriya)\
**Post date:** [June 22, 2022, 5:19pm UTC](https://discourse.itk.org/t/3d-multi-modal-medical-image-registration-using-user-defined-function-for-similarity-metric-computation/5064/12 "2022-06-22T17:19:41Z")

</div>

Thank you @zivy .

I saved my modified code as `itkJointHistogramMutualInformationImageToImageEfficientEntropyMetricv4.hxx`.  
I configured and generated ITK using CMake and built solution in Visual Studio 2019.  
Now, when I open `itkJointHistogramMutualInformationImageToImageEfficientEntropyMetricv4.hxx`, it says it cannot open any source file except `itkJointHistogramMutualInformationImageToImageEfficientEntropyMetricv4.h`.

Apparently, it cannot find the source files which are located in other folders.  
What step am I missing?

Below is a screenshot of the message

 ![image](https://discourse.itk.org/uploads/default/original/2X/b/bd704fc0d3c66884aae453b6f2dffab4814610c6.png)

---

<div class="post-metadata">

**Author:** ![zivy](https://discourse.itk.org/user_avatar/discourse.itk.org/zivy/32/1726_2.png) [@zivy](https://discourse.itk.org/u/zivy)\
**Post date:** [June 22, 2022, 5:45pm UTC](https://discourse.itk.org/t/3d-multi-modal-medical-image-registration-using-user-defined-function-for-similarity-metric-computation/5064/13 "2022-06-22T17:45:22Z")

</div>

Hello @debapriya,

I suspect this is an issue with the VS IDE configuration files generated via CMake but really don’t know. As this is on windows, possibly close the program and open it again?

Possibly @dzenanz or @matt.mccormick may have an idea.

---

<div class="post-metadata">

**Author:** ![debapriya](https://discourse.itk.org/letter_avatar_proxy/v4/letter/d/bc79bd/32.png) [@debapriya](https://discourse.itk.org/u/debapriya)\
**Post date:** [June 23, 2022, 11:42am UTC](https://discourse.itk.org/t/3d-multi-modal-medical-image-registration-using-user-defined-function-for-similarity-metric-computation/5064/14 "2022-06-23T11:42:12Z")

</div>

I deleted the `ITK_build` folder and re-configured ITK afresh. However, the same problem persists. Not only my newly added file `itkJointHistogramMutualInformationImageToImageEfficientEntropyMetricv4`, but also existing files like `itkJointHistogramMutualInformationImageToImageMetricv4` and `itkImageRegistrationMethodv4` are showing the same error.

I tried to run one of the Example codes with breakpoint, but the code is bypassing the lines where `itkImageRegistrationMethodv4` or `itkJointHistogramMutualInformationImageToImageMetricv4` are called.

These codes were running perfectly well before I configured `itkJointHistogramMutualInformationImageToImageEfficientEntropyMetricv4`.

Please tell me what is happening and how to resolve this.

---

<div class="post-metadata">

**Author:** ![debapriya](https://discourse.itk.org/letter_avatar_proxy/v4/letter/d/bc79bd/32.png) [@debapriya](https://discourse.itk.org/u/debapriya)\
**Post date:** [June 24, 2022, 7:05am UTC](https://discourse.itk.org/t/3d-multi-modal-medical-image-registration-using-user-defined-function-for-similarity-metric-computation/5064/15 "2022-06-24T07:05:13Z")

</div>

Please tell me the process of adding new `.h` and `.hxx` files in the `ITK\Modules\Registration\Metricsv4\include` folder and compiling them. What all files need to be updated?

```auto
using MetricType =
    itk::JointHistogramMutualInformationImageToImageEfficientEntropyMetricv4<
  FixedImageType,
  MovingImageType>;

```

When I add the above code snippet, I get a message `namespace *itk* has no member *JointHistogramMutualInformationImageToImageEfficientEntropyMetricv4*`.

How can I add `JointHistogramMutualInformationImageToImageEfficientEntropyMetricv4` to namespace `itk`?

Or, is there a process to add and compile `.h` or `.hxx` files from outside and call them in the codes?

---

<div class="post-metadata">

**Author:** ![zivy](https://discourse.itk.org/user_avatar/discourse.itk.org/zivy/32/1726_2.png) [@zivy](https://discourse.itk.org/u/zivy)\
**Post date:** [June 24, 2022, 1:02pm UTC](https://discourse.itk.org/t/3d-multi-modal-medical-image-registration-using-user-defined-function-for-similarity-metric-computation/5064/16 "2022-06-24T13:02:25Z")

</div>

Hello @debapriya ,

Because this is a templated class the code isn’t compiled when you build ITK, so you don’t really know if the syntax is even valid. What you need to do when adding a new metric to ITK:

1. Make sure that the class is indeed in the itk namespace (like [this](https://github.com/InsightSoftwareConsortium/ITK/blob/e27a8a70c778b08c1074fe7e8ee2adc9d114759d/Modules/Registration/Metricsv4/include/itkJointHistogramMutualInformationImageToImageMetricv4.hxx#L25)), and in the [expected directory](https://github.com/InsightSoftwareConsortium/ITK/tree/master/Modules/Registration/Metricsv4/include).
2. Add a test which instantiates the template. The compiler will create the code and validate that the syntax is correct (like [this](https://github.com/InsightSoftwareConsortium/ITK/blob/master/Modules/Registration/Metricsv4/test/itkJointHistogramMutualInformationImageToImageMetricv4Test.cxx)).
3. Add the test to the CMakeLists.txt file (see [here](https://github.com/InsightSoftwareConsortium/ITK/blob/e27a8a70c778b08c1074fe7e8ee2adc9d114759d/Modules/Registration/Metricsv4/test/CMakeLists.txt)).
4. Build ITK with testing turned on.

---

<div class="post-metadata">

**Author:** ![debapriya](https://discourse.itk.org/letter_avatar_proxy/v4/letter/d/bc79bd/32.png) [@debapriya](https://discourse.itk.org/u/debapriya)\
**Post date:** [June 24, 2022, 4:21pm UTC](https://discourse.itk.org/t/3d-multi-modal-medical-image-registration-using-user-defined-function-for-similarity-metric-computation/5064/17 "2022-06-24T16:21:42Z")

</div>

Thank you so much @zivy . Finally, I could make some progress with your help. However, I am getting errors while building ITK following your steps. My lack of knowledge in advanced C++ is making things worse. Probably, I’ll go through some advanced C++ tutorial and try this again.

Can you clear a basic doubt for me?

My metric `itkJointHistogramMutualInformationImageToImageEfficientEntropyMetricv4.hxx` and `itkJointHistogramMutualInformationImageToImageEfficientEntropyMetricv4.h` are photocopies of `itkJointHistogramMutualInformationImageToImageMetricv4.hxx` and `itkJointHistogramMutualInformationImageToImageMetricv4.h`, with modifications done [here](https://github.com/InsightSoftwareConsortium/ITK/blob/e27a8a70c778b08c1074fe7e8ee2adc9d114759d/Modules/Registration/Metricsv4/include/itkJointHistogramMutualInformationImageToImageMetricv4.hxx#L353-L383) and [here](https://github.com/InsightSoftwareConsortium/ITK/blob/e27a8a70c778b08c1074fe7e8ee2adc9d114759d/Modules/Registration/Metricsv4/include/itkJointHistogramMutualInformationImageToImageMetricv4.h#L248) respectively. The modification consists of algebraic computations only. How come `itkJointHistogramMutualInformationImageToImageMetricv4.hxx` is compiled errorless while `itkJointHistogramMutualInformationImageToImageEfficientEntropyMetricv4.hxx` is throwing errors in huge numbers?

Below is a screenshot of the list of errors.

 ![image](https://discourse.itk.org/uploads/default/original/2X/f/f240c0128107784f69f9b4d403138f3e9163413d.png)

---

<div class="post-metadata">

**Author:** ![debapriya](https://discourse.itk.org/letter_avatar_proxy/v4/letter/d/bc79bd/32.png) [@debapriya](https://discourse.itk.org/u/debapriya)\
**Post date:** [June 24, 2022, 5:36pm UTC](https://discourse.itk.org/t/3d-multi-modal-medical-image-registration-using-user-defined-function-for-similarity-metric-computation/5064/18 "2022-06-24T17:36:28Z")

</div>

Hello @zivy,  
Finally some good news!! My code compiled without error!! All those huge list of errors in the above screenshot was due to some silly mess that I had created. Didn’t take advanced C++ knowledge to debug 😀. Big thanks to you.

Now I have to expose this code in SimpleITK. New challenge ahead 😀 !!

---

<div class="post-metadata">

**Author:** ![debapriya](https://discourse.itk.org/letter_avatar_proxy/v4/letter/d/bc79bd/32.png) [@debapriya](https://discourse.itk.org/u/debapriya)\
**Post date:** [June 25, 2022, 3:17pm UTC](https://discourse.itk.org/t/3d-multi-modal-medical-image-registration-using-user-defined-function-for-similarity-metric-computation/5064/19 "2022-06-25T15:17:54Z")

</div>

Hello,

I am trying to expose my new ITK metric in SimpleITK. However, when I add  
`#include "itkMutualInformationImageToImageEfficientEntropyMetricv4.h"` to `sitkImageRegistrationMethod_CreateMetric.hxx`, I get the following two errors.

1. `Cannot open include file: 'itkMutualInformationImageToImageEfficientEntropyMetricv4.h': No such file or directory [C:\Users\...\Desktop\SITK_build\SimpleITK-build\Code\Registration\src\SimpleITKRegistration.vcxproj]`

2. `Custom build for 'C:\Users\...\Desktop\SITK_build\CMakeFiles\6ac7ef3cb40dea1d1e5f86faaa9d2a21\SimpleITK-configure.rule;C:\Users\...\Desktop\SITK_build\CMakeFiles\6ac7ef3cb40dea1d1e5f86faaa9d2a21\SimpleITK-forcebuild.rule;C:\Users\...\Desktop\SITK_build\CMakeFiles\6ac7ef3cb40dea1d1e5f86faaa9d2a21\SimpleITK-build.rule;C:\Users\...\Desktop\SITK_build\CMakeFiles\6ac7ef3cb40dea1d1e5f86faaa9d2a21\SimpleITK-install.rule;C:\Users\...\Desktop\SITK_build\CMakeFiles\afb7cf371e783ad4daf13950766043ce\SimpleITK-complete.rule;C:\Users\...\Desktop\SITK_build\CMakeFiles\ca4cabce27f87d6108ded18de02ff787\SimpleITK.rule' exited with code 1.`

Earlier, I have compiled `itkMutualInformationImageToImageEfficientEntropyMetricv4.h` in ITK successfully, following the steps given by @zivy and have used it to debug and run Example image registration codes in ITK.

Please help me resolve this.

P.S. I have built SimpleITK on the SuperBuild sub-directory. Could that be a reason for the error?

---

<div class="post-metadata">

**Author:** ![zivy](https://discourse.itk.org/user_avatar/discourse.itk.org/zivy/32/1726_2.png) [@zivy](https://discourse.itk.org/u/zivy)\
**Post date:** [June 26, 2022, 10:17pm UTC](https://discourse.itk.org/t/3d-multi-modal-medical-image-registration-using-user-defined-function-for-similarity-metric-computation/5064/20 "2022-06-26T22:17:42Z")

</div>

Hello @debapriya,

Please follow the [SimpleITK build instructions on read-the-docs](https://simpleitk.readthedocs.io/en/master/building.html). Take the [Building Manually](https://simpleitk.readthedocs.io/en/master/building.html#building-manually) path, as you need to point to your modified ITK version.

---

<div class="post-metadata">

**Author:** ![debapriya](https://discourse.itk.org/letter_avatar_proxy/v4/letter/d/bc79bd/32.png) [@debapriya](https://discourse.itk.org/u/debapriya)\
**Post date:** [July 4, 2022, 6:38am UTC](https://discourse.itk.org/t/3d-multi-modal-medical-image-registration-using-user-defined-function-for-similarity-metric-computation/5064/21 "2022-07-04T06:38:09Z")

</div>

I am trying to build SimpleITK using the [Building Manually](https://simpleitk.readthedocs.io/en/master/building.html#building-manually) path. I am getting the following error in CMake

 ![CMakeError](https://discourse.itk.org/uploads/default/original/2X/4/41b05c3de97447d8ec560d0ef24ecfe0d1e016b9.png)

Looks like `ITK_FACTORY_LIST` is empty, but I do not know how to populate it. Please help.

[Next page](https://discourse.itk.org/t/3d-multi-modal-medical-image-registration-using-user-defined-function-for-similarity-metric-computation/5064.md?page=2)