Changes In ITK Mattes Mutual Information Metric

machadoL · October 26, 2019, 9:47pm

Hi, Everyone!

I have little experience on ITK, and little knowledge on C++, although, I am doing some studies on Mutual Information Metric in Registration Pipelines.

I’ve been reading a lot about registration, types of Mutual Information and its calculation, Metrics, Optimizers, and so on. However, I still do not have the insight on how to make it all into code.

I guess if I understand the ITK code for calculation of Mattes Mutual Information (READING THE DEFINITION FILE) I can make some changes to adapt the other concepts of Mutual Information and move on! And certainly, these edited files could work properly with ITK library.

So far, I am very much lost on the whole ITK documentation . I found these two C++ files:

github.com

hinerm/ITK/blob/master/Modules/Registration/Common/include/itkMattesMutualInformationImageToImageMetric.hxx

/*=========================================================================
 *
 *  Copyright Insight Software Consortium
 *
 *  Licensed under the Apache License, Version 2.0 (the "License");
 *  you may not use this file except in compliance with the License.
 *  You may obtain a copy of the License at
 *
 *         http://www.apache.org/licenses/LICENSE-2.0.txt
 *
 *  Unless required by applicable law or agreed to in writing, software
 *  distributed under the License is distributed on an "AS IS" BASIS,
 *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 *  See the License for the specific language governing permissions and
 *  limitations under the License.
 *
 *=========================================================================*/
#ifndef __itkMattesMutualInformationImageToImageMetric_hxx
#define __itkMattesMutualInformationImageToImageMetric_hxx

This file has been truncated. show original

and this:

github.com

InsightSoftwareConsortium/ITK/blob/master/Modules/Registration/Common/include/itkMattesMutualInformationImageToImageMetric.h

/*=========================================================================
 *
 *  Copyright Insight Software Consortium
 *
 *  Licensed under the Apache License, Version 2.0 (the "License");
 *  you may not use this file except in compliance with the License.
 *  You may obtain a copy of the License at
 *
 *         http://www.apache.org/licenses/LICENSE-2.0.txt
 *
 *  Unless required by applicable law or agreed to in writing, software
 *  distributed under the License is distributed on an "AS IS" BASIS,
 *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 *  See the License for the specific language governing permissions and
 *  limitations under the License.
 *
 *=========================================================================*/
#ifndef itkMattesMutualInformationImageToImageMetric_h
#define itkMattesMutualInformationImageToImageMetric_h

This file has been truncated. show original

Although, I’m not sure if they do contain the calculation of Mattes MI, and I’m not certain if the “.hxx” file contains the practical definitions of the methods I need to know.

I’d really be grateful to get any clue on how to get started with that.

dzenanz · October 27, 2019, 1:59pm

Yes, .hxx file does contain the important code. But you should also look at the parent class, ImageToImageMetric and its implementation, especially GetValueThread method. But beware, this code is optimized for speed, not clarity. In particular, a large part of code deals with multi-threading.

lassoan · October 27, 2019, 11:46pm

For understanding how a software works, it can help if you run it in a debugger, stepping through the source code line by line.

machadoL · October 28, 2019, 2:31pm

Wow… Yes you made a point. I’m afraid I cannot make it through. It’s hard to understand the whole auxiliary multi-thread processing to then understand the metrics execution, But thanks anyway for the precise clue.

I hope I can find a way around.

Thanks!

machadoL · October 28, 2019, 2:31pm

Thanks, Lasso.
I’ll work it out! Sure it is a good direction!

Thank you!

machadoL · November 8, 2019, 2:25pm

Hey @lassoan and @dzenanz,
I’ve made a good progress understanding ImageToImageMetric and MattesMutualInformationImageToImageMetric classes. Read some key papers and guess I got where I need to work to achieve my goals.

Although, I still have some unsolved issues and I’d like to ask for some help.

I did not get clearly where in the code, if in these two classes or elsewhere, parzen windowed PDFs (marginal and joint) are calculated.
They calculate it for both fixed and moving Image?
I’m talking precisely about this term in Mattes (2003) PET-CT paper:

Also in the code they mention a Parzen Window term/argument. Are they talking about the
and terms the Eq. above?

Thanks again, buddies!

dzenanz · November 8, 2019, 2:57pm

itkMattesMutualInformationImageToImageMetric of course. But also JointHistogramMutualInformationComputeJointPDFThreaderBase, and same file further down.

The entire itkJointHistogramMutualInformationGetValueAndDerivativeThreader. Also these files are involved too: itkJointHistogramMutualInformationImageToImageMetricv4 and itkMattesMutualInformationImageToImageMetricv4GetValueAndDerivativeThreader.

machadoL · November 8, 2019, 4:30pm

Wow. I got it!
Thanks @dzenanz!
Will be reading and keep you guys here posted about anything!
Thanks again!

machadoL · November 8, 2019, 9:04pm

Guys,

firstly, thanks for the help.

Now that I’ve done the changes I wanted. What is the pathway I should walk now to plug in the, let’s say, itkMachadoMutualInformationImageToImageMetricMetric.h and itkMachadoMutualInformationImageToImageMetricMetric.hxx , files resulted from the changes I did, and by changing itkMattes… everywhere in the original files by itkMachado… , in order to use it inside ITK environment?

I mean, Can I call those new files just by putting them inside the project folder and including them in the beginning of the main file? is that all? Or should I copy and paste those files somewhere else?

Thanks!

dzenanz · November 8, 2019, 10:19pm

Yes, that should be all. Most filters get started exactly like that. And if you later conclude that the filter(s) are valuable, you can put them into a remote module.

machadoL · November 11, 2019, 6:53pm

Hi @dzenanz,
I’m here again!

I made the changes in the itkMattes[…].h and in itkMattes[…].hxx.
And tested inserting those files inside a registration pipeline (ImageRegistration4.cpp from ITK software Guide examples), then I started to get many basic errors like:

"class has not any field named 'm_PerThread'"
"vcl_numeric_limits not recognized"

So, I looked carefully and found out that the classes […]Mattes[…].h and in […]Mattes[…].hxx used in the ITK5 software guide examples are somewhat different. I found in the ITK source files the following classes:

Now, I’m wondering if this is the latest implementation of itkMattesMultualInformation… , and if I should use those files or the ones available at ITK-Doxygen documentation.
Can you, please, help me to understand which ones I should use in order to compile every thing?

Thank you!

dzenanz · November 11, 2019, 8:26pm

If you started your modification from ITKv5 master, that’s the version you should get ImageRegistration4.cpp from. If you started from ITKv4, you should use examples from there. Preferably the exact same version.

Please copy-paste the entire error message.

Finally, the only files with word Tsallis in the name on my computer are related to itk::JensenHavrdaCharvatTsallisPointSet. Which version of ITK are you working with, and where did you get it?

machadoL · November 11, 2019, 9:05pm

Sorry… I did not make my goals much clear.
I’m inserting other entropy definitions, instead of Shannon’s Entropy (in Mattes classes). I guess I made a complete mess with ITK versions.*

I just downloaded .h and .hxx from master… But I guess now I’ll have to build the ITKv5… Did not realize all that until now.

The screenshot I posted in the earlier message refers to my edited classes. Makes completely no sense since they are part from ItkV5, part from Itkv4. I just compared with the ones in master and I got all that. Hence, the error messages has no much meaning.

I’ll make the proper corrections and let you know if it did work.

Thanks again!

machadoL · November 11, 2019, 11:42pm

@dzenanz,

I double-checked all the files. I downloaded the Master folder from Git and then built that. Then, I realized that the examples’ folder do not contain Registration ITK5, only the ITK4-fashion examples. Am I correct?
Is there any set of ITK5 Registration examples?

dzenanz · November 12, 2019, 2:44pm

There are two registration frameworks. They are currently called v3 and v4. v4 was introduced with version 4, v3 existed prior to version 4. Both are still supported. But you can’t really mix classes from these two frameworks.

v3 is simpler, v4 is more flexible and powerful. Pick one, and go entirely with that one.

machadoL · November 12, 2019, 8:55am

I got it. So, I picked the .h and .hxx classes from ITK master, which is ITK5, I suppose. Therein, I found Registration V4, it means they are compatible. Is that right?

Thanks!

dzenanz · November 12, 2019, 3:19pm

v4 classes are mostly in Registration/Metricsv4, while v3 classes are mostly in Registration/Common. There was no significant registration refactoring between ITKv4 and ITKv5. But registration was affected by multi-threading refactoring done in v5 [migration guide].

machadoL · November 12, 2019, 9:22am

Fine. Now, I guess I can do it right. Thanks!

machadoL · November 20, 2019, 9:19pm

Hey Guys!!!
A lotta progress down here! Thank you guys again for the help. I would not give any step without it!

Now, I am trying to understand the way through which the .hxx class calculates the (jointPDF) derivatives.
I’m a bit confuse.
And why exactly it needs a gradient Image In this process…
Do I have access to the marginal moving PDF derivative according to the parameters?

Thanks.

dzenanz · November 20, 2019, 9:57pm

@hjmjohnson might know that. Also you could look at the blame for the file to see who has been editing it.