ImageRegistrationMethodv4 questions

gcsharp · May 11, 2021, 4:46pm

Hi all. I have been investigating the ITK v4 registration framework, and have a few questions/comments for discussion.

There seem to be some v3 functionality missing in v4. For example, no FRPROptimizer, no MutualInformationImageToImageMetric, no NormalizedMutualInformationHistogramImageToImageMetric. Are these omissions intentional or accidental?
Is there a way to use ImageRegistrationMethodv4 for multiple objectives? For example, finding a transform that optimizes both image similarity and pointset distnances, or image similarity of multiple image planes.
Does ITK intend to maintain both registration frameworks? Will one or the other be phased out or deprecated?

Thank you,
Greg

dzenanz · May 11, 2021, 5:55pm

Not all optimizers had been refactored into a v4 version.
I think that Mattes MI metric is usually used, so the other flavors of MI have been left out of refactoring during v4.

Both of these were probably done due to cost/benefit reasons.

ImageRegistrationMethodv4 optimizes a single objective function. You can define your own custom metric function which will invoke, then weigh and combine contributions by these other things.
Both v3 and v4 registration frameworks are planned to be maintained.

Maybe someone who participated in the development of v4 could pitch in with more insight? @hjmjohnson @Stephen_Aylward

gcsharp · May 11, 2021, 6:29pm

Thank you. This looks promising! Apparently one needs to use ObjectToObjectMultiMetricv4 as the metric, then add fixed and moving objects using an index argument, like SetFixedImage(3,image) or SetMovingPointSet(3,pointset).

gcsharp · May 12, 2021, 6:42pm

In addition, I have found differences in the way Observer events are emitted between v3 and v4.

Is there any specification for the Observer events?

dzenanz · May 12, 2021, 9:42pm

Trying to write code which will be agnostic to v3/v4 registration framework is hard, due to many minor differences. I would always choose one and stick with it.

This might be something close to a specification for events.

gcsharp · May 14, 2021, 12:28pm

Thanks Dzenan. There appears to be a bug that the RSG v4 optimizer updates the search position before emitting the iteration event. It would be good to have a specification on the events.

dzenanz · May 14, 2021, 2:35pm

I am not sure whether this is a bug, or an inconsistency about when do optimizers generate events (before or after). @blowekamp can you chime in?

gcsharp · May 14, 2021, 3:03pm

I’m sorry that comment was a little bit terse. I am still learning the code. But I am suspicious that the problem is not limited to incorrect parameter values at the time the event is emitted. Consider GradientDescentOptimizerv4Template::ResumeOptimization(). On line 100, the value and gradient are calculated. Then, on line 144, the current position is updated. Thus, by inspection:

The “best parameters” on line 150 mixes the current value with the next position, so if m_ReturnBestParametersAndValue is true, the wrong position will be returned.
If the convergence test on line 83 is positive, and if m_ReturnBestParametersAndValue is false, once again the wrong position will be returned.

Please advise. If you agree this is a bug I will file a ticket.

dzenanz · May 14, 2021, 5:48pm

I responded earlier without looking at the code.

That sounds like a bug. Submitting a bug report is good, making a PR that fixes it even better And it is probably not that hard, now that you have dived into the code.

blowekamp · May 17, 2021, 1:04pm

There are several v4 Optimizers that are based on the gradient descent optimizer, that should maintain consistent behavior before them.

IMHO, the next step to to write a test which checks/demonstrates the expected behavior. Likely this will need a mock metric class. If the test is written generically it could be used to test the other optimizers too.

gcsharp · June 2, 2021, 4:39pm

A ticket has been filed.

github.com/InsightSoftwareConsortium/ITK

GradientDescentOptimizerv4 can return incorrect positions

opened 04:32PM - 02 Jun 21 UTC

gregsharp

type:Bug

### Description The GradientDescentOptimizerv4 algorithm does not correctly… keep track of the score and position (Parameters). This causes issues: - When a new best score is found, an incorrect position may be recorded - Observer callbacks are made with incorrect position See Discourse for discussion https://discourse.itk.org/t/imageregistrationmethodv4-questions/4117/7 ### Steps to Reproduce The link below is a test program that reproduces the problem. It needs a bit of work to become a standalone test but should illustrate how to reproduce. At iteration zero, the observer for AmoebaOptimizerv4 correctly reports that the position is zero whereas the observer for RegularStepGradientDescentOptimizerv4 reports the correct score, but the wrong position. https://gitlab.com/plastimatch/plastimatch/-/blob/master/src/plastimatch/test/itk_registration_v4.cxx ### Expected behavior At iteration zero, the position is zero and so the printout should display 0 17208.6 [0, 0, 0] ### Actual behavior But instead it displays 0 17208.6 [14.999996789911513, -0.006939114154077091, -0.006939116585946947] ### Reproducibility 100% reproducible ### Versions ITK 4.12 and 5.1.2 both exhibit this problem. Probably other versions do too. ### Environment  ### Additional Information

I’m not very familiar with ITK test generation, but I can try. Should I just copy an existing test in RegistrationMethodsv4/test?

blowekamp · June 2, 2021, 5:02pm

As I understand the issue it is just with the optimizer, that not whole registration framework.

I would look at some of the tests in “ITK/Modules/Numerics/Optimizersv4/test” such as:

github.com

InsightSoftwareConsortium/ITK/blob/master/Modules/Numerics/Optimizersv4/test/itkGradientDescentOptimizerv4Test.cxx

/*=========================================================================
 *
 *  Copyright NumFOCUS
 *
 *  Licensed under the Apache License, Version 2.0 (the "License");
 *  you may not use this file except in compliance with the License.
 *  You may obtain a copy of the License at
 *
 *         https://www.apache.org/licenses/LICENSE-2.0.txt
 *
 *  Unless required by applicable law or agreed to in writing, software
 *  distributed under the License is distributed on an "AS IS" BASIS,
 *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 *  See the License for the specific language governing permissions and
 *  limitations under the License.
 *
 *=========================================================================*/
#include "itkGradientDescentOptimizerv4.h"
#include "itkTestingMacros.h"

This file has been truncated. show original

This one has a mock metric to create a something closer to a unit test of the optimizer as opposed to a registration framework integration test. I don’t find this test to be very clean as it print too much useless information to std::cout and does not do enough comparisons and checks.

I find that the GTest library facilitates writing tests with more validation and checks.

github.com

InsightSoftwareConsortium/ITK/blob/master/Modules/Core/Common/test/itkBuildInformationGTest.cxx

/*=========================================================================
 *
 *  Copyright NumFOCUS
 *
 *  Licensed under the Apache License, Version 2.0 (the "License");
 *  you may not use this file except in compliance with the License.
 *  You may obtain a copy of the License at
 *
 *         https://www.apache.org/licenses/LICENSE-2.0.txt
 *
 *  Unless required by applicable law or agreed to in writing, software
 *  distributed under the License is distributed on an "AS IS" BASIS,
 *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 *  See the License for the specific language governing permissions and
 *  limitations under the License.
 *
 *=========================================================================*/

/*
 * Tests for the itk::BuildInformation class

This file has been truncated. show original

If recommend copying some of that old optimizer test work to a new one to tests the events and demonstrate the expected behavior.

gcsharp · June 2, 2021, 5:26pm

Got it. Thanks!

gcsharp · June 2, 2021, 6:30pm

Any idea why ITKOptimizersv4TestDriver doesn’t rebuild when I edit itkGradientDescentOptimizerv4Test2.cxx?

dzenanz · June 2, 2021, 6:34pm

You are editing the wrong one, e.g. in the wrong directory (in case you have multiple copies of ITK)? Or maybe you disabled tests in CMake configuration?

gcsharp · June 2, 2021, 6:39pm

Ha ha, you are correct! I have too many buffers open in emacs and edited the wrong one!

dzenanz · June 2, 2021, 6:40pm

Editing the wrong one happens to me sometimes, that’s why I was able to diagnose your problem so readily

gcsharp · June 2, 2021, 6:55pm

One more question. Is there an emacs c-mode settings for ITK style? I found an old VTK one from 2011, is it the same?

https://itk.org/Wiki/Elisp_Code_for_VTK-Style_C_Indentation

dzenanz · June 2, 2021, 6:57pm

I have not heard of this before, so probably no. There are .editorconfig and .clang-format, if that helps.

cookpa · June 2, 2021, 7:00pm

Thanks for investigating this issue @gcsharp

I don’t know how up to date this is, but here’s the emacs config I used when I used emacs

https://itk.org/Wiki/ITK/FAQ#How_do_I_configure_Emacs_for_ITK_coding_style.3F