# ImageRegistrationMethodv4 questions

**URL:** https://discourse.itk.org/t/imageregistrationmethodv4-questions/4117
**Category:** Algorithms
**Created:** [May 11, 2021, 4:46pm UTC](https://discourse.itk.org/t/imageregistrationmethodv4-questions/4117 "2021-05-11T16:46:45Z")
**Posts on this page:** 20
**Page:** 1

<div class="post-metadata">

### Author: ![gcsharp](https://discourse.itk.org/letter_avatar_proxy/v4/letter/g/7bcc69/32.png) [@gcsharp](https://discourse.itk.org/u/gcsharp)
#### Post date: [May 11, 2021, 4:46pm UTC](https://discourse.itk.org/t/imageregistrationmethodv4-questions/4117/1 "2021-05-11T16:46:45Z")

</div>

Hi all. I have been investigating the ITK v4 registration framework, and have a few questions/comments for discussion.

- There seem to be some v3 functionality missing in v4. For example, no FRPROptimizer, no MutualInformationImageToImageMetric, no NormalizedMutualInformationHistogramImageToImageMetric. Are these omissions intentional or accidental?
- Is there a way to use ImageRegistrationMethodv4 for multiple objectives? For example, finding a transform that optimizes both image similarity and pointset distnances, or image similarity of multiple image planes.
- Does ITK intend to maintain both registration frameworks? Will one or the other be phased out or deprecated?

Thank you,  
Greg

---

<div class="post-metadata">

### Author: ![dzenanz](https://discourse.itk.org/user_avatar/discourse.itk.org/dzenanz/32/1093_2.png) [@dzenanz](https://discourse.itk.org/u/dzenanz)
#### Post date: [May 11, 2021, 5:55pm UTC](https://discourse.itk.org/t/imageregistrationmethodv4-questions/4117/2 "2021-05-11T17:55:02Z")

</div>

- Not all optimizers had been refactored into a v4 version.
- I think that Mattes MI metric is usually used, so the other flavors of MI have been left out of refactoring during v4.

Both of these were probably done due to cost/benefit reasons.

- ImageRegistrationMethodv4 optimizes a single objective function. You can define your own custom metric function which will invoke, then weigh and combine contributions by these other things.
- Both v3 and v4 registration frameworks are planned to be maintained.

Maybe someone who participated in the development of v4 could pitch in with more insight? @hjmjohnson @Stephen_Aylward

---

<div class="post-metadata">

### Author: ![gcsharp](https://discourse.itk.org/letter_avatar_proxy/v4/letter/g/7bcc69/32.png) [@gcsharp](https://discourse.itk.org/u/gcsharp)
#### Post date: [May 11, 2021, 6:29pm UTC](https://discourse.itk.org/t/imageregistrationmethodv4-questions/4117/3 "2021-05-11T18:29:25Z")

</div>

> [@dzenanz](#):
>
> ImageRegistrationMethodv4 optimizes a single objective function. You can define your own custom metric function which will invoke, then weigh and combine contributions by these other things.

Thank you. This looks promising! Apparently one needs to use ObjectToObjectMultiMetricv4 as the metric, then add fixed and moving objects using an index argument, like SetFixedImage(3,image) or SetMovingPointSet(3,pointset).

---

<div class="post-metadata">

### Author: ![gcsharp](https://discourse.itk.org/letter_avatar_proxy/v4/letter/g/7bcc69/32.png) [@gcsharp](https://discourse.itk.org/u/gcsharp)
#### Post date: [May 12, 2021, 6:42pm UTC](https://discourse.itk.org/t/imageregistrationmethodv4-questions/4117/4 "2021-05-12T18:42:09Z")

</div>

In addition, I have found differences in the way Observer events are emitted between v3 and v4.

Is there any specification for the Observer events?

---

<div class="post-metadata">

### Author: ![dzenanz](https://discourse.itk.org/user_avatar/discourse.itk.org/dzenanz/32/1093_2.png) [@dzenanz](https://discourse.itk.org/u/dzenanz)
#### Post date: [May 12, 2021, 9:42pm UTC](https://discourse.itk.org/t/imageregistrationmethodv4-questions/4117/5 "2021-05-12T21:42:15Z")

</div>

Trying to write code which will be agnostic to v3/v4 registration framework is hard, due to many minor differences. I would always choose one and stick with it.

[This](https://itk.org/Doxygen/html/itkEventObject_8h.html) might be something close to a specification for events.

---

<div class="post-metadata">

### Author: ![gcsharp](https://discourse.itk.org/letter_avatar_proxy/v4/letter/g/7bcc69/32.png) [@gcsharp](https://discourse.itk.org/u/gcsharp)
#### Post date: [May 14, 2021, 12:28pm UTC](https://discourse.itk.org/t/imageregistrationmethodv4-questions/4117/6 "2021-05-14T12:28:09Z")

</div>

Thanks Dzenan. There appears to be a bug that the RSG v4 optimizer updates the search position before emitting the iteration event. It would be good to have a specification on the events.

---

<div class="post-metadata">

### Author: ![dzenanz](https://discourse.itk.org/user_avatar/discourse.itk.org/dzenanz/32/1093_2.png) [@dzenanz](https://discourse.itk.org/u/dzenanz)
#### Post date: [May 14, 2021, 2:35pm UTC](https://discourse.itk.org/t/imageregistrationmethodv4-questions/4117/7 "2021-05-14T14:35:35Z")

</div>

> [@gcsharp](#):
>
> RSG v4 optimizer updates the search position before emitting the iteration event

I am not sure whether this is a bug, or an inconsistency about when do optimizers generate events (before or after). @blowekamp can you chime in?

---

<div class="post-metadata">

### Author: ![gcsharp](https://discourse.itk.org/letter_avatar_proxy/v4/letter/g/7bcc69/32.png) [@gcsharp](https://discourse.itk.org/u/gcsharp)
#### Post date: [May 14, 2021, 3:03pm UTC](https://discourse.itk.org/t/imageregistrationmethodv4-questions/4117/8 "2021-05-14T15:03:40Z")

</div>

I’m sorry that comment was a little bit terse. I am still learning the code. But I am suspicious that the problem is not limited to incorrect parameter values at the time the event is emitted. Consider GradientDescentOptimizerv4Template::ResumeOptimization(). On line 100, the value and gradient are calculated. Then, on line 144, the current position is updated. Thus, by inspection:

- The “best parameters” on line 150 mixes the current value with the next position, so if m\_ReturnBestParametersAndValue is true, the wrong position will be returned.
- If the convergence test on line 83 is positive, and if m\_ReturnBestParametersAndValue is false, once again the wrong position will be returned.

Please advise. If you agree this is a bug I will file a ticket.

---

<div class="post-metadata">

### Author: ![dzenanz](https://discourse.itk.org/user_avatar/discourse.itk.org/dzenanz/32/1093_2.png) [@dzenanz](https://discourse.itk.org/u/dzenanz)
#### Post date: [May 14, 2021, 5:48pm UTC](https://discourse.itk.org/t/imageregistrationmethodv4-questions/4117/9 "2021-05-14T17:48:59Z")

</div>

I responded earlier without looking at the code.

That sounds like a bug. Submitting a bug report is good, making a PR that fixes it even better 😃 And it is probably not that hard, now that you have dived into the code.

---

<div class="post-metadata">

### Author: ![blowekamp](https://discourse.itk.org/user_avatar/discourse.itk.org/blowekamp/32/79_2.png) [@blowekamp](https://discourse.itk.org/u/blowekamp)
#### Post date: [May 17, 2021, 1:04pm UTC](https://discourse.itk.org/t/imageregistrationmethodv4-questions/4117/10 "2021-05-17T13:04:00Z")

</div>

There are several v4 Optimizers that are based on the gradient descent optimizer, that should maintain consistent behavior before them.

IMHO, the next step to to write a test which checks/demonstrates the expected behavior. Likely this will need a mock metric class. If the test is written generically it could be used to test the other optimizers too.

---

<div class="post-metadata">

### Author: ![gcsharp](https://discourse.itk.org/letter_avatar_proxy/v4/letter/g/7bcc69/32.png) [@gcsharp](https://discourse.itk.org/u/gcsharp)
#### Post date: [June 2, 2021, 4:39pm UTC](https://discourse.itk.org/t/imageregistrationmethodv4-questions/4117/11 "2021-06-02T16:39:01Z")

</div>

A ticket has been filed.

> <https://github.com/InsightSoftwareConsortium/ITK/issues/2570>
>
> \### Description
> 
> The GradientDescentOptimizerv4 algorithm does not correctly… keep track of the score and position (Parameters). This causes issues:
> 
> \- When a new best score is found, an incorrect position may be recorded
> \- Observer callbacks are made with incorrect position
> 
> See Discourse for discussion
> 
> https://discourse.itk.org/t/imageregistrationmethodv4-questions/4117/7
> 
> \### Steps to Reproduce
> 
> The link below is a test program that reproduces the problem. It needs a bit of work to become a standalone test but should illustrate how to reproduce. At iteration zero, the observer for AmoebaOptimizerv4 correctly reports that the position is zero whereas the observer for RegularStepGradientDescentOptimizerv4 reports the correct score, but the wrong position.
> 
> https://gitlab.com/plastimatch/plastimatch/-/blob/master/src/plastimatch/test/itk\_registration\_v4.cxx
> 
> \### Expected behavior
> 
> At iteration zero, the position is zero and so the printout should display
> 
> 0 17208.6 \[0, 0, 0\]
> 
> \### Actual behavior
> 
> But instead it displays
> 
> 0 17208.6 \[14.999996789911513, -0.006939114154077091, -0.006939116585946947\]
> 
> \### Reproducibility
> 
> 100% reproducible
> 
> \### Versions
> 
> ITK 4.12 and 5.1.2 both exhibit this problem. Probably other versions do too.
> 
> \### Environment
> 
> \<!-- Which your OS, CMake, and compiler versions are, or your Python and
> installed package versions are. --\>
> 
> \### Additional Information

I’m not very familiar with ITK test generation, but I can try. Should I just copy an existing test in RegistrationMethodsv4/test?

---

<div class="post-metadata">

### Author: ![blowekamp](https://discourse.itk.org/user_avatar/discourse.itk.org/blowekamp/32/79_2.png) [@blowekamp](https://discourse.itk.org/u/blowekamp)
#### Post date: [June 2, 2021, 5:02pm UTC](https://discourse.itk.org/t/imageregistrationmethodv4-questions/4117/12 "2021-06-02T17:02:46Z")

</div>

> [@gcsharp](#):
>
> Should I just copy an existing test in RegistrationMethodsv4/test?

As I understand the issue it is just with the optimizer, that not whole registration framework.

I would look at some of the tests in “ITK/Modules/Numerics/Optimizersv4/test” such as:

> <https://github.com/InsightSoftwareConsortium/ITK/blob/master/Modules/Numerics/Optimizersv4/test/itkGradientDescentOptimizerv4Test.cxx>

This one has a mock metric to create a something closer to a unit test of the optimizer as opposed to a registration framework integration test. I don’t find this test to be very clean as it print too much useless information to std::cout and does not do enough comparisons and checks.

I find that the [GTest](https://github.com/google/googletest/blob/master/docs/primer.md) library facilitates writing tests with more validation and checks.

> <https://github.com/InsightSoftwareConsortium/ITK/blob/master/Modules/Core/Common/test/itkBuildInformationGTest.cxx>

If recommend copying some of that old optimizer test work to a new one to tests the events and demonstrate the expected behavior.

---

<div class="post-metadata">

### Author: ![gcsharp](https://discourse.itk.org/letter_avatar_proxy/v4/letter/g/7bcc69/32.png) [@gcsharp](https://discourse.itk.org/u/gcsharp)
#### Post date: [June 2, 2021, 5:26pm UTC](https://discourse.itk.org/t/imageregistrationmethodv4-questions/4117/13 "2021-06-02T17:26:40Z")

</div>

Got it. Thanks!

---

<div class="post-metadata">

### Author: ![gcsharp](https://discourse.itk.org/letter_avatar_proxy/v4/letter/g/7bcc69/32.png) [@gcsharp](https://discourse.itk.org/u/gcsharp)
#### Post date: [June 2, 2021, 6:30pm UTC](https://discourse.itk.org/t/imageregistrationmethodv4-questions/4117/14 "2021-06-02T18:30:17Z")

</div>

Any idea why ITKOptimizersv4TestDriver doesn’t rebuild when I edit itkGradientDescentOptimizerv4Test2.cxx?

---

<div class="post-metadata">

### Author: ![dzenanz](https://discourse.itk.org/user_avatar/discourse.itk.org/dzenanz/32/1093_2.png) [@dzenanz](https://discourse.itk.org/u/dzenanz)
#### Post date: [June 2, 2021, 6:34pm UTC](https://discourse.itk.org/t/imageregistrationmethodv4-questions/4117/15 "2021-06-02T18:34:06Z")

</div>

You are editing the wrong one, e.g. in the wrong directory (in case you have multiple copies of ITK)? Or maybe you disabled tests in CMake configuration?

---

<div class="post-metadata">

### Author: ![gcsharp](https://discourse.itk.org/letter_avatar_proxy/v4/letter/g/7bcc69/32.png) [@gcsharp](https://discourse.itk.org/u/gcsharp)
#### Post date: [June 2, 2021, 6:39pm UTC](https://discourse.itk.org/t/imageregistrationmethodv4-questions/4117/16 "2021-06-02T18:39:26Z")

</div>

Ha ha, you are correct! I have too many buffers open in emacs and edited the wrong one!

---

<div class="post-metadata">

### Author: ![dzenanz](https://discourse.itk.org/user_avatar/discourse.itk.org/dzenanz/32/1093_2.png) [@dzenanz](https://discourse.itk.org/u/dzenanz)
#### Post date: [June 2, 2021, 6:40pm UTC](https://discourse.itk.org/t/imageregistrationmethodv4-questions/4117/17 "2021-06-02T18:40:38Z")

</div>

Editing the wrong one happens to me sometimes, that’s why I was able to diagnose your problem so readily 😃

---

<div class="post-metadata">

### Author: ![gcsharp](https://discourse.itk.org/letter_avatar_proxy/v4/letter/g/7bcc69/32.png) [@gcsharp](https://discourse.itk.org/u/gcsharp)
#### Post date: [June 2, 2021, 6:55pm UTC](https://discourse.itk.org/t/imageregistrationmethodv4-questions/4117/18 "2021-06-02T18:55:14Z")

</div>

One more question. Is there an emacs c-mode settings for ITK style? I found an old VTK one from 2011, is it the same?

[https://itk.org/Wiki/Elisp\_Code\_for\_VTK-Style\_C\_Indentation](https://itk.org/Wiki/Elisp_Code_for_VTK-Style_C_Indentation)

---

<div class="post-metadata">

### Author: ![dzenanz](https://discourse.itk.org/user_avatar/discourse.itk.org/dzenanz/32/1093_2.png) [@dzenanz](https://discourse.itk.org/u/dzenanz)
#### Post date: [June 2, 2021, 6:57pm UTC](https://discourse.itk.org/t/imageregistrationmethodv4-questions/4117/19 "2021-06-02T18:57:48Z")

</div>

I have not heard of this before, so probably no. There are [.editorconfig](https://github.com/InsightSoftwareConsortium/ITK/blob/master/.editorconfig) and [.clang-format](https://github.com/InsightSoftwareConsortium/ITK/blob/master/.clang-format), if that helps.

---

<div class="post-metadata">

### Author: ![cookpa](https://discourse.itk.org/user_avatar/discourse.itk.org/cookpa/32/255_2.png) [@cookpa](https://discourse.itk.org/u/cookpa)
#### Post date: [June 2, 2021, 7:00pm UTC](https://discourse.itk.org/t/imageregistrationmethodv4-questions/4117/20 "2021-06-02T19:00:21Z")

</div>

Thanks for investigating this issue @gcsharp

I don’t know how up to date this is, but here’s the emacs config I used when I used emacs

[https://itk.org/Wiki/ITK/FAQ#How\_do\_I\_configure\_Emacs\_for\_ITK\_coding\_style.3F](https://itk.org/Wiki/ITK/FAQ#How_do_I_configure_Emacs_for_ITK_coding_style.3F)

[Next page](https://discourse.itk.org/t/imageregistrationmethodv4-questions/4117.md?page=2)