# Image Registration: Learning rate for optimizer has no influence

**URL:** <https://discourse.itk.org/t/image-registration-learning-rate-for-optimizer-has-no-influence/4631>\
**Category:** Algorithms\
**Tags:** simpleitk, registration, python\
**Created:** [December 1, 2021, 11:42am UTC](https://discourse.itk.org/t/image-registration-learning-rate-for-optimizer-has-no-influence/4631 "2021-12-01T11:42:35Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![sr1](https://discourse.itk.org/letter_avatar_proxy/v4/letter/s/c67d28/32.png) [@sr1](https://discourse.itk.org/u/sr1)\
**Post date:** [December 1, 2021, 11:42am UTC](https://discourse.itk.org/t/image-registration-learning-rate-for-optimizer-has-no-influence/4631/1 "2021-12-01T11:42:35Z")

</div>

Hi,  
I just followed the tutorial on [61\_Registration\_Introduction\_Continued](http://insightsoftwareconsortium.github.io/SimpleITK-Notebooks/Python_html/61_Registration_Introduction_Continued.html) and noted, that the convergence-speed does not change when I modify the learningRate of the optimizer

 ![Screenshot 2021-12-01 124051](https://discourse.itk.org/uploads/default/original/2X/9/9200731d00e0ac2e6dc9db925514ca3dc8f3330f.png)

I changed the value to learningRate=0 and learningRate=10000, but the resulting metric at each step was almost identical. Do I get something wrong here?

Thanks!

---

<div class="post-metadata">

**Author:** ![zivy](https://discourse.itk.org/user_avatar/discourse.itk.org/zivy/32/1726_2.png) [@zivy](https://discourse.itk.org/u/zivy)\
**Post date:** [December 1, 2021, 2:39pm UTC](https://discourse.itk.org/t/image-registration-learning-rate-for-optimizer-has-no-influence/4631/2 "2021-12-01T14:39:34Z")

</div>

Hello @sr1,

For the learning rate to have an effect you will have to change the default parameter value for the `estimateLearningRate` in the `SetOptimizerAsGradientDesent` method to `sitk.ImageRegistrationMethod.Never`. Otherwise it has a default value of `Once` and the learning rate is estimated once per pyramid level and the user supplied value is ignored.

---

<div class="post-metadata">

**Author:** ![sr1](https://discourse.itk.org/letter_avatar_proxy/v4/letter/s/c67d28/32.png) [@sr1](https://discourse.itk.org/u/sr1)\
**Post date:** [December 7, 2021, 8:47am UTC](https://discourse.itk.org/t/image-registration-learning-rate-for-optimizer-has-no-influence/4631/3 "2021-12-07T08:47:07Z")

</div>

Yes, that’s the solution! Thanks a alot.

```auto
reg_method.SetOptimizerAsGradientDescentLineSearch(1.5, estimateLearningRate = sitk.ImageRegistrationMethod.Never, ...)

```

I think this issue could potentially confuse a lot of user, as the learning Rate is explicitly specified in the example mentioned above, but it has no effect.  
**Maybe the default should be “Never” when learningRate is set by the user?!**

---

<div class="post-metadata">

**Author:** ![zivy](https://discourse.itk.org/user_avatar/discourse.itk.org/zivy/32/1726_2.png) [@zivy](https://discourse.itk.org/u/zivy)\
**Post date:** [December 7, 2021, 2:26pm UTC](https://discourse.itk.org/t/image-registration-learning-rate-for-optimizer-has-no-influence/4631/4 "2021-12-07T14:26:12Z")

</div>

Hello @sr1,

[Notebook documentation](https://github.com/InsightSoftwareConsortium/SimpleITK-Notebooks/blob/master/Python/61_Registration_Introduction_Continued.ipynb) has been added to explain the setting.

Changing the default behavior will not be backwards compatible so we may possibly change it in the next major toolkit release.

---

<div class="post-metadata">

**Author:** ![sr1](https://discourse.itk.org/letter_avatar_proxy/v4/letter/s/c67d28/32.png) [@sr1](https://discourse.itk.org/u/sr1)\
**Post date:** [December 7, 2021, 2:32pm UTC](https://discourse.itk.org/t/image-registration-learning-rate-for-optimizer-has-no-influence/4631/5 "2021-12-07T14:32:46Z")

</div>

Great, thank you! 🙂

---

<div class="post-metadata">

**Author:** ![Lee\_Newberg](https://discourse.itk.org/user_avatar/discourse.itk.org/lee_newberg/32/1461_2.png) [@Lee\_Newberg](https://discourse.itk.org/u/Lee_Newberg)\
**Post date:** [December 7, 2021, 2:53pm UTC](https://discourse.itk.org/t/image-registration-learning-rate-for-optimizer-has-no-influence/4631/6 "2021-12-07T14:53:13Z")

</div>

Should we report a warning or error if the learning rate is explicitly supplied in a case where it is ignored?

---

<div class="post-metadata">

**Author:** ![dzenanz](https://discourse.itk.org/user_avatar/discourse.itk.org/dzenanz/32/1093_2.png) [@dzenanz](https://discourse.itk.org/u/dzenanz)\
**Post date:** [December 7, 2021, 3:41pm UTC](https://discourse.itk.org/t/image-registration-learning-rate-for-optimizer-has-no-influence/4631/7 "2021-12-07T15:41:01Z")

</div>

@Lee_Newberg yes, I think we should.