# 3D Pointset to Pointset registration issue

**URL:** <https://discourse.itk.org/t/3d-pointset-to-pointset-registration-issue/5043>\
**Category:** Algorithms\
**Tags:** pointset, cpp, registration\
**Created:** [June 2, 2022, 12:06pm UTC](https://discourse.itk.org/t/3d-pointset-to-pointset-registration-issue/5043 "2022-06-02T12:06:13Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![rfenioux](https://discourse.itk.org/user_avatar/discourse.itk.org/rfenioux/32/2711_2.png) [@rfenioux](https://discourse.itk.org/u/rfenioux)\
**Post date:** [June 2, 2022, 12:06pm UTC](https://discourse.itk.org/t/3d-pointset-to-pointset-registration-issue/5043/1 "2022-06-02T12:06:13Z")

</div>

Hi everyone,

I’m trying to make [this example](https://examples.itk.org/src/registration/metricsv4/registertwopointsets/documentation) work with 3D pointsets.  
I’m using the same methods as the example (AffineTransform, JensenHavrdaCharvatTsallisPointSetToPointSetMetricv4, RegistrationParameterScalesFromPhysicalShift and GradientDescentOptimizerv4)

However, it seems like the optimization doesn’t converge towards a reasonable solution. I would appreciate any thoughts or ideas about what could be causing this issue.

Here are my input data : fix in blue, moving in yellow

 ![sphere_input](https://discourse.itk.org/uploads/default/original/2X/8/83b93ac4e606135ede11b1683ba9f9f4c04c73d9.png)

And the output : fix in blue, registered moving data in red

 ![sphere_output](https://discourse.itk.org/uploads/default/original/2X/3/32ccef90093e33cf47075bec17085519b9c64eb6.png)

The inputs I have are vtkPolyData, so I first create an itk pointset from the polydata using this simple method:

```auto
vtkPoints* points = polyData->GetPoints();
PointSetType::Pointer mesh = PointSetType::New();
mesh->Initialize();
for (unsigned int i = 0; i < points->GetNumberOfPoints(); ++i) {
	mesh->SetPoint(i, points->GetPoint(i));
}	

```

The parameters for the metric:

```auto
using PointSetMetricType = itk::JensenHavrdaCharvatTsallisPointSetToPointSetMetricv4<PointSetType>;
PointSetMetricType::Pointer metric = PointSetMetricType::New();
metric->SetFixedPointSet(fixedPoints);
metric->SetMovingPointSet(movingPoints);
metric->SetMovingTransform(transform);
metric->SetPointSetSigma(1.0);
metric->SetUseAnisotropicCovariances(false);
metric->SetEvaluationKNeighborhood(50); 
metric->SetCovarianceKNeighborhood(5); 
metric->SetKernelSigma(10.0); 
metric->SetAlpha(1.1);
metric->Initialize();

```

And for the optimizer:

```auto
using OptimizerType = itk::GradientDescentOptimizerv4;
OptimizerType::Pointer optimizer = OptimizerType::New();
optimizer->SetMetric(metric);
optimizer->SetNumberOfIterations(10);
optimizer->SetScalesEstimator(shiftScaleEstimator);
optimizer->SetMaximumStepSizeInPhysicalUnits(3.0);
optimizer->SetConvergenceWindowSize(10);
optimizer->StartOptimization();

```

---

<div class="post-metadata">

**Author:** ![dzenanz](https://discourse.itk.org/user_avatar/discourse.itk.org/dzenanz/32/1093_2.png) [@dzenanz](https://discourse.itk.org/u/dzenanz)\
**Post date:** [June 2, 2022, 12:57pm UTC](https://discourse.itk.org/t/3d-pointset-to-pointset-registration-issue/5043/2 "2022-06-02T12:57:09Z")

</div>

Maybe @Pranjal_Sahu has a comment?

---

<div class="post-metadata">

**Author:** ![Pranjal\_Sahu](https://discourse.itk.org/user_avatar/discourse.itk.org/pranjal_sahu/32/2698_2.png) [@Pranjal\_Sahu](https://discourse.itk.org/u/Pranjal_Sahu)\
**Post date:** [June 2, 2022, 1:09pm UTC](https://discourse.itk.org/t/3d-pointset-to-pointset-registration-issue/5043/3 "2022-06-02T13:09:25Z")

</div>

Can you please print the metric values after each iteration also ?

```auto
def print_iteration():
    print( f"It: {optimizer.GetCurrentIteration()}"
            f" metric value: {optimizer.GetCurrentMetricValue():.6f} ")

optimizer.AddObserver(itk.IterationEvent(), print_iteration)

```

Do the values decrease with each iteration ?  
Also, you can try this example: [Register Two Point Sets — v5.3.0](https://examples.itk.org/src/registration/metricsv4/registertwopointsets/registertwopointsets)

---

<div class="post-metadata">

**Author:** ![rfenioux](https://discourse.itk.org/user_avatar/discourse.itk.org/rfenioux/32/2711_2.png) [@rfenioux](https://discourse.itk.org/u/rfenioux)\
**Post date:** [June 2, 2022, 1:39pm UTC](https://discourse.itk.org/t/3d-pointset-to-pointset-registration-issue/5043/4 "2022-06-02T13:39:48Z")

</div>

Thanks for the insight, I should have thought of doing that.  
The metric is jumping around, so I’m guessing this has to do with the learning rate or the scale estimator maybe?

```auto
It: 0 metric value: -0.00289889
It: 1 metric value: -0.00346599
It: 2 metric value: -0.00335903
It: 3 metric value: -0.00317761
It: 4 metric value: -0.00172806
It: 5 metric value: -0.00258818
It: 6 metric value: -0.00177315
It: 7 metric value: -0.00245881
It: 8 metric value: -0.00201539

```

---

<div class="post-metadata">

**Author:** ![Pranjal\_Sahu](https://discourse.itk.org/user_avatar/discourse.itk.org/pranjal_sahu/32/2698_2.png) [@Pranjal\_Sahu](https://discourse.itk.org/u/Pranjal_Sahu)\
**Post date:** [June 2, 2022, 1:42pm UTC](https://discourse.itk.org/t/3d-pointset-to-pointset-registration-issue/5043/5 "2022-06-02T13:42:55Z")

</div>

Yes, you can try changing some hyperparameters (learning rate, number of iterations, etc.) and observe the output.

---

<div class="post-metadata">

**Author:** ![rfenioux](https://discourse.itk.org/user_avatar/discourse.itk.org/rfenioux/32/2711_2.png) [@rfenioux](https://discourse.itk.org/u/rfenioux)\
**Post date:** [June 2, 2022, 3:23pm UTC](https://discourse.itk.org/t/3d-pointset-to-pointset-registration-issue/5043/6 "2022-06-02T15:23:29Z")

</div>

Thank you for your help, it was indeed a hyperparameter problem.  
It works when using a fixed learning rate with `optimizer->SetLearningRate()`.

But when trying to estimate the learning rate with the `optimizer->SetDoEstimateLearningRateAtEachIteration();` and `optimizer->SetDoEstimateLearningRateOnce();` methods, the optimizer doesn’t converge.