# Need guidance to address the regression error ?
**URL:** https://discourse.itk.org/t/need-guidance-to-address-the-regression-error/5569
**Category:** Beginner Questions
**Created:** [December 13, 2022, 12:45pm UTC](https://discourse.itk.org/t/need-guidance-to-address-the-regression-error/5569 "2022-12-13T12:45:18Z")
**Posts on this page:** 15
**Page:** 1
### Author:  [@parth721](https://discourse.itk.org/u/parth721)
#### Post date: [December 13, 2022, 12:45pm UTC](https://discourse.itk.org/t/need-guidance-to-address-the-regression-error/5569/1 "2022-12-13T12:45:18Z")
As a beginner I take up an issue, where I need to perform regression testing.
In order to understand the code, I had read the **ItkSoftwareGuide** pdf, google about it, watch lectures on youtube about regression testing, asked people on linkedin through direct messages.
I understand what is regression testing? when we perform it?
But I got stuck on the question :
1.how to perform it ?
2. do I need any tool like selerium?
Because everywhere showing what is testing cycle etc Just theoretical stuffs. But not how to run the test.
If you can suggest some resources or any piece of advice, it will be helpful.
Thank you 🙂
---
### Author:  [@dzenanz](https://discourse.itk.org/u/dzenanz)
#### Post date: [December 13, 2022, 4:07pm UTC](https://discourse.itk.org/t/need-guidance-to-address-the-regression-error/5569/2 "2022-12-13T16:07:02Z")
`selenium` is used for browser-testing the code of web applications.
For testing ITK’s code, we use CTest ([docs](https://cmake.org/cmake/help/book/mastering-cmake/chapter/Testing%20With%20CMake%20and%20CTest.html#testing-using-ctest)). Here is a somewhat outdated [description](https://itk.org/Wiki/ITK/Testing).
Are you looking to add a new test to ITK, or to start adding tests to your application?
---
### Author:  [@parth721](https://discourse.itk.org/u/parth721)
#### Post date: [December 14, 2022, 11:06am UTC](https://discourse.itk.org/t/need-guidance-to-address-the-regression-error/5569/3 "2022-12-14T11:06:25Z")
yes I am looking to add a new regression test to ITK.
---
### Author:  [@dzenanz](https://discourse.itk.org/u/dzenanz)
#### Post date: [December 14, 2022, 2:19pm UTC](https://discourse.itk.org/t/need-guidance-to-address-the-regression-error/5569/4 "2022-12-14T14:19:32Z")
You should read [Tests](https://itk.org/ITKSoftwareGuide/html/Book1/ITKSoftwareGuide-Book1ch9.html#x55-1510009.4) section of the software guide, and possibly skim the rest of that chapter.
For concrete examples, you could take a look at tests of one of ITK’s built-in modules, such as [ITKSmoothing](https://github.com/InsightSoftwareConsortium/ITK/tree/master/Modules/Filtering/Smoothing/test), or one of remote modules, such as [ITKCuberille](https://github.com/InsightSoftwareConsortium/ITKCuberille/tree/master/test). [CuberilleTest04.cxx](https://github.com/InsightSoftwareConsortium/ITKCuberille/blob/master/test/CuberilleTest04.cxx) is on the simpler side.
---
### Author:  [@parth721](https://discourse.itk.org/u/parth721)
#### Post date: [December 26, 2022, 1:19am UTC](https://discourse.itk.org/t/need-guidance-to-address-the-regression-error/5569/5 "2022-12-26T01:19:11Z")
Really I am still clueless.
Somebody changed the instance variable value to 100.
so, I am trying to test is it showing any error. But my knowledge lags behind with few question :-
1. where should I write the code `add_test(...)` in visual studio/ notepad?
2. how I get to know my tests going right ?
3. do I need the knowledge of root directory ? etc.
I want to learn something from the issue. At minimum performing regression testing.
Thank you.
---
### Author:  [@dzenanz](https://discourse.itk.org/u/dzenanz)
#### Post date: [December 27, 2022, 5:49pm UTC](https://discourse.itk.org/t/need-guidance-to-address-the-regression-error/5569/6 "2022-12-27T17:49:29Z")
`add_test` should go into CMakeLists.txt, see [example](https://github.com/InsightSoftwareConsortium/ITK/blob/v5.3.0/Modules/Segmentation/LabelVoting/test/CMakeLists.txt#L32-L33). Here is an example of [quantitative checking](https://github.com/InsightSoftwareConsortium/ITK/blob/v5.3.0/Modules/Segmentation/LabelVoting/test/itkLabelVotingImageFilterTest.cxx#L134-L145).
To execute the test, run `ctest` in ITK build directory. Example invocation: `ctest -VV -C Debug -R itkMultiLabelSTAPLEImageFilterTest`. This will print the actual command line, which you can then use in your IDE (Visual Studio). Sample output:
```nohighlight
test 3
Start 3: itkOMEZarrNGFFImageIOReadConvertTestCT1
3: Test command: C:\Dev\ITKIOOMEZarrNGFF-vs19\bin\Debug\IOOMEZarrNGFFTestDriver.exe "--compare" "C:/Dev/ITKIOOMEZarrNGFF-vs19/ExternalData/test/Input/cthead1.mha" "C:/Dev/ITKIOOMEZarrNGFF-vs19/Testing/Temporary/cthead1.zarr" "itkOMEZarrNGFFImageIOTest" "C:/Dev/ITKIOOMEZarrNGFF-vs19/ExternalData/test/Input/cthead1.mha" "C:/Dev/ITKIOOMEZarrNGFF-vs19/Testing/Temporary/cthead1.zarr"
3: Working Directory: C:/Dev/ITKIOOMEZarrNGFF-vs19/test
```
And how it translates into Visual Studio debugging setup:
Then you can use the usual debugging process: run to, breakpoints, watches etc.
---
### Author:  [@parth721](https://discourse.itk.org/u/parth721)
#### Post date: [January 1, 2023, 4:37pm UTC](https://discourse.itk.org/t/need-guidance-to-address-the-regression-error/5569/7 "2023-01-01T16:37:45Z")
Happy new year to all 😄.
Ok I checked both the files `itkMultiLabelSTAPLEImageFilterTest.cxx` and the `itkLabelVotingImageFilterTest.cxx` for quantitative checking. They are almost same.
Now, as the issue states that :
" a regression error was introduced by setting the `m_MaximumNumberOfIterations` instance Value to 0."
But the Value of `MaximumNumberOfIterations` is changed to 100 from 0.
Does this means that the issue was solved? or I need to perform certain kind of tasks?
---
### Author:  [@parth721](https://discourse.itk.org/u/parth721)
#### Post date: [January 1, 2023, 5:06pm UTC](https://discourse.itk.org/t/need-guidance-to-address-the-regression-error/5569/8 "2023-01-01T17:06:11Z")
How to verify the output values of the confusion matrices ? and compare them to check that no regressions are introduced ? since I don’t have any past data of matrices to compare
thank you for your kindness 🙂
---
### Author:  [@dzenanz](https://discourse.itk.org/u/dzenanz)
#### Post date: [January 2, 2023, 4:35pm UTC](https://discourse.itk.org/t/need-guidance-to-address-the-regression-error/5569/9 "2023-01-02T16:35:10Z")
The usual way is to establish a baseline from current numbers. That is practically done by expecting all zeroes, and the test failure will tell you what are the actual numbers produced. Then put those numbers as expected.
---
### Author:  [@parth721](https://discourse.itk.org/u/parth721)
#### Post date: [January 8, 2023, 2:20pm UTC](https://discourse.itk.org/t/need-guidance-to-address-the-regression-error/5569/10 "2023-01-08T14:20:07Z")
I ran the test using `ctest -R itkMultiLabelSTAPLEImageFilterTest -VV -C Debug`. And it provides some output but the test was passed with MaximumNumberOfIteration = 0.
and showing ITK ERROR with m\_PriorProbabilities array.
what should I do now because no test failure its showing ?
Thanks you 🙂
---
### Author:  [@dzenanz](https://discourse.itk.org/u/dzenanz)
#### Post date: [January 9, 2023, 5:24pm UTC](https://discourse.itk.org/t/need-guidance-to-address-the-regression-error/5569/11 "2023-01-09T17:24:51Z")
This is expected, testing that the class throws an error in some conditions:
```nohighlight
2073: Trying filter->Update()
2073: Caught expected exception
2073:
2073: itk::ExceptionObject (00000064534FE6F8)
2073: Location: "void __cdecl itk::MultiLabelSTAPLEImageFilter,class itk::Image,float>::InitializePriorProbabilities(void)"
2073: File: C:\Dev\ITK-git\Modules\Segmentation\LabelVoting\include\itkMultiLabelSTAPLEImageFilter.hxx
2073: Line: 185
2073: Description: ITK ERROR: MultiLabelSTAPLEImageFilter(000001AFB44A0DA0): m_PriorProbabilities array has wrong size [0]; should be at least 9
```
Look at how `combinationABC` is quantitatively checked here:
>
The ground truth is provided here:
>
What that issue was suggesting to add similar quantitative comparison for the 3 matrices and `PriorProbabilities`.
---
### Author:  [@parth721](https://discourse.itk.org/u/parth721)
#### Post date: [January 14, 2023, 5:07am UTC](https://discourse.itk.org/t/need-guidance-to-address-the-regression-error/5569/12 "2023-01-14T05:07:47Z")
Hoping you doing well. 😇 🙂
I want develop some clarity, in order to find the value of `maximumNumberOfIteration`
STEP 1 : I need to perform the ctest and test failure tell me what is the value of `maximumNumberOfIteration`.
STEP 2 : Assign the value to `maximumNumberOfIteration`
Now the challenge is, my test was passed with `maximumNumberOfIteration = 0` instead of failed test. As shown before. So I tried hit & trial methodology to find the solution 😅 but I knew its not right and nor an efficient approach.
So, how can I find what should be the original value of `maximumNumberOfIteration`?
Am I applying a right approach to fix the issue or I am missing some concepts?
Thank you for your precious time 🙂
---
### Author:  [@dzenanz](https://discourse.itk.org/u/dzenanz)
#### Post date: [January 16, 2023, 2:23pm UTC](https://discourse.itk.org/t/need-guidance-to-address-the-regression-error/5569/13 "2023-01-16T14:23:18Z")
> [@parth721](#):
>
> STEP 1 : I need to perform the ctest and test failure tell me what is the value of `maximumNumberOfIteration`.
>
> STEP 2 : Assign the value to `maximumNumberOfIteration`
This works for outputs, not inputs!
The main point of [Make the `itkMultiLabelSTAPLEImageFilterTest` quantitative · Issue #3657 · InsightSoftwareConsortium/ITK · GitHub](https://github.com/InsightSoftwareConsortium/ITK/issues/3657) is to regression-check the outputs, so if an input (maximumNumberOfIteration) changes from 100 to 0 the change will be detected via test errors.
---
### Author:  [@parth721](https://discourse.itk.org/u/parth721)
#### Post date: [January 20, 2023, 12:49am UTC](https://discourse.itk.org/t/need-guidance-to-address-the-regression-error/5569/14 "2023-01-20T00:49:29Z")
Good morning/afternoon/evening/night 😃
1.I tried the test syntax like :
`itk_add_test(NAME itkMulti... COMMAND ITKLabelV... itkMulti... )`
by looking at the “Test command”. But it doesn’t work.
2. Then checkout this approach
- if I create a content link of baseline image from kitware, does it can check `MaximumNumberOfIterator`?
- should I go with python to build the image with given matrices?
- is the file type is correct or I need to convert them into .mha/.png?
- what 5 100 0 at the end of `${ITK_TEST_OUTPUT_DIR}` how they are decided?
- should I try out other macros like`ITK_TRY_EXCEPT_NO_EXCEPTION()` or something else & put if `MaximumNumberOfIterator = 0` give error.
thanks ✌
---
### Author:  [@dzenanz](https://discourse.itk.org/u/dzenanz)
#### Post date: [January 20, 2023, 2:48pm UTC](https://discourse.itk.org/t/need-guidance-to-address-the-regression-error/5569/15 "2023-01-20T14:48:41Z")
The test should have [argc/argv parameters](https://github.com/InsightSoftwareConsortium/ITK/blob/v5.3.0/Modules/Segmentation/LabelVoting/test/itkVotingBinaryImageFilterTest.cxx#L84-L100) which get translated into internal parameters which are currently constants, such as `maximumNumberOfIterations`.
To get an output image which can be regression tested against a baseline, it needs to be [written](https://github.com/InsightSoftwareConsortium/ITK/blob/v5.3.0/Modules/Segmentation/LabelVoting/test/itkVotingBinaryImageFilterTest.cxx#L72) first. But that is not really necessary in this case, as the inputs are constant, and the outputs are [“manually” checked](https://github.com/InsightSoftwareConsortium/ITK/blob/v5.3.0/Modules/Segmentation/LabelVoting/test/itkMultiLabelSTAPLEImageFilterTest.cxx#L172-L182).
I would like @jhlegarreta to pitch in here.