MetaIO precision

Engineering
1

I dug into this numeric change in the test a little further. It was narrowed down for a MetaIO change from float to double internally.

One of the failing SimpleITK test is with the LabelIntensityStatisticsImageFilter:

/home/circleci/SimpleITK-build/SimpleITK-build/Testing/Unit/sitkLabelIntensityStatisticsImageFilterTest.cxx:209: Failure
The difference between 3004.1542777485 and filter.GetPhysicalSize( 1 ) is 0.00020916997527820058, which exceeds 1e-08, where
3004.1542777485 evaluates to 3004.1542777485001,
filter.GetPhysicalSize( 1 ) evaluates to 3004.1544869184754, and
1e-08 evaluates to 1e-08.

Does the change of ~2e-4 seem excessive for such a simple computation as the PhysicalSize (# of pixel X voxel size ). So I looked into it a bit further.

The input file has the following header:

TransformMatrix = 1 0 0 1
Offset = 0 0
CenterOfRotation = 0 0
ElementSpacing = 0.352778 0.352778
DimSize = 256 256

With SimpleITK 1.0.1 (ITK 4.12) I get the following when reading the image:

In [6]: print img.GetSpacing()
(0.35277798771858215, 0.35277798771858215)

While SimpleITK with ITK 4.13rc02 I get the following:

In [4]: print img.GetSpacing()
(0.352778, 0.352778)

The later ITK 4.13 is clearly more accurate to what is in the file, the difference is ~1.2e-8. This is inline with float numbers having about 8 digits of accuracy.

The object of this test has about 24,000 pixels so the error is 24000*1.2e-8=0.000288. Which about what the test was reporting.

It is interesting to see the way inaccuracy, differences or portability of the image meta-information can effect results.

1 Like
2

Out of curiosity, did you increase the epsilon of make the test use the average per pixel error instead of total error?

3

@blowekamp I am not sure if you realized but there is a way to preserve 100% legacy compat, see the original PR:

http://review.source.kitware.com/#/c/22847/

Let me know if this works for you

4

@dzenanz For this test I I updated the base line value. I the presumption is that the new behavior is more accurate and “correct”. It will cause the test to fail if an older version of ITK is used, but the generally policy of SimpleITK is that the SuperBuild version of ITK represents the minimum expected version of ITK. So this is OK.

@mathieu.malaterre That is useful option to enable a specific test to pass, or enforce compatibility for a specific tool. However, what is the “correct” behavior? What should the default behavior be of package such as SimpleITK? Ideally, one does not like the values in their files to change overtime. If it would be possible for the file its self to contain the information of “precision” that would be the ideal case. If that was implemented ( defaulting to the legacy value ), then existing files would have the old behavior and new files would benefit from increased preserved precision.

1 Like
5

@blowekamp Thanks for the feedback. I am not sure I understand your question. In particular what do you mean by “contain the information of precision”. By definition this is just a matter of counting the number of digits after the dot symbol to deduce the “precision”. Would you like to have an API to query this ? I suspect this is non-trivial since it will need to go through the MetaImage API.

As posted before, please comment on the actual API explaining what is missing. Put the existing API in comparison with something you would like to see instead.

1 Like
6

My questions are around “What is correct behavior?”, should a user have to manually specify the the precision if they want the same file information they had before? It’s seems rather tedious to do that and not practical.

Specifically, my suggestion to consider for ideal behavior is to modify the MetaIO library to:

  • All new MetaIO files would write an additional “precision” field to files, the current value would be the new higher precision (17?).

  • When a file is read the “precision” would default to the legacy 6, if it does not exist in the file. If it does exist in the file that is the value that would be used.

2 Likes
7

What about .mha images that were not written using MetaIO library, and already have higher precision of fields? Were they already treated as having lower precision by being parsed into floats? Or would this break the backwards compatibility for such (probably rare) images?

8

@matt.mccormick (or other Discourse admins): It seems that a new discussion (MetaIO behavior) is started from the original topic (release notes). It would be useful to move all messages related to MetaIO discussion to a separate topic.

2 Likes
9

This seems like a good suggestion. The most convenient from the aspect of backwards compatibility.