# How to assert that images are equal, in Python unit test?

**URL:** https://discourse.itk.org/t/how-to-assert-that-images-are-equal-in-python-unit-test/7302
**Category:** Engineering
**Created:** [November 6, 2024, 4:19pm UTC](https://discourse.itk.org/t/how-to-assert-that-images-are-equal-in-python-unit-test/7302 "2024-11-06T16:19:55Z")
**Posts on this page:** 20
**Page:** 1

<div class="post-metadata">

### Author: ![Niels\_Dekker](https://discourse.itk.org/letter_avatar_proxy/v4/letter/n/9d8465/32.png) [@Niels\_Dekker](https://discourse.itk.org/u/Niels_Dekker)
#### Post date: [November 6, 2024, 4:19pm UTC](https://discourse.itk.org/t/how-to-assert-that-images-are-equal-in-python-unit-test/7302/1 "2024-11-06T16:19:55Z")

</div>

How would you check that the “actual” output image of a filter is equal to the “excepted” image, in a Python unit test? Is there a helper function, like `assert_equal_image(actual_image, expected_image)`? I tried the following:

```auto
class MyTestCase(unittest.TestCase):

    def _assert_equal_image(self, actual_image, expected_image) -> None:
        """Asserts that the actual image is equal to the expected one."""
        self.assertTrue(actual_image.IsSameImageGeometryAs(expected_image))
        self.assertEqual(
            actual_image.GetBufferedRegion(), expected_image.GetBufferedRegion()
        )
        region = expected_image.GetBufferedRegion()
        region_index = region.GetIndex()
        region_size = region.GetSize()

        for pixel_index in itertools.product(
            range(region_index[0], region_index[0] + region_size[0]),
            range(region_index[1], region_index[1] + region_size[1]),
        ):
            self.assertEqual(
                actual_image.GetPixel(pixel_index), expected_image.GetPixel(pixel_index)
            )

```

My attempt may be nice (hopefully), but it isn’t complete (no 3D support yet, for example), and I’m afraid of re-inventing the wheel… any suggestion?

---

<div class="post-metadata">

### Author: ![phcerdan](https://discourse.itk.org/user_avatar/discourse.itk.org/phcerdan/32/286_2.png) [@phcerdan](https://discourse.itk.org/u/phcerdan)
#### Post date: [November 6, 2024, 4:33pm UTC](https://discourse.itk.org/t/how-to-assert-that-images-are-equal-in-python-unit-test/7302/2 "2024-11-06T16:33:09Z")

</div>

You can convert image to numpy array (`itk.array_view_from_image`) and use [np.testing.assert\_array\_equal](https://numpy.org/doc/stable/reference/generated/numpy.testing.assert_array_equal.html)

---

<div class="post-metadata">

### Author: ![Niels\_Dekker](https://discourse.itk.org/letter_avatar_proxy/v4/letter/n/9d8465/32.png) [@Niels\_Dekker](https://discourse.itk.org/u/Niels_Dekker)
#### Post date: [November 7, 2024, 10:18am UTC](https://discourse.itk.org/t/how-to-assert-that-images-are-equal-in-python-unit-test/7302/3 "2024-11-07T10:18:44Z")

</div>

Thank you very much @phcerdan So something like this, right?

```python
def _assert_equal_image(self, actual_image, expected_image) -> None:
    """Asserts that the actual image is equal to the expected one."""
    self.assertTrue(actual_image.IsSameImageGeometryAs(expected_image))
    np.testing.assert_array_equal(
        actual=itk.array_view_from_image(actual_image),
        desired=itk.array_view_from_image(expected_image),
        strict=True,
    )

```

---

<div class="post-metadata">

### Author: ![blowekamp](https://discourse.itk.org/user_avatar/discourse.itk.org/blowekamp/32/79_2.png) [@blowekamp](https://discourse.itk.org/u/blowekamp)
#### Post date: [November 7, 2024, 1:38pm UTC](https://discourse.itk.org/t/how-to-assert-that-images-are-equal-in-python-unit-test/7302/4 "2024-11-07T13:38:34Z")

</div>

I generally end up having to use [assert\_approx\_equal](https://numpy.org/doc/2.0/reference/generated/numpy.testing.assert_approx_equal.html) for all but trivial operations. Also do you need to check that the images are the same type?

---

<div class="post-metadata">

### Author: ![Niels\_Dekker](https://discourse.itk.org/letter_avatar_proxy/v4/letter/n/9d8465/32.png) [@Niels\_Dekker](https://discourse.itk.org/u/Niels_Dekker)
#### Post date: [November 7, 2024, 2:50pm UTC](https://discourse.itk.org/t/how-to-assert-that-images-are-equal-in-python-unit-test/7302/5 "2024-11-07T14:50:47Z")

</div>

Thanks for the suggestion. Here is the use case that I have in mind: [ENH: Python dispatch on the first RequiredInputName by thewtex · Pull Request #4921 · InsightSoftwareConsortium/ITK · GitHub](https://github.com/InsightSoftwareConsortium/ITK/pull/4921#discussion_r1832488571) As follows:

```auto
    assert_equal_image(
        itk.median_image_filter(primary=image),
        itk.median_image_filter(image),
    )

```

So it would test that in Python, a keyword argument (primary=image) would give the same result as a positional argument (image).

The unit test may use a synthetic input image, so that both function calls would produce _exactly_ the same image.

---

<div class="post-metadata">

### Author: ![matt.mccormick](https://discourse.itk.org/user_avatar/discourse.itk.org/matt.mccormick/32/7_2.png) [@matt.mccormick](https://discourse.itk.org/u/matt.mccormick)
#### Post date: [November 7, 2024, 3:37pm UTC](https://discourse.itk.org/t/how-to-assert-that-images-are-equal-in-python-unit-test/7302/6 "2024-11-07T15:37:35Z")

</div>

Hi,

A more complete way to compare images is with `itk.comparison_image_filter`:

```python
comparison = itk.comparison_image_filter(
    itk.median_image_filter(primary=image),
    itk.median_image_filter(image),
    verify_input_information=True
)
assert np.sum(comparison) == 0.0

```

This will also check the image metadata.

For general usage, another option for this purpose is the [`compare_images`](https://insightsoftwareconsortium.github.io/ITK-Wasm/compare-images/py/docs/apidocs/itkwasm_compare_images/itkwasm_compare_images.compare_images.html) function from the [`itkwasm-compare-images` package](https://pypi.org/project/itkwasm-compare-images/).

This looks like:

```python
from itkwasm_compare_images import compare_images
from itkwasm import Image

test_image = itk.median_image_filter(primary=image)
# Convert itk.Image to itkwasm.Image
test_image = Image(**itk.dict_from_image(test_image))

baseline_image = itk.median_image_filter(image)
baseline_image = Image(**itk.dict_from_image(baseline_image))

metrics, difference_image, difference_image_rendering = compare_images(
    test_image,
    baseline_images=[baseline_image,],
)
assert metrics['almostEqual']

```

---

<div class="post-metadata">

### Author: ![Niels\_Dekker](https://discourse.itk.org/letter_avatar_proxy/v4/letter/n/9d8465/32.png) [@Niels\_Dekker](https://discourse.itk.org/u/Niels_Dekker)
#### Post date: [November 7, 2024, 4:56pm UTC](https://discourse.itk.org/t/how-to-assert-that-images-are-equal-in-python-unit-test/7302/7 "2024-11-07T16:56:25Z")

</div>

> [@matt.mccormick](#):
>
> A more complete way to compare images is with `itk.comparison_image_filter`:
> 
> ```auto
> comparison = itk.comparison_image_filter(
> itk.median_image_filter(primary=image),
> itk.median_image_filter(image),
> verify_input_information=True
> )
> assert np.sum(comparison) == 0.0
> 
> ```
> 
> This will also check the image metadata.

Cool! Thanks Matt! Can you please explain, what is `comparison`? Is it also an image?

---

<div class="post-metadata">

### Author: ![dzenanz](https://discourse.itk.org/user_avatar/discourse.itk.org/dzenanz/32/1093_2.png) [@dzenanz](https://discourse.itk.org/u/dzenanz)
#### Post date: [November 7, 2024, 5:52pm UTC](https://discourse.itk.org/t/how-to-assert-that-images-are-equal-in-python-unit-test/7302/8 "2024-11-07T17:52:32Z")

</div>

> `comparison = ...`

it is a variable 😃

---

<div class="post-metadata">

### Author: ![dzenanz](https://discourse.itk.org/user_avatar/discourse.itk.org/dzenanz/32/1093_2.png) [@dzenanz](https://discourse.itk.org/u/dzenanz)
#### Post date: [November 7, 2024, 5:53pm UTC](https://discourse.itk.org/t/how-to-assert-that-images-are-equal-in-python-unit-test/7302/9 "2024-11-07T17:53:23Z")

</div>

Aha, yes, I think it is an image where each pixel contains the difference between the two input images.

---

<div class="post-metadata">

### Author: ![Niels\_Dekker](https://discourse.itk.org/letter_avatar_proxy/v4/letter/n/9d8465/32.png) [@Niels\_Dekker](https://discourse.itk.org/u/Niels_Dekker)
#### Post date: [November 7, 2024, 7:45pm UTC](https://discourse.itk.org/t/how-to-assert-that-images-are-equal-in-python-unit-test/7302/10 "2024-11-07T19:45:03Z")

</div>

Thanks @dzenanz Would `comparison` just be the difference, or the _absolute_ difference of the two images?

If `itk.comparison_image_filter` creates a new image, I guess it’s more expensive than calling `np.testing.assert_array_equal` on the views returned by `itk.array_view_from_image` (as was suggested by @phcerdan), right?

---

<div class="post-metadata">

### Author: ![matt.mccormick](https://discourse.itk.org/user_avatar/discourse.itk.org/matt.mccormick/32/7_2.png) [@matt.mccormick](https://discourse.itk.org/u/matt.mccormick)
#### Post date: [November 7, 2024, 9:55pm UTC](https://discourse.itk.org/t/how-to-assert-that-images-are-equal-in-python-unit-test/7302/11 "2024-11-07T21:55:13Z")

</div>

`comparison` is an image with the signed difference between pixels.

There is a cost to create the difference image, but this is not prohibitive in general.

And it tests more than the pixel arrays.

There are also [more options, if needed](https://itk.org/Doxygen/html/classitk_1_1Testing_1_1ComparisonImageFilter.html).

---

<div class="post-metadata">

### Author: ![Niels\_Dekker](https://discourse.itk.org/letter_avatar_proxy/v4/letter/n/9d8465/32.png) [@Niels\_Dekker](https://discourse.itk.org/u/Niels_Dekker)
#### Post date: [November 7, 2024, 10:02pm UTC](https://discourse.itk.org/t/how-to-assert-that-images-are-equal-in-python-unit-test/7302/12 "2024-11-07T22:02:01Z")

</div>

Thanks for explaining, @matt.mccormick.

> [@matt.mccormick](#):
>
> `comparison` is an image with the signed difference between pixels.

OK, so then `np.sum(comparison)` may have summarized positive and negative pixel differences together, when it returns zero?

---

<div class="post-metadata">

### Author: ![Niels\_Dekker](https://discourse.itk.org/letter_avatar_proxy/v4/letter/n/9d8465/32.png) [@Niels\_Dekker](https://discourse.itk.org/u/Niels_Dekker)
#### Post date: [November 8, 2024, 12:19pm UTC](https://discourse.itk.org/t/how-to-assert-that-images-are-equal-in-python-unit-test/7302/13 "2024-11-08T12:19:22Z")

</div>

> [@matt.mccormick](#):
>
> There are also [more options, if needed](https://itk.org/Doxygen/html/classitk_1_1Testing_1_1ComparisonImageFilter.html).

Thanks for [the link](https://itk.org/Doxygen/html/classitk_1_1Testing_1_1ComparisonImageFilter.html)! I see, the documentation does not say that the image created by ComparisonImageFilter has absolute values. While this is essential if you want to do `np.sum(comparison)`.

---

<div class="post-metadata">

### Author: ![blowekamp](https://discourse.itk.org/user_avatar/discourse.itk.org/blowekamp/32/79_2.png) [@blowekamp](https://discourse.itk.org/u/blowekamp)
#### Post date: [November 8, 2024, 1:04pm UTC](https://discourse.itk.org/t/how-to-assert-that-images-are-equal-in-python-unit-test/7302/14 "2024-11-08T13:04:42Z")

</div>

> [@Niels\_Dekker](#):
>
> The unit test may use a synthetic input image, so that both function calls would produce _exactly_ the same image.

If all you need is exact comparison then you may want to look at the [HashImageFilter](https://itk.org/Doxygen/html/classitk_1_1Testing_1_1HashImageFilter.html). This filter generates an MD5 hash from the image’s buffer which can be used for efficient comparison.

---

<div class="post-metadata">

### Author: ![Niels\_Dekker](https://discourse.itk.org/letter_avatar_proxy/v4/letter/n/9d8465/32.png) [@Niels\_Dekker](https://discourse.itk.org/u/Niels_Dekker)
#### Post date: [November 8, 2024, 1:36pm UTC](https://discourse.itk.org/t/how-to-assert-that-images-are-equal-in-python-unit-test/7302/15 "2024-11-08T13:36:26Z")

</div>

> [@blowekamp](#):
>
> If all you need is exact comparison then you may want to look at the [HashImageFilter](https://itk.org/Doxygen/html/classitk_1_1Testing_1_1HashImageFilter.html)

Nice suggestion, thank Bradley. I guess such hash values would especially be useful when having large test images.

For unit tests, I would prefer to use small images, when possible, having around 16x16 pixels, for example. So for such small images, a simple straightforward “assert\_array\_equal” on the pixel data should be just fine, in my opinion.

I think it would be nice if `itk.Image` would support value-based equality comparison by `image1 == image2`. Just like Python lists:

```python
list1 = [1, 2, 3]
list2 = [1, 2, 3]
assert list1 == list2 # OK, list1 == list2 is True :-)

```

What do you think?

---

<div class="post-metadata">

### Author: ![blowekamp](https://discourse.itk.org/user_avatar/discourse.itk.org/blowekamp/32/79_2.png) [@blowekamp](https://discourse.itk.org/u/blowekamp)
#### Post date: [November 8, 2024, 2:03pm UTC](https://discourse.itk.org/t/how-to-assert-that-images-are-equal-in-python-unit-test/7302/16 "2024-11-08T14:03:31Z")

</div>

There is ambiguity on if these comparison operators should be “broadcast” per pixel, or comparison on the whole image. Numpy has the following:

```auto

In [2]: a = np.array(range(4)); b = np.array(range(4))

In [3]: a == b
Out[3]: array([True, True, True, True])

```

SimpleITK has these comparison operators overloaded as well. You can experiment with them there as well. In C++ they are an “optional” header, and work well with the move operator too.

> <https://github.com/SimpleITK/SimpleITK/blob/master/Code/BasicFilters/include/sitkImageOperators.h>

---

<div class="post-metadata">

### Author: ![matt.mccormick](https://discourse.itk.org/user_avatar/discourse.itk.org/matt.mccormick/32/7_2.png) [@matt.mccormick](https://discourse.itk.org/u/matt.mccormick)
#### Post date: [November 8, 2024, 8:45pm UTC](https://discourse.itk.org/t/how-to-assert-that-images-are-equal-in-python-unit-test/7302/17 "2024-11-08T20:45:36Z")

</div>

> [@Niels\_Dekker](#):
>
> OK, so then `np.sum(comparison)` may have summarized positive and negative pixel differences together, when it returns zero?

Yes, this is technically possible.

---

<div class="post-metadata">

### Author: ![matt.mccormick](https://discourse.itk.org/user_avatar/discourse.itk.org/matt.mccormick/32/7_2.png) [@matt.mccormick](https://discourse.itk.org/u/matt.mccormick)
#### Post date: [November 8, 2024, 8:53pm UTC](https://discourse.itk.org/t/how-to-assert-that-images-are-equal-in-python-unit-test/7302/18 "2024-11-08T20:53:02Z")

</div>

> [@matt.mccormick](#):
>
> Yes, this is technically possible.

And, if it is a concern, `np.sum(np.abs(comparison))` can also be used.

---

<div class="post-metadata">

### Author: ![Niels\_Dekker](https://discourse.itk.org/letter_avatar_proxy/v4/letter/n/9d8465/32.png) [@Niels\_Dekker](https://discourse.itk.org/u/Niels_Dekker)
#### Post date: [November 8, 2024, 9:06pm UTC](https://discourse.itk.org/t/how-to-assert-that-images-are-equal-in-python-unit-test/7302/19 "2024-11-08T21:06:16Z")

</div>

Looking at `ComparisonImageFilter::ThreadedGenerateData`:

> <https://github.com/InsightSoftwareConsortium/ITK/blob/53750566cc8f0265ff189390939c23000391555d/Modules/Core/TestKernel/include/itkTestingComparisonImageFilter.hxx#L168-L172>

Maybe it _does_ take the absolute value of the pixel-wise differences. But if it does, apparently it’s an undocumented feature 🤷

---

<div class="post-metadata">

### Author: ![blowekamp](https://discourse.itk.org/user_avatar/discourse.itk.org/blowekamp/32/79_2.png) [@blowekamp](https://discourse.itk.org/u/blowekamp)
#### Post date: [November 11, 2024, 2:08pm UTC](https://discourse.itk.org/t/how-to-assert-that-images-are-equal-in-python-unit-test/7302/20 "2024-11-11T14:08:01Z")

</div>

The ComparisonImageFilter class also has the following methods to get the Min, Max, Mean, and Total difference.  
[https://itk.org/Doxygen/html/classitk\_1\_1Testing\_1\_1ComparisonImageFilter.html#a49803d4d6781b90f19aa21f5c0d171fe](https://itk.org/Doxygen/html/classitk_1_1Testing_1_1ComparisonImageFilter.html#a49803d4d6781b90f19aa21f5c0d171fe)

So you may not need to use numpy to compute the sum.

[Next page](https://discourse.itk.org/t/how-to-assert-that-images-are-equal-in-python-unit-test/7302.md?page=2)