SimpleITK memory access violation when using large images.

Beginner Questions
1

Not really sure if this is a beginner question, but at least it’s my first question. I’m having problems when using newer versions of SimpleITK with relatively large images (500M+ voxels), it causes memory access violations with almost all filters I try to use.

We use SimpleITK for a software we are developing but we haven’t been able to upgrade SimpleITK for a long time because of this issue, the version we currently use I complied myself in 2016 and we have never seen this issue with that version. I have complied ITK/SimpleITK recently but the result has the same memory access violation problem. (I’m using the C# wrapper)

Any ideas?

2

Are you sure you’ve done a 64 bit build?

3

Yes, I get the same error with my own 64-bit build and the pre-built x64 linked from the sitk website. We get this error on multiple machines and in tests outside our software as well, we have only tested the windows binaries though.

Creating the image object works well but it crashes when we use it with a filter, 100% reproducible.

4

Hello mattias,

When you build SimpleITK yourself, you say you are using MS Visual Studio 2016 and what CMake options do you set?

Additionally, can you please enable testing in your compilation and run the test suite to see if the tests suit passes on you configuration.

5

Unfortunately I don’t have the settings I used for the 2016 build that’s working, but so far I have tried many settings both with ITK and SITK to try and get a build that fixes this problem. I can get back with specifics later this week.

It’s unfortunate since we really want some new functionality introduced in the last couple of stable builds of SITK. If you want to try it out yourself just use the x64 dll’s distributed via the webpage. Create a float32 image with dimensions like 1000x1000x580 and run it trough a gaussian filter, most times you will get an access violation in windows.

6

I just ran the following script on a Windows system, and it ran fine

import SimpleITK as sitk

img = sitk.GaussianSource(sitk.sitkFloat32, [1000,1000,1000], [200,200,200], [500,500,500], 200)

img2 = sitk.Derivative(img, 0, 1)

sitk.Show(img2)
sitk.WriteImage(img2, "big_derivative.mha")

The image it created was 4 gig, and the process used 8 gig of memory.

7

When using the C# library did you follow the steps here:
https://simpleitk.readthedocs.io/en/master/Documentation/docs/source/setUp.html#building-an-example

Specifically what versions of the distributed library is working for you and what version is not?

8

The version we are using that has worked since 2016 is 0.10.0.dev265-gf2793 (built locally), and we have about 70 or so filters implemented in the software without any problems over the years. I have tried sitk releases periodically but I keep getting this problem, I specifically remember trying 1.1 and now 1.2 of the distributed library.

First I thought it was an issue with threading since the image is created on a different thread than the filter runs in, but if I run everything in the same thread I still get the same error.

This is the exact exception I get from the 1.2.0 distributed x64 dll.

Exception thrown: ‘System.AccessViolationException’ in SimpleITKCSharpManaged.dll
An unhandled exception of type ‘System.AccessViolationException’ occurred in SimpleITKCSharpManaged.dll
Attempted to read or write protected memory. This is often an indication that other memory is corrupt.

Usually the debugger in VS can’t even catch the exception.

I haven’t investigated if it is related to the image being large or if it is just a random error that is more likely to occur when using a large image, but it seems to me it has to do with the image using a lot of memory.

9

Ok, so I think that I finally figured out a part of the problem. In the software I convert image formats back and forth between the internal image format and vtk, itk images. To speed this up I use code like this to convert the internal float array to an itk image.

private static Image ToItkImage(Image4DFloat image, int frame)
{
    GCHandle pinned = GCHandle.Alloc(image.GetArray(frame), GCHandleType.Pinned);
    IntPtr address = pinned.AddrOfPinnedObject();
    Image result = SimpleITK.ImportAsFloat(address,
                                           new VectorUInt32(new uint[] { (uint)image.SizeX, (uint)image.SizeY, (uint)image.SizeZ }),
                                           new VectorDouble(new double[] { image.VoxelSizeX, image.VoxelSizeY, image.VoxelSizeZ }),
                                           new VectorDouble(image.Position.ToArray()),
                                           new VectorDouble(image.Orientation.ToArrayColumnMajor()));
    pinned.Free();
    return result;
}

The method image.GetArray(frame) returns a new float[] containing the voxel values for the selected frame and is pinned. The pinned array is released after the import and becomes eligible for garbage collection.

This all worked fine in our old build of sitk but it seems the temporary array is causing problems in the newer versions (when the temporary array is garbage collected probably?), so I guess there could be a change in the inner workings of sitk.ImportAsFloat() that’s causing this?

Anyway, this is something that is easily fixed but it would still be interesting to know what exactly caused this behavior change in sitk, or if it was pure luck that this even worked in the old version?.

Thanks for the response!

1 Like
10

This looks like the change:

It looks like the commit message, and class documentation make it clear that the image shares the buffer and no copy is done.

To make a deep copy of the image it should be something like:

Image shallow_result = SimpleITK.ImportAsFloat(...);
Image result = new Image(shallow_result); // SimpleITK lazy copy done
result.MakeUnique(); // actually deep copy of buffer
2 Likes
11

I see, that explains it, but it seems that there must have been a change that made the buffer eligible for garbage collection even when referenced in the new image, since we never had any issues with this approach in the older version.

Anyway, thanks for the help!

12

Before the “bug” fix, the buffer passed to ImportAsFloat was copied, with the current “fixed” and documented method there is not copy. So in the old implementation you were relying upon there was not need for an c# reference count the sitk::Image’s buffer was created an owned by itself.

13

Well I’m glad I finally got to the bottom of this and can start keeping up with new versions in our software.

1 Like