long run-time issues medical image registration

Hello,

I am a beginner at coding in python and using Simple ITK.
I am doing a non-rigid registration of 2 set of CT from differents patients and I am facing severals issues…

The first one is the error : “Failed to allocate memory for image”
indeed, i can not select the whole directory which contains my CT (around 600 CT of 500 ko)
But if i select only a few CT from the directory, it works (around 200)

The second thing is the run time.
Indeed, the code takes 1h30 to run and show the registration.
I changed some parameters like “number of iteration” or “convergence window” to reduce the run-time but nothing… still takes too long
How I can reduce this run-time ? Do I have to change the parameter or my code is not good itself ?

Can someone please help …

The code i am using is the following :

sitk registration 2.py (4.0 KB)

(if you cannot click on the link, here s the code)

from __future__ import print_function

import SimpleITK as sitk
import sys
import os

data_directory = "E:/document/_11F06DD_Y5_M6_P50_Copie"
series_IDs = sitk.ImageSeriesReader.GetGDCMSeriesIDs(data_directory)
if not series_IDs:
    print("ERROR: given directory \"" + data_directory + "\" does not contain a DICOM series.")
    sys.exit(1)
series_file_names = sitk.ImageSeriesReader.GetGDCMSeriesFileNames(data_directory, series_IDs[0])

data_directory2 = "E:/Document/ctpatient"
series_IDs2 = sitk.ImageSeriesReader.GetGDCMSeriesIDs(data_directory2)
if not series_IDs2:
    print("ERROR: given directory \"" + data_directory2 + "\" does not contain a DICOM series.")
    sys.exit(1)
series_file_names2 = sitk.ImageSeriesReader.GetGDCMSeriesFileNames(data_directory2, series_IDs2[0])


def command_iteration(method, bspline_transform):
    if method.GetOptimizerIteration() == 0:
              print(bspline_transform)


    print("{0:3} = {1:10.5f}".format(method.GetOptimizerIteration(), method.GetMetricValue()))



def command_multi_iteration(method):
    if R.GetCurrentLevel() > 0:
        print("Optimizer stop condition: {0}".format(R.GetOptimizerStopConditionDescription()))
        print(" Iteration: {0}".format(R.GetOptimizerIteration()))
        print(" Metric value: {0}".format(R.GetMetricValue()))

    print("--------- Resolution Changing ---------")


fixed = sitk.ReadImage(series_file_names, sitk.sitkFloat32)
moving = sitk.ReadImage(series_file_names2, sitk.sitkFloat32)

transformDomainMeshSize = [2]*fixed.GetDimension()
tx = sitk.BSplineTransformInitializer(fixed, transformDomainMeshSize)

print("Initial Number of Parameters: {0}".format(tx.GetNumberOfParameters()))

R = sitk.ImageRegistrationMethod()
R.SetMetricAsJointHistogramMutualInformation()

R.SetOptimizerAsGradientDescentLineSearch(50,
                                          100,
                                          convergenceMinimumValue=1e-2,
                                          convergenceWindowSize=3)

R.SetInterpolator(sitk.sitkLinear)

R.SetInitialTransformAsBSpline(tx,
                               inPlace=True,
                               scaleFactors=[1,2,5])
R.SetShrinkFactorsPerLevel([4,2,1])
R.SetSmoothingSigmasPerLevel([4,2,1])

R.AddCommand(sitk.sitkIterationEvent, lambda: command_iteration(R, tx))
R.AddCommand(sitk.sitkMultiResolutionIterationEvent, lambda: command_multi_iteration(R))

R_finale = R.Execute(fixed, moving)

print("-------")
print(tx)
print(R_finale)
print("Optimizer stop condition: {0}".format(R.GetOptimizerStopConditionDescription()))
print(" Iteration: {0}".format(R.GetOptimizerIteration()))
print(" Metric value: {0}".format(R.GetMetricValue()))

sitk.WriteTransform(R_finale, 'W:/DICOMresultats/resultats.txt')

if ( not "SITK_NOSHOW" in os.environ ):

    resampler = sitk.ResampleImageFilter()
    resampler.SetReferenceImage(fixed);
    resampler.SetInterpolator(sitk.sitkLinear)
    resampler.SetDefaultPixelValue(100)
# get the transformation
    resampler.SetTransform(R_finale)
# transform and resample moving image
    out = resampler.Execute(moving)

# rescaling image intensities
    simg1 = sitk.Cast(sitk.RescaleIntensity(fixed), sitk.sitkUInt8)
    simg2 = sitk.Cast(sitk.RescaleIntensity(out), sitk.sitkUInt8)
    cimg = sitk.Compose(simg1, simg2, simg1//2.+simg2//2.)
    sitk.Show(cimg, "Image Registration Composition")

Hello @moeez,

Welcome to SimpleITK!

Several features in your current implementation can be improved:

  1. Similarity metric: You are performing intra-modal registration (CT/CT) so you can use a simpler similarity metric SetMetricAsMeanSquares or SetMetricAsCorrelation . This should speed things up.
  2. Transform: Your BSpline control grid is very coarse (2x2x2 over the whole image), you will probably need a finer grid to get reasonable deformations. Do you need a Free Form Deformation (FFD), or will affine be good enough? Are your images close enough to each other that you only need to start with the FFD (implicitly you have an identity global transformation), or do you need some global intialization?
  3. Similarity metric evaluation: You should probably not use all the voxels in your images to evaluate the metric. Use the SetMetricSamplingStrategy with regular or random sampling. This will significantly speed things up (in some cases you can use 1% of the data and obtain good registration).

Before you continue, I suggest you familiarize yourself with the SimpleITK registration framework by going over our Jupyter notebook examples. The notebooks starting with 6* illustrate the registration features and basic strategies.

3 Likes

Thank you @zivy

I’ll try to modifi my code thanks to your info and see what happen

yeah, i am currently reading the jupyter notebook and try to get familiar with it

thank you

yes, the 2 images are close enough so i think i ll try the FFD transformation.
I don t need a global initialisation…

Hi @zivy

Can you explain me a bit about the Transform ?
Like, when, in which case, I should use which one ? What s the difference between the different Transform ?

I tried to understand that thanks to the Jupyter notebook but didn t get it…

Moeez

Hello @moeez,

The choice of which transformation to use depends on your model of the world. The more constraints you can use, the fewer degrees of freedom and the easier the problem (a stronger prior on the solution space). As you are working with a set of head CT’s from different individuals I would start with an affine transformation. This allows for changes in pose (rigid transformation), scale and shearing.

Then evaluate if the results are good enough for your task. If not, then you probably need more degrees of freedom, use a FFD or if that isn’t enough use a DeformationField transform.

The rationale for this process is that you want to use the simplest model (Occam’s Razor). If a model does not fit the world accurately enough for the task, increase its complexity (the degrees of freedom).

1 Like

Hmmm OK thank you very much.

I did want you said and it works very good. I used FFD, my registrated images are good and the run-time is about 5 min, which is way better then 1h30

How about the other Error message : “Failed to allocate memory for image”
it happend when i select the directory with all the CT (around 600 CT in total)
To avoid this error i have to select only some of the CT, like around 150, which is not for my work

"RuntimeError: Exception thrown in SimpleITK ImageRegistrationMethod_Execute: d:\a\1\work\b\itk-prefix\include\itk-4.13\itkImportImageContainer.hxx:199:
Failed to allocate memory for image."

The thing is that the algorithm run well on my laptop which is an i5 processor and has 16Go RAM and on an other computer which is a i7 processor and has 32Go RAM
The one is use is the following :
Intel Xeon E5-16030
32 Go RAM - 64bit
Is the porblem comes from the computer ?

Moeez

Assuming your CT is 512x512x600 and you are reading it as float (4 bytes per pixel) it should fit in memory. The simplest test is to see if you can create an image of the desired size:

import SimpleITK as sitk
sitk.Image([512,512,600], sitk.sitkFloat32)

And use system monitoring to see how much memory this consumes (unix: top, mac: Activity Monitor or top, win: Resource Monitor).

If this works, then the number of slices in your series may be greater than 600. What is the length of the series_file_names list?

I did the test and it works. It took nothing in term of memory…
The different bench of CT I am using are respectively :
443x209x580
485x245x624
536x536x419

The length of series_file_names correspond of the number of CT in the directory

print(len(series_file_names))
580

so i still don t know from where the error come from…

Ok, so it isn’t a memory issue as you were able to allocate an image that has the desired size. What I find interesting is that your CT scans seem to have very strange dimensions in the x and y axes. From commercial machines in clinical use these are usually 512x512, but definitely not 443x209 or 485x245, 536x536.

Were these images processed by some other software and cropped? Possibly something is problematic with one or more of the files
(corrupted for some reason). The next step in debugging is to try and read each of the images independently. Something like:

import glob
import SimpleITK as sitk

file_names = glob.glob('*.dcm')
for f in file_names: 
   sitk.ReadImage(f)

Please let us know if this exposes an issue or provide one of your failing datasets so that we can better identify the problem.

You could also try to read the DICOM series using 3D Slicer, and if it is problematic try DICOM patcher on it.

Ok so let me explain the strange dimensions.
These are indeed images processed from some other software : they are whole Body Phantom made by computer. While making them, we adjusted the size of the field of view to not have some aberrant CT, that s why they are different dimensions in x and y axes. So they are artificial CT

About the code you gave me to try, it works fine; there is no error showing up…
“Process finished with exit code 0”
So apparently, it can read all the 580 CT

Futhermore, i can read all of them on other softwere like 3D slicer or ImageJ or dicompyler
I just don t know why i cannot in my python s code?

Based on the information up to now it looks like none of the files is corrupted, you have enough memory on your machine, SimpleITK is able to read them individually…

The only difference that I can think of is that there is something strange going on with the strings returned by the sitk.ImageSeriesReader.GetGDCMSeriesFileNames. Other than this, we’d need access to the data to figure out what is going on.

Hmmm ok
Do you know an other way to read Dicom data with ITK ?
Like, can you suggest me, or do you have in mind, some other code lines to read my CT series ?

You could read it using Slicer, then save it as .nrrd image. Then your could would not have to deal with DICOM at all.

if I do that, whould I be able to read that .nrrd format with ITK ?

Yes, both ITK and SimpleITK read nrrd.

I saved them in .nrrd format like you said.
How can I read them with sitk ?
By using the function ‘sitk.ReadImage’ ?

Yes.

Will I have to change my code for that ?
Cause I tried it and I get an error…

ERROR:
File “C:/Users/PycharmProjects/sitk registration 2.py”, line 46, in
fixed = sitk.ReadImage(series_file_names, sitk.sitkFloat32)
File “C:\Users\Ibrahima.conda\envs\recalage\lib\site-packages\SimpleITK\SimpleITK.py”, line 8876, in ReadImage
return _SimpleITK.ReadImage(*args)
NotImplementedError: Wrong number or type of arguments for overloaded function ‘ReadImage’.
Possible C/C++ prototypes are:
itk::simple::ReadImage(std::vector< std::string,std::allocator< std::string > > const &,itk::simple::PixelIDValueEnum,std::string const &)
itk::simple::ReadImage(std::string const &,itk::simple::PixelIDValueEnum,std::string const &)

The thing I change in my code above is :
A = “E:/IGR/1 11F06DD_Y5_M6_P50.nrrd”
series_file_names = sitk.ReadImage(A)
B = “E:/IGR/1 11F06DD_Y6_M0_P99.nrrd”
series_file_names2 = sitk.ReadImage(B)