how to pickle SimpleITK transforms?

I’m trying to use python’s multiprocessing Pool.map_async() with a function that returns a SimpleITK.Transform object, i.e.:

    res = []
    for ref_im in ref_imgs:
        res.append(self.pool.apply_async(register_src_ref,
                                         kwds={'moving_img': ref_im, 'fixed_img': src_im, 'scale': scale},
                                         error_callback= self.log_error))
    registration_results = [r.get() for r in res]

But I get an error saying:

SimpleITK.SimpleITK.Transform; proxy of <Swig Object of type ‘itk::simple::Transform *’ at 0x000001FB02EC84E0> >)’. Reason: ‘TypeError(“can’t pickle SwigPyObject objects”,)’

Any idea how to do this?

Thanks

Hello Adi,

Currently SimpleITK does not support pickling of transformations and images, so this will not work when using a process pool. If you change to thread pool (from multiprocessing.pool import ThreadPool instead of from multiprocessing.pool import Pool) it will work.

As you are performing registration, using the process pool is likely more appropriate than a thread pool. To achieve this, you should have your registration function fully encapsulated (input: image file names, output: transformation written to disk, steps: read images -> register -> write resulting transform). This way you will only pass it the file names which are pickled. Even if you read the fixed image multiple times instead of only once, this is a small constant overhead as compared to the time spent on registration.

Hope this helps
Ziv
p.s. Please update the thread to let us know what worked for you (or if you have more questions).

1 Like

Thanks Ziv,
I will try your second suggestion.

Adding pickling would be great to add to SimpleITK! Could you please create a feature request on Github?

The pickling should be fairly easy for most transforms. A transform is usually just defined by the transform name, the dimensions ( some transforms only work with one dimension size ), the fixed parameters, and the parameters ( the ones which are optimized ). It should be fairly easy to get code working for this. There may need to be some special handling for BSpline transforms and displacement fields, most of the effort would likely need to be put into good testing.

github issue: https://github.com/SimpleITK/SimpleITK/issues/460

1 Like

It looks like pickable images has been added with Add support to Python for “pickling” an Image object, but do transforms also fall into this bucket?
or do they have to written to file?

I am about to test this, but the process is long running.

Hello,

Just support for pickling images was added, for now…

Please let us know how your testing going.

as I expected from your reply:

Traceback (most recent call last):
  File "process_case.py", line 198, in <module>
    main()
  File "process_case.py", line 169, in main
    nx.write_gpickle(G, pickle_path)
  File "</home/krs/anaconda3/envs/reg/lib/python3.7/site-packages/decorator.py:decorator-gen-620>", line 2, in write_gpickle
  File "/home/krs/anaconda3/envs/reg/lib/python3.7/site-packages/networkx/utils/decorators.py", line 240, in _open_file
    result = func_to_be_decorated(*new_args, **kwargs)
  File "/home/krs/anaconda3/envs/reg/lib/python3.7/site-packages/networkx/readwrite/gpickle.py", line 70, in write_gpickle
    pickle.dump(G, path, protocol)
TypeError: can't pickle SwigPyObject objects

Is pickling of images working?

I need to try it still.

You may want to try this PR while you are at it. I still want to add more test but it appears to be working.

1 Like