Build errors : ITK Module Python Wrapping

@matt.mccormick :
Ah, correct. The binary packages on PyPI cannot be used to build additional module packages. There are some limitations on the build location, but the ITKPythonPackage scripts could be used locally, e.g. macpython-download-cache-and-build-module-wheels.sh. These scripts download the build trees that were used to generate the PyPI packages. But, these are not intended to be used locally, and GitHub CI generated packages are preferred.

Ok, thats what I thought. I would not recommend using macpython-download-cache-and-build-module-wheels.sh outside a container as it might mess with your setup …

Bad news, Python test of the wrapped module ends up with a nice:
Bus error: 10
Have to look into the debugging part of the wrapping
:slight_smile:

Hmm, is this for the macOS binaries? All the binaries?

Yes, it seems so,

macOS: Bus error: 10
windows: runtime error Python has stopped working
linux: did not tried yet

what’s bothers me is that it does not seems to be linked to the C++ code/module as it passes the CTests with success… still learning pdb to investigate the python module

so far, the PyPI module can be imported, seems like the instanciation of the BinaryThinningImageFiltrer3D throw a seg fault… digging into SWIG atm

I don’t know if it related, but these lines should be removed:

https://github.com/T4mmi/ITKBinaryThinning3D/blob/91d7526747862631dafec65b22cffaa6d8650439/include/itkBinaryThinningImageFilter3D.hxx#L40-L41

The output is already created by the itk::ImageSource base class:

https://github.com/InsightSoftwareConsortium/ITK/blob/fe02f8a96627bad5694a9920f54fe2e91d5fd336/Modules/Core/Common/include/itkImageSource.hxx#L49-L50

and not creating it with MakeOutput could cause issues.

@T4mmi the CI builds were recently updated to use ITK 5.0, whose binaries will not be compatible with ITK 4.0 because of the transition to C++11. This is likely causing the issue.

I will follow-up in another thread with instructions on how to address this.

In the meantime, you could try running your Python packages against the ITK 5.0 Python packages by installing itk with the --pre option for pip:

python -m pip install --upgrade --pre itk
1 Like

Great, I’ll try this as soon as I come back to the office.

Thanks a lot for your help !

No luck, even with itk v5 still got the Bus error: 10

:frowning: That is interesting.

Build configuration for ITK v4 is described here. Do the ITKv4 packages result in the same error?

1 Like

Sorry for the delay, I was out of the office last 2 weeks.

Yes, that seems to fix the problem with the ITKv4 build. I manage to create a filter object :

In [7]: itk.BinaryThinningImageFilter3D[UnsignedCharImageType, UnsignedCharImageType].New()
Out[7]: <itkBinaryThinningImageFilter3DPython.itkBinaryThinningImageFilter3DIUC3IUC3; proxy of <Swig Object of type 'itkBinaryThinningImageFilter3DIUC3IUC3 *' at 0x10848bae0> >

I will stick with this solution for now since ITKv5 is still in beta,
Thanks a lot for your help

I also added a simple python test file that mimics the CTest.
However, it ends with a Floating point exception: 8, something I already experienced using the itk.AffineTransform but never solved…
Could it come from a difference between the itkCtype and the itk.ImagePython ?

In [5]: InputPixelType
Out[5]: <itkCType unsigned char>

In [6]: UnsignedCharImageType
Out[6]: itkImagePython.itkImageUC3

in

# IO buffer
imageIO = itk.ImageIOFactory.CreateImageIO(source, itk.ImageIOFactory.ReadMode)
imageIO.SetFileName(source)
imageIO.ReadImageInformation()
# Typedef
Dimension = imageIO.GetNumberOfDimensions()
InputPixelType = itk.ctype(itk.ImageIOBase.GetComponentTypeAsString(
    imageIO.GetComponentType()).replace('_', ' '))  # prevent 'unsigned_char' error
InputImageType = itk.Image[InputPixelType, Dimension]
UnsignedCharImageType = itk.Image[itk.UC, Dimension]
# Image reader
imageFileReader = itk.ImageFileReader[InputImageType].New()
imageFileReader.SetFileName(source)
1 Like

Very nice!

Interesting.

This can be written much more succinctly,

image = itk.imread(source)
InputImageType = type(image)

Overall, the entire pipeline can be written,

input_image = itk.imread(source)
# By passing the input_image to .New(), it sets it as the filter input and picks the required filter type automatically
thinning_filter = itk.BinaryThinningImageFilter3D.New(input_image)
itk.imwrite(thinning_filter, outputFilePath)

Thanks for the tip, I thought such syntax only applied to simpleITK…
This is far easier to read :slight_smile:

I end up with a Segmentation fault: 11 that should correspond to the previous Floating point exception: 8.
I blindly trusted the CTest that passed on CI tools, but is it possible that it succeed even with a segfault ?

I will try to extract the test resulting output image to see if the C++ module performs correctly…

Ok even stranger,
I tried the under Windows, everything works fine !

I’ll try to reinstall python from scratch under macOS to see if the error is not related to my own installation. I should also try under Linux…

1 Like

Ok, summary of the day :

Windows 7 / Python 3.5 (miniconda) / ITK 4.12 (conda-forge) - OK
Windows 7 / Python 3.5 (miniconda) / ITK 4.13 (conda-forge) - OK
Windows 7 / Python 3.5 (miniconda) / ITK 5.0 (conda-forge) - OK

Ubunutu 18.04 / Python 3.6 (anaconda) / ITK 4.13 (conda-forge) - OK

macOS 10.13.3 / Python 3.5 (anaconda) / ITK 4.13 (conda-forge) - Bus error: 10
macOS 10.13.3 / Python 3.6 (anaconda) / ITK 4.13 (conda-forge) - Floating point exception: 8

… seems like the macOS wrapping is not working …

[edit] I also tried using native Python installations (without conda) and various ITK version (4.12, 4.13, 5.0) … all ended with errors under macOS while everything seems to work perfectly with Windows/Linux

__[edit 2]__I used macOS 10.13

1 Like

Outstanding investigation @T4mmi !

I will try to reproduce…

Could not find the reason why it doesn’t work on mac …
I do suspect a problem with travis build but to be fair I did not investigate further.

Last ‘problem’ that I’m facing that you may help with concerns the versionning / dependencies requirements. Since I’ve build the module for both ITK 4 & 5, I would like to distribute them through PyPI. I created two versions (4.x and 5.x, one from each branch - yes bad naming of the versions) but pip install fails with ITK 4.13, I do have to force pip install module==4.x
Does it come from the version numbering ? is there a ‘standard’ way to do this without forcing the upgrade to ITK v5 ?

if someone have an answer it might be a good addition to the external module tuto

1 Like

I tried to reproduce locally, but I could not reproduce the segfault with a Debug or Release build. @T4mmi Do you observe the segfault with local builds, the TravisCI package, or both?

Inspecting the code, these lines

needs to be changed to:

template <class TInputImage, class TOutputImage>
class ITK_TEMPLATE_EXPORT BinaryThinningImageFilter3D : public ImageToImageFilter<TInputImage, TOutputImage>

The ITK_TEMPLATE_EXPORT is required for correct symbol exports on macOS. It is likely that a missing ITK_TEMPLATE_EXPORT somewhere causes the segfaults that are observed.

Good question and suggestion. I added the following to the external module tutorial:

  • On the 5.X development branch, set the version= value in setup.py to a pre-release version, i.e. X.YaN, X.YbN, or X.YrcN where X.Y are higher numbers than the versions on the 4.X branch. This will ensure that a user can install the 4.X-based package with pip install itk-mymodule or the 5.X-based package with pip install --pre itk-mymodule.

Please let me know if that is unclear or can be improved.

I tried to reproduce locally, but I could not reproduce the segfault with a Debug or Release build.

I would say good news then ! The problem might come from my setup (sorry for the issues) …

@T4mmi Do you observe the segfault with local builds, the TravisCI package, or both?

I observe it using the TravisCI package (directly and through PyPI) since I did not compile locally the module (I should try it).

The ITK_TEMPLATE_EXPORT is required for correct symbol exports on macOS. It is likely that a missing ITK_TEMPLATE_EXPORT somewhere causes the segfaults that are observed.

Ok, thanks I’ll change that ! ( it’s always good to learn :slight_smile: )

On the 5.X development branch, set the version= value in setup.py to a pre-release version, i.e. X.YaN, X.YbN, or X.YrcN where X.Y are higher numbers than the versions on the 4.X branch. This will ensure that a user can install the 4.X-based package with pip install itk-mymodule or the 5.X-based package with pip install --pre itk-mymodule.

Please let me know if that is unclear or can be improved.

That’s perfect !
I didn’t know for the letters that were considered as pre-release version…

Thanks again for your help during this first ITK wrapping
I’ll try to port some of our algorithms to ITK and provide Python modules for wider use.

1 Like

That is fantastic! Please let use know if we can help!

Does adding ITK_TEMPLATE_EXPORT to itkBinaryThinningImageFilter3D.h help with the macOS package segfaults?

Does adding ITK_TEMPLATE_EXPORT to itkBinaryThinningImageFilter3D.h help with the macOS package segfaults?

unfortunately nope, I should try on a different machine but I don’t have another mac (should mount a virtual one to test though) …

1 Like