Tensorflow and ITK


I would like to use trained tensorflow graphs in ITK (for segmentation).
Has someone experience using tensorflow in combination with the ITK?


1 Like

Hello @Gordian,

If what you are trying to do is preprocess the images and then use those results in TensorFlow, you could do it in Python. I haven’t worked with TensorFlow, but I have done similar work with Scikit-learn.
You can view a tutorial here about how to convert ITK images to NumPy arrays, and you can then convert the NumPy arrays to TensorFlow data

1 Like

Hello @fbudin,
thank you for the link and the suggestion. I think this is the way to go at the moment.

On the other hand I dont like to switch between languages in my current project that is C++ based. I thought I could use the C++ API of Tensorflow and wrap it in filter. But that seems to be quite complex (C++ API Tensorflow, Compile Tensorflow C++ API with CMake, they only provide guidelines a non-debug version). Therefore I asked for experiences with this topic. The basic idea is to train the tensorflow networks in python, export those networks (and weights) and use the C++ interface to load the networks into an ITK filter.
I will have some discussion with my coworkers if we have time to manage it this way.
Still, any help or suggestions are very appreciated.

1 Like

Hi there,

Just dropping in this thread to say that this whole question of using efficiently networks (from Tensorflow or other libs) in combinaison with ITK interest me also, as I will soon have to deal with such project.

@Gordian, that is what I was planning to try. If you make some progress in this way, would you mind reporting it here ?



I will comment and give updates as we move a long this path. I am just not sure about when we have free resources.


@Gordian It must be possible to create a filter (mapper) to convert ITK data to TensorFlow data. I haven’t looked at the details, but I would think that it shouldn’t be too difficult. Things to keep in mind are:

  • Is the filter doing a deep copy or a shallow copy
  • Who is the owner of the data after conversion.

Also, one option to implement such a filter is to implement it as a bridge, like what is done for Python and VTK here or in the video module. Note that some of the examples I pointed you to are implemented as ITK filter types (deriving from ITKProcessObject directly or indirectly), and some are stand alone classes.