Tensorflow and ITK

Engineering
1

Hello,

I would like to use trained tensorflow graphs in ITK (for segmentation).
Has someone experience using tensorflow in combination with the ITK?

Cheers,
Gordian

1 Like
2

Hello @Gordian,

If what you are trying to do is preprocess the images and then use those results in TensorFlow, you could do it in Python. I haven’t worked with TensorFlow, but I have done similar work with Scikit-learn.
You can view a tutorial here about how to convert ITK images to NumPy arrays, and you can then convert the NumPy arrays to TensorFlow data

1 Like
3

Hello @fbudin,
thank you for the link and the suggestion. I think this is the way to go at the moment.

On the other hand I dont like to switch between languages in my current project that is C++ based. I thought I could use the C++ API of Tensorflow and wrap it in filter. But that seems to be quite complex (C++ API Tensorflow, Compile Tensorflow C++ API with CMake, they only provide guidelines a non-debug version). Therefore I asked for experiences with this topic. The basic idea is to train the tensorflow networks in python, export those networks (and weights) and use the C++ interface to load the networks into an ITK filter.
I will have some discussion with my coworkers if we have time to manage it this way.
Still, any help or suggestions are very appreciated.

1 Like
4

Hi there,

Just dropping in this thread to say that this whole question of using efficiently networks (from Tensorflow or other libs) in combinaison with ITK interest me also, as I will soon have to deal with such project.

@Gordian, that is what I was planning to try. If you make some progress in this way, would you mind reporting it here ?

Cheers,

Tim

5

I will comment and give updates as we move a long this path. I am just not sure about when we have free resources.

2 Likes
6

@Gordian It must be possible to create a filter (mapper) to convert ITK data to TensorFlow data. I haven’t looked at the details, but I would think that it shouldn’t be too difficult. Things to keep in mind are:

  • Is the filter doing a deep copy or a shallow copy
  • Who is the owner of the data after conversion.

Also, one option to implement such a filter is to implement it as a bridge, like what is done for Python and VTK here or in the video module. Note that some of the examples I pointed you to are implemented as ITK filter types (deriving from ITKProcessObject directly or indirectly), and some are stand alone classes.

2 Likes