# Tensorflow and ITK

**URL:** <https://discourse.itk.org/t/tensorflow-and-itk/999>\
**Category:** Engineering\
**Tags:** tensorflow\
**Created:** [June 14, 2018, 7:35am UTC](https://discourse.itk.org/t/tensorflow-and-itk/999 "2018-06-14T07:35:53Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Gordian](https://discourse.itk.org/user_avatar/discourse.itk.org/gordian/32/335_2.png) [@Gordian](https://discourse.itk.org/u/Gordian)\
**Post date:** [June 14, 2018, 7:35am UTC](https://discourse.itk.org/t/tensorflow-and-itk/999/1 "2018-06-14T07:35:54Z")

</div>

Hello,

I would like to use trained tensorflow graphs in ITK (for segmentation).  
Has someone experience using tensorflow in combination with the ITK?

Cheers,  
Gordian

---

<div class="post-metadata">

**Author:** ![fbudin](https://discourse.itk.org/user_avatar/discourse.itk.org/fbudin/32/14_2.png) [@fbudin](https://discourse.itk.org/u/fbudin)\
**Post date:** [June 14, 2018, 12:07pm UTC](https://discourse.itk.org/t/tensorflow-and-itk/999/2 "2018-06-14T12:07:11Z")

</div>

Hello @Gordian,

If what you are trying to do is preprocess the images and then use those results in TensorFlow, you could do it in Python. I haven’t worked with TensorFlow, but I have done similar work with Scikit-learn.  
You can view a tutorial [here](https://github.com/KitwareMedical/2018-05-30-KRSCourseInBiomedicalImageAnalysisAndVisualization/blob/master/4_ITK_and_NumPy.ipynb) about how to convert ITK images to NumPy arrays, and you can then [convert the NumPy arrays to TensorFlow data](https://stackoverflow.com/questions/36926140/how-to-convert-numpy-arrays-to-standard-tensorflow-format)

---

<div class="post-metadata">

**Author:** ![Gordian](https://discourse.itk.org/user_avatar/discourse.itk.org/gordian/32/335_2.png) [@Gordian](https://discourse.itk.org/u/Gordian)\
**Post date:** [June 14, 2018, 2:48pm UTC](https://discourse.itk.org/t/tensorflow-and-itk/999/3 "2018-06-14T14:48:28Z")

</div>

Hello @fbudin,  
thank you for the link and the suggestion. I think this is the way to go at the moment.

On the other hand I dont like to switch between languages in my current project that is C++ based. I thought I could use the C++ API of Tensorflow and wrap it in filter. But that seems to be quite complex ([C++ API Tensorflow](https://medium.com/@shiweili/building-tensorflow-c-shared-library-on-windows-e79c90e23e6e), [Compile Tensorflow C++ API with CMake](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/cmake), they only provide guidelines a non-debug version). Therefore I asked for experiences with this topic. The basic idea is to train the tensorflow networks in python, export those networks (and weights) and use the C++ interface to load the networks into an ITK filter.  
I will have some discussion with my coworkers if we have time to manage it this way.  
Still, any help or suggestions are very appreciated.

---

<div class="post-metadata">

**Author:** ![tim-evain](https://discourse.itk.org/user_avatar/discourse.itk.org/tim-evain/32/220_2.png) [@tim-evain](https://discourse.itk.org/u/tim-evain)\
**Post date:** [June 14, 2018, 3:09pm UTC](https://discourse.itk.org/t/tensorflow-and-itk/999/4 "2018-06-14T15:09:27Z")

</div>

Hi there,

Just dropping in this thread to say that this whole question of using efficiently networks (from Tensorflow or other libs) in combinaison with ITK interest me also, as I will soon have to deal with such project.

> [@Gordian](#):
>
> The basic idea is to train the tensorflow networks in python, export those networks (and weights) and use the C++ interface to load the networks into an ITK filter.

@Gordian, that is what I was planning to try. If you make some progress in this way, would you mind reporting it here ?

Cheers,

Tim

---

<div class="post-metadata">

**Author:** ![Gordian](https://discourse.itk.org/user_avatar/discourse.itk.org/gordian/32/335_2.png) [@Gordian](https://discourse.itk.org/u/Gordian)\
**Post date:** [June 14, 2018, 7:48pm UTC](https://discourse.itk.org/t/tensorflow-and-itk/999/5 "2018-06-14T19:48:57Z")

</div>

> [@tim-evain](#):
>
> @Gordian, that is what I was planning to try. If you make some progress in this way, would you mind reporting it here ?

I will comment and give updates as we move a long this path. I am just not sure about when we have free resources.

---

<div class="post-metadata">

**Author:** ![fbudin](https://discourse.itk.org/user_avatar/discourse.itk.org/fbudin/32/14_2.png) [@fbudin](https://discourse.itk.org/u/fbudin)\
**Post date:** [June 14, 2018, 8:39pm UTC](https://discourse.itk.org/t/tensorflow-and-itk/999/6 "2018-06-14T20:39:16Z")

</div>

@Gordian It must be possible to create a filter (mapper) to convert ITK data to TensorFlow data. I haven’t looked at the details, but I would think that it shouldn’t be too difficult. Things to keep in mind are:

- Is the filter doing a deep copy or a shallow copy
- Who is the owner of the data after conversion.

Also, one option to implement such a filter is to implement it as a bridge, like what is done for Python and VTK [here](https://github.com/InsightSoftwareConsortium/ITK/tree/master/Modules/Bridge) or in the [video module](https://github.com/InsightSoftwareConsortium/ITK/tree/master/Modules/Video). Note that some of the examples I pointed you to are implemented as ITK filter types (deriving from ITKProcessObject directly or indirectly), and some are stand alone classes.