Hi all. In our research on declarative image analysis (see www.voxlogica.org) we have used with quite some success SimpleITK from fsharp. I now went to check whether ITK 5 is available in SimpleITK and the answer (gotten from checking the build directory) seems “NO”.
[EDIT: the answer is “YES”, see the discussion below]
Now since I’ve seen that the python interface to ITK (the main library) has been made “more functional”, and on the other hand I see there are wrappers for Java for the main ITK library, I wonder whether it may be easier to bind ITK 5 to C# without having to pass by SimpleITK. I’ll add I’m using dotnet core; I’m seeking for pointers to understand the wrapping architecture for java, and for python.
ITK, like SimpleITK, uses SWIG for generating wrapper interfaces. SWIG supports C# and Java wrapper generation. In ITK/Wrapping/Generators, we have infrastructure for Java – a similar configuration could be added for CSharp.
Technically, there are not any hurdles – a champion is required with the time and resources to bring support for these languages up to the level of Python.
A good overview of the wrapping infrastructure can be found in the ITK Software Guide.
SimpleITK is ITKv5 ready. We nightly build and test SimpleITK compiled against ITKv5. If you build it yourself with the SimpleITK Superbuild all you have to do is add the CMake definition “ITK_GIT_TAG:STRING= v5.1b01”.
We will be releasing SimpleITK 2.0 build against ITK 5.0 in a couple months. Perhaps we should package a SimpleITK 2.0 Alpha compiled against ITK 5.1 when it’s tagged?
thanks for your quick reply. Here are some considerations:
it’s great to see that SimpleITK is already about to release itk5 support. I am now building with itk5.1b1 as you suggested. I believe that for the current development of VoxLogicA that may suffice, and I can certainly wait the release of SimpleITK 2.0. It would also help a lot to distribute nuget binaries for the main operating systems (currently I need to recompile the csharp wrappers when I want to update them). I think we already discussed this some months ago, I will try to understand how to do this as I myself have no idea about that.
However I have some questions about SimpleITK (before even considering to allocate the effort required for #2 below).
First of all, do you think it could be possible to take advantage of the iterators in the experimental namespace of ITK5? In particular the neighbours iterators may be very helpful.
Second, we are planning to investigate using the GPU quite soon, as some (non-ITK-based) operations that we do are expensive and really parallel. But is there support for the GPU also in ITK? And in SimpleITK? There’s no occurrence of the word GPU in the software guide.
Third, I’ve had quite some difficulties in using ConnectedThresholdImageFilter in SimpleITK because converting the array of seed points from dotnet to the SWIG type is terribly slow. In my use case I found an alternative method, but that left me wondering (but this is likely a problem also for #2 below). Providing low-level access to an uninitialised array could be a solution (similar to GetBufferAsXXX, but for arrays). BTW I’ve had to externally lock the images as GetBufferAsXXX is not thread-safe.
I am also happy to get information about the wrappers architecture in ITK. I checked the java examples a bit closer. It would be very useful in my opinion to have access to the full ITK from dotnet, however, I am unsure about some technicalities (that’s also because I never used ITK myself, since I’ve been using SimpleITK.
First, the mangling of types goes against the fact that fsharp (and probably also csharp) have generics, so it would be useful to “demangle” again these types by auto-generating a higher-level fsharp library. Or just don’t mangle them in the beginning and generate sensible fsharp generic types. But this is something that would require some help from members of the fsharp community I believe. I don’t have the manpower to do such a reasoning on my own.
Second, what about pipelining? And parallel processing? VoxLogicA in essence builds a (large,sophisticated) processing pipeline and runs it in parallel on the available cpu cores. Using the pipelining functions in ITK could be a major reason to switch from SimpleITK to ITK, however I think it may be very difficult to get these right in languages different from C++ or am I wrong? And what about inserting also user-defined filters in the pipeline? Is it possible to write such filters outside of C++/python? Also: is it possible to access the ITK thread pool “externally” so that the thread pool of the dotnet virtual machine and the one that ITK uses don’t fight for processors?
It would be great to get SimpleITK into nuget. Help is quite welcomed on this topic. The SimpleITK build infrastructure is quite automated now so this is a great time to start on this topic.
Iterators do not perform well in any wrapped interface as there are too many layers causing significant performance issues. The new neighborhood iterators are only going to be a direct benefit when using C++, otherwise the benefit will come through improved performance in the filters which use them. Are more are coming!
The ITK GPU classes are not currently used much and it has been a while since I used them last. The architecture allows for auto dispatching filters to a GPU implementation if enabled. So if the ITK version SimpleITK is using has GPU enabled, and the filter is implemented on the GPU, SimpleITK may use the GPU implementation. It has been quite a while since I have done this so their may be other hurdles.
You should open a SimpleITK issue on Github for this. In short, you can a global threshold then a connected components, then select you desired components to perform the equivalent operation more efficient in parallel.
Please open and SimpleITK issue on GitHub. This is likely an issue with how you are particularly using it and SimpleITK’s lazy copying, you may just need to call “MakeUnique” before the GetBuffer method.
Hope this helps.
p.s. This is a lot of questions to be asking all at once, and I only got half of them
As @blowekamp mentioned, there is some OpenCL-based GPU support, but there is interest in expanding this support and supporting other types of GPU support, e.g. CUDA, ArrayFire, SYCL, PyTorch.
ITK processing pipelines work well in wrapped languages, e.g. Python.
You can create your own ITK module with C++ code that can be wrapped as described in the ITKModuleTemplate and the ITK Software Guide.
In Python, you can also create your own processing pipeline filters:
This functionality could be added to other languages.
Since the native ITK libraries and the dotnet virtual machine have different runtime environments, the threads cannot be shared. So, limiting the threads on both sides may be helpful to reduce oversubscription.
thanks for your detailed replies. In the meantime I’ve checked, and there is no issue at all running itk5 instead of itk4 in current SimpleITK. This is great! Although for the specific filters I use (connected components, boolean operations, thresholds, boolean morphology) I did not get performance improvements at all.
I think I can safely stick to SimpleITK for the time being. There are a lot of topics to explore starting from your answers, so I will say no more right now, except on the GPU aspects, which would be a quantum leap in application of VoxLogicA, and are going to be a priority for us in the next months. A good GPU architecture for our use case would be one where there is a type for “GPU blobs” denoting an image which is residing in GPU, and the possibility to submit an arbitrary (OpenCL) kernel to an ITK function to be evaluated on a given GPU blob, and obviously converting back and forth between GPU blobs and normal images, which would indeed be expensive.
So one can actually just not load OpenCL at all and use ITK, this looks great. Where can I find more info e.g. on how to compile ITK for GPU, how to configure drivers, etc?
When configuring ITK, turning on ITK_USE_GPU should be enough to get you started. You will need OpenCL development headers and library from some source (e.g. this).
Thanks! But at this point, do filters (which ones?) use the GPU automatically? I guess for instance the connectedComponents filter doesn’t (or if it does, I’m very interested). Otherwise, how to proceed? Where is a list of filters that can operate on GPUImage? (sorry if this is obvious, not to me!)
Directories with GPU prefix are modules containing GPU filters. Connected component isn’t one of them. As it has single-threaded sections, it is not easy to translate it into GPU formulation.
Hi, for connected components, I know quite well, that is also going to be part of my research plan for next year; if we get to a sensible result we may consider proposing patches to ITK. But also binary erosion is not implemented in GPU yet, or am I just wrong in interpreting that directory? [EDIT} Also what about boolean operations (and, not, or)?
There are two additional GPU-related folders, GPUCommon and GPUFiniteDifference. But yes, only a very small subset of ITK filters have GPU variants. I don’t think binary erosion is among them.
@blowekamp I’m now starting to experiment with the GPU. Specifically, I want to be able to use the ITK classes for managing memory from CPU to GPU and back, and also to be able to write my own custom kernels that would then be called on instances of the said ITK class. But as mentioned earlier, I’m using simpleitk.
I think that the “itk” namespace of SimpleITK already contains the needed functionality, but I don’t have it in the .NET dll (at least it doesn’t autocomplete). I suppose I need to enable GPU in cmake, but when I run ccmake in my SimpleITK-build directory, I see no option related to GPU. Is there something I should do, or maybe GPU support is not in SimpleITK yet?
