I am working on a new functor filter implementation in OTB, and feedbacks suggest that this is something that could be interesting for ITK. Basically you can use it very simply with any functor with operator() or lambda that has any number of arguments. It will deduce all the types at compile time (supporting scalar pixels, VariableLengthVector, Neighborhoods …).
double scale = 10.;
// dummy lambda that multiply by a scalar
auto lambda = [scale](double p)
{
return scale*p;
};
// Create a filter that will accept an otb::Image<double> as input and produce an otb::Image<double>
auto filterLambda = NewFunctorFilter(lambda);
// Set the inputs using variadic SetVInputs
filterLambda->SetVInputs(image);
// All good, lets run it
filterLambda->Update();
With a functor that has 3 different input types:
// dummy lambda that sums 3 images with different types
auto lambda = [](double p, unsigned int p1, short p2)
{
return static_cast<double>(p)+static_cast<double>(p1) + static_cast<double>(p2);
};
// Create a filter that will accept otb::Image<double>, otb::Image<unsigned int>,otb::Image<short> as inputs and produce an otb::Image<double>
auto filterLambda = NewFunctorFilter(lambda);
// Set the inputs using variadic SetVinputs
filterLambda->SetVInputs(image, image0, image1); // image is otb::Image<double>, image0 is otb::Image<unsigned int> and image1 is otb::Image<short>
// All good, lets run it
filterLambda->Update();
With a lambda working on neighborhoods:
// lambda that performs neighborhood averaging
auto lambda = [](const itk::Neighborhood<short> & in)
{
double out(0);
for(auto it = in.Begin(); it!=in.End();++it)
out+=static_cast<TOut>(*it);
out/=in.Size();
return out;
};
// Create a filter that will accept otb::Image<short>, as inputs and produce an otb::Image<double>
auto filterLambda = NewFunctorFilter(lambda,{{3,3}});
// Set the inputs using variadic SetVinputs
filterLambda->SetVInputs(image); // image is otb::VectorImage<short>
// All good, lets run it
filterLambda->Update();
The are called UnaryFunctorGenerator and BinaryFunctorGeneratorImageFilter These filters were replacements for the prior UnaryFunctorImageFilter and BinaryFunctor filter. They have some similar functionality to what you propose, but don’t work with neighborhoods. Please see if these new classes in ITK meet some of your needs and if you could help improve them!
The way that the functor (or the lambda) defines the number of inputs and the type of the output is pretty neat in the OTB implementation! This is great stuff, it would be awesome to merge efforts!
I’ll have a deeper look at the code in gitlab, I wonder if there is a cleaner interface for the different cases of NewFunctorFilter (with neighborhoods, with output bands), will comment there. But it is exciting this genericity!
Yes, There are quite a number of cool new features going on here!
One is the new idea of “given” a lambda or function construct a filter for it. This is an interesting way to construct a filter, as I usually expect to provide some images to get a filter, where the images define the type the filters operate on.
There is also a nice veridic template base class. I think there is a need for some better NaryImageToImageFilter base classes in ITK. And then a NaryFunctorGenerator class could be derived.
It may be a good idea to keep separate the “functions” which construct an appropriate Image filter from the image filters. There may be multiple ways to construct a filter.
I just had a look at those two filters. They indeed offer similar functionalities. Major differences with my implementation I can see are:
Mine covers both cases (Unary and Binary), and any other number of inputs with a single class. This actually part of my motivations, which I will explain in another post to keep things focused.
With those Generators, you still have to match the image types with the functor/function/lambda expected types. There is an hidden paradigm shift here : UnaryFunctorImageFilter (as I understand it) was essentially meant for developer to derive new classes (such as AddImageFilter), whereas those Generators as well as my implementation are meant to be used by final users of the API. We need to make things easy for them (another part of my motivations).
Generators implementation supports std::cos and the rest, I am not sure my code does.
I guess we are not going to purely replace those Generators with my implementation (and there are probably many things to be improved in my code to be a suitable contribution to ITK), but there are some intermediate steps that could be interesting:
Import the few meta-programming functions that allows to support VariableLengthVector outputs, Neighborhood and so on in the Generators, to bring more power to them.
Provide helper free functions to build the correct instance of Generators classes from the functor.
About the VariadicImageToImageFilter : this would be a fairly easy contribution, and I think it could become the base class for all ImageToImageFilter. It trades a lot of derived class SetInput*() with dynamic casting inside (that may fail at run-time) for a compile-time check of types, which seems a good move to me.
Question is are the user ready to trade SetInput1() with SetVInput<0>() ?
And if it were to be contributed alone, it would not have any practical use.
Thanks a lot for all your very positive feedbacks. There is a lot to discuss to see how we can move all or part of this code to ITK, and I replied to some of your points directly. In this post I just want to expose my motivation for writing this code (and other ideas I have in mind for OTB / ITK)
It is designed to be easy to use : in the remote sensing field, the current trend for users is to turn to python/numpy/scikitlearn/dask, if not directly to Google Earth Engine. It is quite difficult to explain to those guys that you need to write 2 or 3 cryptic classes before being able to perform a simple pixel-wise operation between images. Yet, when it comes to design processing chains that will process millions of images for many years, I still believe we have a lot to offer, starting with compile time checks and static analysis. You never know when you are going to hit that failing untested path in your fancy python processing chain …
2 it is designed to reduce the amount of code : I would like to drastically reduce the amount of code in OTB, in order to improve maintainability, build and testing time, and focus on things that matters. And the functor filter family as it is designed and heavily used in OTB spawns an awful lot of boiler plate code. Code we need to build, test (most often poorly) and maintain.
3 It is designed for better testing : In OTB we followed a lot of ITK redesign, but we miss the one on testing. We still have the read/compute/write/compare which is painfully slow, requires high maintenance and is in fact mainly testing the IO classes. My idea is that if the functor filter is well tested on its side, we can write unit tests for functors (which would be fairly easy and fast), and we do not have to test their combination with the filter, which will save time and tests maintenance.
Even if we have to resort to, e.g., preprocessor macros, is it possible to generate and use backwards compatible APIs and APIs that can be used from Python, e.g. SetInput1(input1), SetInput(0, input1), … ?
When there has been more than one fixed input there typically is also a SetInputName method. I could see the SetInput<1>() being more of an internal method. Another tricky case would be for filters like join series which can take any number of inputs.
Also consider the ImageSource, there are some filters which produce multiple outputs.
