Implementation of a Custom ImageToImage Similarity Metric for Registration

Beginner Questions
,
1

Hello ITK community! I am in desperate need for help as I am finding no resources on this.

I am trying to implement a custom ImageToImage similarity metric for the purpose of registration. For a project, I want to register two 3D binary segmentation masks as a first attempt. For this, I want a simple DICE metric to be used as the similarity metric, however, this does not exist in the C++ ITK library. How do I implement this? So far I simply tried to inherit from the ImageToImageMetricv4 class and implement my own value and derivative calculation (see implementation below) but I just keep getting errors that I honestly do not quite understand. Can anybody help me out here? Or at least point me in the right direction on how to do this?

template <typename TFixedImage, typename TMovingImage>
class DiceMetric : public itk::ImageToImageMetricv4<TFixedImage, TMovingImage>
{
    public:
        using Self = DiceMetric;
        using Superclass = itk::ImageToImageMetricv4<TFixedImage, TMovingImage>;
        using Pointer = itk::SmartPointer<Self>;

        itkTypeMacro(DiceMetric, ImageToImageMetricv4);

        // Define some type aliases for convenience
        using FixedImageType = TFixedImage;
        using MovingImageType = TMovingImage;
        using FixedImagePixelType = typename FixedImageType::PixelType;
        using MovingImagePixelType = typename MovingImageType::PixelType;
        using MeasureType = typename Superclass::MeasureType;
        using DerivativeType = typename Superclass::DerivativeType;
        using FixedImageRegionType = typename FixedImageType::RegionType;

    // Override GetValue function
    MeasureType GetValue() const override
    {
        // Get the fixed and moving images
        const FixedImageType *fixedImage = this->GetFixedImage();
        const MovingImageType *movingImage = this->GetMovingImage();

        // Get the region of overlap between the two images
        FixedImageRegionType region = fixedImage->GetBufferedRegion();
        region.Crop(movingImage->GetBufferedRegion());

        // Compute the Dice coefficient
        MeasureType dice = 0.0;
        MeasureType intersection = 0.0;
        MeasureType sum = 0.0;

        itk::ImageRegionConstIterator<FixedImageType> fixedIt(fixedImage, region);
        itk::ImageRegionConstIterator<MovingImageType> movingIt(movingImage, region);

        while (!fixedIt.IsAtEnd())
        {
            FixedImagePixelType fixedValue = fixedIt.Get();
            MovingImagePixelType movingValue = movingIt.Get();

            intersection += static_cast<MeasureType>(fixedValue && movingValue);
            sum += static_cast<MeasureType>(fixedValue || movingValue);

            ++fixedIt;
            ++movingIt;
        }

        if (sum > 0.0)
        {
            dice = (2.0 * intersection) / sum;
        }

        return dice;
    }

    // Override GetDerivative function
    void GetDerivative(DerivativeType &derivative) const override
    {
        // Set the derivative to zero manually
        derivative.Fill(0.0);
    }

    protected:
    DiceMetric() = default;
    ~DiceMetric() override = default;
};

// Command Observer to print optimizer paramters at each iteration
// to monitor the evolution of the registration process
class CommandIterationUpdate : public itk::Command {
    public:
        using Self = CommandIterationUpdate;
        using Superclass = itk::Command;
        using Pointer = itk::SmartPointer<Self>;
        itkNewMacro(Self);
    
    protected:
        CommandIterationUpdate() = default;
    
    public:
        using OptimizerType = itk::RegularStepGradientDescentOptimizerv4<double>;
        using OptimizerPointer = const OptimizerType *;
        void Execute(itk::Object * caller, const itk::EventObject & event) override {
            Execute((const itk::Object *)caller, event);
        }
        void Execute(const itk::Object * object, const itk::EventObject & event) override {
            auto optimizer = static_cast<OptimizerPointer>(object);
            if (!itk::IterationEvent().CheckEvent(&event)) {
                return;
            }
            std::cout << optimizer->GetCurrentIteration() << "   ";
            std::cout << optimizer->GetValue() << "   ";
            std::cout << optimizer->GetCurrentPosition() << std::endl;
        }
};

When I then try to set this metric in the registration method with
registration->SetMetric(metric)
I get the error:

no suitable conversion function from “itk::Object::Pointer” to "itk::ObjectToObjectMetricBaseTemplate

Any help is appreciated!

2

There are two separate registration frameworks within ITK, v3 (classes with no suffix), and v4 (classes with v4 suffix). You might be mixing them up.

MatchCardinality is something like a Dice metric. Why don’t you give that a try?

1 Like
3

That actually sounds great! Just to make sure I understand this correctly, MatchCardinality is only available in v3? So this means I can also only use it with v3 registration components?

4

So I managed to use the MatchCardinality Metric with only v3 components, which is great! However, I now have a problem that I have not seen before and it seems to be due to the interpolator I am using. I have tried both itk::LinearInterpolateImageFunction and itk::NearestNeighborInterpolateImageFunction and for both I get the same artifact.

The original input binary mask of the lung looks like this:

OriginalLung
OriginalLung731×384 29.2 KB

After registration, the registered binary mask of the lung has this weird artifact:

RegisteredLung
RegisteredLung727×383 31.2 KB

It is almost like a “wall” built along the coordinate system axis. What could be causing this and how do I avoid it? It is obviously a problem if I am registering along multiple iterations and this artifact starts showing up.

5

Okay so the other problem with this is that I am trying to stack metrics using ObjectToObjectMultiMetricv4. Therefore the MatchCardinalityImageToImageMetric is not compatible. Maybe it helps explaining my overall idea.

I have the moving and fixed CT image. In addition to that, I have binary segmentation masks of various anatomies such as the lung for each image. I would like the registration to basically focus on both MSE between the two CTs, but also at the same time on the DICE metric between the masks I am passing and potentially even multiple masks of different anatomies, for which each DICE metric is weighted differently.
I thought that if I successfully implement a DICE metric by inheriting from ImageToImageMetricv4, that I could stack and weight the different inputs, and their corresponding similarity metrics and weights, kind of like you explained here. Is my logic off? Is there no MatchCardinalityImageToImageMetricv4? If not, are there any resources on implementing your own metric in the v4 registration?

6

You can use multi-metric to combine multiple metrics. For developing one’s own metric, there is no tutorial (as far as I know). Suggestion: pick the most similar existing metric, copy it, rename, and modify to suit your needs.

The resampling problem above: the default value is 0. For CT images, you might want -1000. For label images, you should use whatever value you use for background label. Hope this helps.

1 Like