Sitk AffineTransform vs Pytorch grid sample

I’m comparing the result of SITK AffineTransform and Pytorch grid_sample. The difference between them is that sitk treats origin as the centre of rotation while Pytorch treats the centre of the image as the centre of rotation.

import SimpleITK as sitk
import numpy as np
import torch
import os
import pickle
import matplotlib.pyplot as plt
import copy
import imageio
import cv2
affine_matrix = np.array([[[ 1.0170,  0.0398,  0.0435, -0.0110],
         [-0.0713,  1.0165,  0.0446,  0.0221],
         [ 0.0387,  0.0189,  0.9905, -0.0033]]], dtype=np.float64)

torch_affine_matrix = torch.from_numpy(affine_matrix).unsqueeze(0)

img_3d = sitk.ReadImage("img.nii")
img_3d_tensor = torch.from_numpy(sitk.GetArrayFromImage(img_3d)).unsqueeze(0).unsqueeze(0)

# sitk transform
affine_3d_transform = sitk.AffineTransform(3)
affine_3d_transform.SetMatrix(theta_3d.squeeze().cpu().numpy()[:3, :3].flatten())
affine_3d_transform.SetTranslation(theta_3d.squeeze().cpu().numpy()[:, 3])

# affine_3d_transform.SetCenter((32, 32, 32))  # center of volume


# pytorch transform
resampled_3d_sitk = sitk.Resample(img_3d, img_3d, affine_3d_transform, sitk.sitkNearestNeighbor, 0.0)

sitk.WriteImage(resampled_3d_sitk, "sitk_3d_resampled.nii")

grid_3d = torch.nn.functional.affine_grid(theta_3d, img_3d_tensor.shape)
resampled_3d_pytorch = torch.nn.functional.grid_sample(img_3d_tensor, grid_3d)

resampled_3d_pytorch = sitk.GetImageFromArray(resampled_3d_pytorch.squeeze())
resampled_3d_pytorch.CopyInformation(img_3d)
sitk.WriteImage(resampled_3d_pytorch, "pytorch_3d_resampled.nii")

I’m not able to figure out why they are behaving differently. I might have missed something here. My assumption is that transforming the same volume by same matrix should give same result. Any help would be highly appreciated. Thanks in advance!

Result difference(left is SITK’s output and right is Pytorch’s output):

imge url: https://www.dropbox.com/s/nyq8qpdnu68sdso/img.nii?dl=0

Hello @prms,

“The difference between them is that sitk treats origin as the centre of rotation while Pytorch treats the centre of the image as the centre of rotation.”

SimpleITK does not use the image origin as center of rotation for the global transformations (rigid, affine…). There are two options, either it uses the center of rotation specified by the user or the default which is [0,0,0]. For most medical images the origin is not at [0,0,0], check with img_3d.GetOrigin(). So you should explicitly set the center of rotation either to the image center or the image origin whichever you want (image center makes more sense in most cases). Please take a look at this Jupyter notebook section titled Resampling.

Hi @Zivy,
Thank you for pointing the resource. I was missing out on this. By origin I meant (0,0,0) in context of voxel space. I tried the same in 2D image by setting the center of rotation to (90, 140) of a (180, 280) image and the results are identical(it had unit direction cosine). As SITK treats volumes as physical objects, is it the case that center also expects the coordinates in physical coordinates instead of voxel coordinates? And would setting center to below values
affine_3d_transform.SetCenter(np.array(img_3d.GetOrigin()) + np.array(img_3d.TransformContinuousIndexToPhysicalPoint(img_3d.GetSize()))/2)

rotate the volume about its origin? I tried with this setting but still, the results are different. Also, I am using AffineTransform instead of Euler3DTransform. Is there anything else that needs to be considered?

Hello @prms,

Yes, you need to use the physical center of the image. You don’t need to add the center to the origin:

affine_3d_transform.SetCenter(np.array(img_3d.TransformContinuousIndexToPhysicalPoint(img_3d.GetSize()))/2)