how to convert a tensor with libtorch into a ITK image?

Engineering
1

I have created a tensor by libtorch , which it’s shape is 688,688,450, and I want to transfer this tensor into image.

2

Hello @Bugmaker,

  1. Convert tensor to numpy array.
    If tensor is on cpu np_arr = x.numpy() otherwise np_arr = x.to("cpu").numpy().
  2. Convert numpy array to image.
    ITK example, SimpleITK Jupyter notebook section titled Conversion between numpy and SimpleITK.
1 Like
3

@zivy
Maybe I didn`t make it clear, libtorch and ITK are C++ libraries, they do not have NumPy.

4

Hello @Bugmaker,

Thanks for the clarification. The relevant ITK filter is ImportImageFilter, see C++ example here.

1 Like
5

This StackOverflow question might be helpful. @Lee_Newberg did you do something with ITK C++ libTorch?

6

I did some torch to C++ array stuff. If the torch tensor is on the GPU, the firs step would be to move it to the CPU. Then you can get a pointer to its first element. Something like

cpuTensor = gpuTensor.to(torch::kCPU);
pFirst = cpuTensor.data_ptr();

Then you have to get that C++ array into an ITK image. If there isn’t some direct way to do that already, maybe use itk::ImportImageFilter::SetImportPointer(pFirst, ...)? Perhaps @dzenanz can help with that.

7

hi @zivy
I have tried that method, but it seems that the itk image data is not the same as tensor. The max value of tensor is 6 and the max value of the converted image is 7.3e-7.

    using PixelType = float;
	constexpr unsigned int Dimension = 3;
	using ImportFilterType = itk::ImportImageFilter<PixelType, Dimension>;
	ImportFilterType::Pointer importFilter = ImportFilterType::New();

	ImageTypeFloat::SizeType shape = img->GetLargestPossibleRegion().GetSize();
	std::cout << " shape value is : " << " " << shape[0] << " " << shape[1] << " " << shape[2] << std::endl;
	importFilter->SetOrigin(img->GetOrigin());
	importFilter->SetSpacing(img->GetSpacing());
	importFilter->SetDirection(img->GetDirection());
	ImportFilterType::IndexType start;
	ImportFilterType::SizeType size;
	size[0] = shape[0];
	size[1] = shape[1];
	size[2] = shape[2];
	//size[3] = data.sizes()[3];
	//start.fill(0);
	start[0] = 0;
	start[1] = 0;
	start[2] = 0;
	std::cout << " size value is : " << " " << size[0] << " " << size[1] << " " << size[2] << std::endl;
	ImportFilterType::RegionType region;
	region.SetIndex(start);
	region.SetSize(size);
	importFilter->SetRegion(region);
	//importfilter->setorigin(image->getorigin());
	//importfilter->setspacing(image->getspacing());
	//importfilter->setdirection(image->getdirection());
	const bool importimagefilterwillownthebuffer = false;
	importFilter->SetImportPointer(static_cast<float*>(data.data_ptr()),
		size[0] * size[1] * size[2],
		importimagefilterwillownthebuffer);
	importFilter->Update();
	ImageTypeFloat::Pointer res = importFilter->GetOutput();

get max code:

    using Calculator = itk::MinimumMaximumImageCalculator<ImageTypeFloat>;
	Calculator::Pointer calculator = Calculator::New();
	calculator->SetImage(res);
	calculator->ComputeMaximum();
	float mx = calculator->GetMaximum();
	std::cout << mx << std::endl;
8

Hello @Bugmaker,

First thing to check is that array_from_tensor = static_cast<float*>(data.data_ptr()) contains the expected values. Possibly a memory issue, try to call data.contiguous().data_ptr() so that the memory is contiguous.

9

hi @zivy,

I have used the method, but the result is still wrong. However, there is one thing confusing me that is the max value of ITK image is 1.4013e-45 which is the minimum value of the float number, and I also get the same value when I printed the tensor by deference such as *(tensor.contiguous.data_ptr)

10

At the risk of stating the obvious, it looks like something that was created as a tensor of int is being treated as if it were a tensor of float. In particular, that small value float(1.401298e-45) that is being seen has a the same bit representation as int(1).

1 Like
11

Sorry, i have not solved this problem. The whole picture becomes fragmented and chaotic, and the whole thing is shifted up. However, when i try to a method like in.set(tensor[i][j][p]), the image was right.