Making a 3D sequence from 4D volume

[blowekamp (Bradley Lowekamp)]

@lassoan

How do you represent XYZTC files in SimpleITK in memory now? How do you plan to improve it?

Currently, I am not operating on XYZTC in SimpleITK. I use the ImageIO’s region of interest mechanism to set an extract region to read into 3d. For example if the file image is of size [100,100,100,60,21], I can specify a size of [100,100,100,0,0], and it will collapse the T,C dimension. I also join the series of processed channel for an XYZC image.

When the first dimension is C, an “itk::VectorImage” or vector pixel type can be used. Then a 5D Image could be represented in SimpleITK as a XYZT 4D image with a vector pixel type. However, if you want to do operation on channel independently this does not have an efficient memory lay out. Say for example, you want to do Gaussian blurring for each channel independently, this is not an efficient memory layout because of the increased data needed for each neighbor would would decrease cache efficiency. But if you want to display an RGB image the vector image or CXYZ then the channel first memory layout is sensible.

My Plan would be to start experimenting with instantiating SimpleITK in 5D similar to the limited support for 4D. This would be basic classes like Image, and IO along with some grid manipulation like the SlicerImageFilter, and the JoinSeriesImageFilter. When that works with my microscopy test case we would then issue a RFC and get further ideas and use cases.

@jrojasUNC It looks like you are doing two steps. Both converting from Numpy to SimpleITK, then writing from SimpleITK to NRRD. Let us go back to your original post:

@jrojasUNC

Is there a way to use the functionality of GetImageFromArray() to change a 4D simpleITK image into a 3D vector (ie kinds: vector domain domain domain)?

So… I don’t know of a filter in ITK which can do this directly. There is the ComposeImageFilter, which allows you to join several images into a VectorImage a.k.a a sitk::Image with vector pixel type. There is the PermuteImageFilter which could allow you to change XYZC into CXYZ 4D scalar image ( note: currently SimpleITK only have this instantiated upto 3D Image ).

The conversion to a vector image could be done with the ImportImageFilter, which would be very similar to using GetArrayViewFromImage followed by a GetImageFromArray(...,isVector=True).

Perhaps there is a need to add this ability to change the fast axis into a VectorImage’s channel component to an existing filter or even a new filter.

[lassoan (Andras Lasso)]

@blowekamp Thanks for the information. If there is anything that we can do on Slicer side then let me know.

@mihail.isakov

space dimension: 4 as it does not make any sense

why not, 4th can be time (3D+t), or 3rd can be time (2D+t)

Space dimension of 4 would mean that we have a single volume that is defined in 4D space (not 3D space + time, actually 4D space), that’s why it would not make sense. domain domain domain domain is not incorrect per se, but it is not well specified, we don’t know which axis is which and instead it should be “space space space time” or “time space space space”. See Teem: nrrd: Definition of NRRD File Format.

@mihail.isakov

it is Nrrd terminology, MetaImage doesn’t have such wording and works well for me.

Since there is no standard way of specifying axis kinds in MetaImage, it can only support 2D and 3D volumes in a standard way. If somebody has the resources to push MetaImage - update specification and improve MetaIO library - then the file format could be extended to properly support higher-dimensional volumes. However, since MetaIO image and nrrd file formats are essentially the same, we’d better choose one to develop and maintain and let the other fade away.

[blowekamp (Bradley Lowekamp)]

@lassoan

For example, I would be reluctant to allow loading anything from a sequence of kinds: domain domain domain domain with space dimension: 4 as it does not make any sense. Space is 3 dimensional (or may be restricted to 2), but 4 … We also would not know which axis we should use for time.

As you confirmed, kinds: domain domain domain list and kinds: list domain domain domain already work (the specification is clear).

I just looked up the NRRD specifications for kinds. They’re are many options here, including “time” which seems most appropriate for how you are using it for sequences. I am uncertain if ITK currently exposes any of these different “kinds” of dimensions. I expect in some of the cases it would exposed as meta-data in the ImageIO, or in the Image class. This would allow the application to interpret the meaning of a particular axis.

I would fall back to how ImageJ loads these high dimensional images, with lack of specific meta-data, the user is just prompted to specify the interpretation of the dimensions. Many other scientific domains are not as rigorous as medial imagining’s DICOM standard.

[dzenanz (Dženan Zukić)]

@lassoan

since MetaIO image and nrrd file formats are essentially the same, we’d better choose one to develop and maintain and let the other fade away.

I agree. There are two stumbling blocks here. MetaIO library comes with a lot of other things besides MetaImage, which makes the library clumsy. NRRD is part of teem, and is out of date. Both libraries are in need of modernization.

NRRD’s advantage is wider adoption outside of ITK/VTK ecosystem. MetaImage’s advantage is already implemented support for uncompressed streaming, and ITK’s copy of it is already synchronized with origin’s master.

[lassoan (Andras Lasso)]

@mihail.isakov

disagree

I’ve been in this business for many years and have been discussing with many people but I don’t remember ever getting such a blunt response. Please try to be more constructive. I’m not even sure what you disagree with and how.

@blowekamp

I am uncertain if ITK currently exposes any of these different “kinds” of dimensions. I expect in some of the cases it would exposed as meta-data in the ImageIO, or in the Image class. This would allow the application to interpret the meaning of a particular axis.

Yes, it would be essential for ITK ImageIO to be able to read/write axis kind information. Otherwise you would need to manually specify this information each time you load the data.

According to nrrd specifications space dimension “determines how many components there are in the vectors given in the space origin, space directions, and measurement frame vectors”. So, I think space dimension: 4 is incorrect and it should be space dimension: 3, even for 4D and 5D volumes. If ITK ImageIO writes space dimension: 4 then it should be probably fixed (simply removing the field is an option, as this is not a mandatory field).

@blowekamp

I would fall back to how ImageJ loads these high dimensional images, with lack of specific meta-data, the user is just prompted to specify the interpretation of the dimensions. Many other scientific domains are not as rigorous as medial imagining’s DICOM standard.

I agree that this may be still needed as a workaround. We can add this to the volume sequence reader in Slicer.

[mihail.isakov]

I’ve been in this business for many years and have been discussing with many people but I don’t remember ever getting such a blunt response. Please try to be more constructive. I’m not even sure what you disagree with and how.

I disagree with

Space dimension of 4 would mean that we have a single volume that is defined in 4D space (not 3D space + time, actually 4D space), that’s why it would not make sense.

because
domain domain domain domain
doesn’t mean
space space space space
it can, but also it can mean
space space space time
or whatever else.

[lassoan (Andras Lasso)]

Sorry, maybe I did not use the “blunt” word correctly (English is not my native language and so I often miss subtle differences). I just had problem with how the discussion was abrupted.

I agree that “domain” 4x is valid, although not very informative. It would be better to use more specific values to avoid the need to manually add the missing info each time the file is loaded.

The field that I think is set incorrectly to 4 is the “space dimension”.

[mihail.isakov]

Sorry, maybe I did not use the “blunt” word correctly

OK, forgotten. I’ll edit my posts and good.