When writing a Nifti image, NiftiImageIO seems to silently orthogonalize the IJK to LPS matrix that is written to the file.
Is this normalization really necessary? Can nifti store images with non-orthogonal axes?
(I’ve tried to decode this from nifti’s documentation, but I’ve found image orientation definition extremely complicated, with lots of ambiguities and redundancy. I hope that there is someone here who knows nifti format well.)
If I remember correctly, Nifti format uses a quaternion to store image orientation, so extra information in 4x4 matrix has to be discarded.
I saw that there orientations can be defined multiple ways in nifti. While qform is indeed a quaternion, sform seems to be a matrix, so it might be possible to save a non-orthogonal matrix.
If nifti really cannot save images with non-orthogonal axes then ITK should throw an error (or at least should have an option for it) when writing of such an image is attempted, instead of silently modifying the image geometry. It leads to data corruption problems as reported by a user here.
Nifti format is causing us so much trouble… I understand that it is popular among neuroimaging people, but its usage as a general medical image file format should be discouraged - nrrd is simple and does not have any of the problems that nifti suffers from.
I would gladly review a PR, and @hjmjohnson would too, probably. But I don’t have enough time to look into it myself.
And there is a related issue in our bug tracker:
I’m the user in @lassoan’s link. My understanding is that as you’ve said, NIFTI headers contain both quaternions (qform) and an affine transformation matrix (sform). The sform can handle shears, while the qform cannot. However, there is also a qform_code field, which describes whether the qform is valid.
My preference as a user in these cases would be to set the qform_code to zero (NIFTI_XFORM_UNKNOWN) and store the full affine warp in the sform. This would preserve the full spatial orientation information and still be a completely correct NIFTI file.
Optionally, one could also store the approximate orthogonalized spatial orientation information in the qform. A user could enable the use of the approximate qform by altering the qform_code if, for example, they needed to use it with software that couldn’t handle the sform and could tolerate the imperfect alignment.
I’m not a big fan of Nifti either, but I think it’s worth noting:
*AFNI does the correct thing and converts Nifti to AFNI format for processing
**ANTs is at least built on top of ITK so I think in theory could read nrrd? I’ve never tried
Hello Henry,
There is a lot of debate about what the qform/sform codes should be, but a significant problem is that various pieces of software are inconsistent about how they handle them.
If you follow https://nifti.nimh.nih.gov/nifti-1/documentation/nifti1fields/nifti1fields_pages/qsform_brief_usage, ideally what should happen is that the qform is not altered by a registration program, and remains set to NIFTI_XFORM_SCANNER_ANAT, while the sform would store the (sheared) transform and be set to NIFTI_XFORM_ALIGNED_ANAT.
The qform should not be set to NIFTI_XFORM_UNKNOWN, as if it is, the qform is essentially ignored completely (see method 0 documented here https://nifti.nimh.nih.gov/nifti-1/documentation/nifti1fields/nifti1fields_pages/qsform.html).
Good points, Tobias. I would ditch Nifti if I could, but it really is the only choice for neuroimaging. My rationale for setting qform_code to NIFTI_XFORM_UNKNOWN is that no truly correct qform exists for the file. I think leaving it as the original scanner coordinates is not widely practiced and likely to be misinterpreted, even though it is technically correct. Despite this, it is probably better than getting rid of the qform entirely or setting it to an approximation, so I agree with you.
My understanding is that if sform_code is set to a non-zero value, the sform should be used rather than the qform. Are you aware of any software that doesn’t do this?
Hello,
My understanding is that your file is the result of a co-registration in Slicer? If that is correct, then there is a valid qform for the image - the one corresponding to the input file’s orientation. That’s the exact point of having SCANNER_ANAT and ALIGNED_ANAT.
To understand the behaviour, I think it’s again important to be aware of history. Originally a lot of Niftis came in two files - the .hdr and .img. The intent was that it should be possible to store the results of a co-registration without actually having to resample or otherwise touch the image data on disk - instead you could modify only the .hdr (or write a new one I guess?). This had the added bonus that only changes to the header would be detected by backup systems, saving storage space (which was important back in 2004/5. I have memories of an SPM mailing list post from John Ashburner explaining this). Cross-reference this to the current SPM behaviour, where for affine co-registrations a separate .mat file is written alongside the input .nii image - it’s the same principle of minimising re-samplings.
These days the .hdr/.img split has (THANKFULLY!) fallen out of common use, but this has also invalidated the above approach. Even if you want to store the co-registration transform in the sform, you are still going to write out a new .nii with a full copy of the input data (unless you are mad and overwriting your input files). And as a result, we are left with these two vestigial transforms in the header leading to huge amounts of confusion.
It is not result of a registration, it is just a tilted-gantry CT acquisition. In such images, axes are not orthogonal.
The main question if there is a way to write a tilted gantry acquisition (non-orthotonal axes image) into a Nifti file that is interpreted correctly by all modern Nifti-capable applications, or it is safer for ITK to refuse writing of such images into Nifti.
Based on what I have learnt from the answers here, it is probably better if ITK’s Nifti IO refuses writing images with non-orthogonal axes.
Ah, I misunderstood. Nifti was specifically introduced for fMRI, I doubt CT acquisitions like this were considered among the original use cases.
If the ITK Nifti writer was updated so that a non-orthogonal sform could be stored, and the sform and qform codes independently controlled by the user, I would now argue that setting the qform code to UNKNOWN and sform to SCANNER_ANAT would be semantically meaningful within the Nifti specification. But I’m probably in a minority on that.
More importantly:
I am not sure if outright refusing to write Niftis with non-orthogonal axes is correct, because:
Sadly I am not sure there is a one-size-fits-all solution.
*As an illustrative aside, FSL’s flirt tool predates the introduction of Nifti and by default ignores orientation information entirely, instead inventing a co-ordinate system (you can force using the qform with a command-line switch). This was a deliberate choice when they wrote it, because ANALYZE header information was notoriously unreliable. The flirt algorithm is robust enough that in most cases it will still find a correct result.
I did some testing and it turns out that ITK allows reading sheared volumes (tested with nrrd and mha) but orthogonalizes axes when writing volumes (tested with mha, nrrd, nifti, mgz, mnc). This is a quite significant inconsistency.
Since ITK allows storage of sheared volumes in image data objects (source), we have two options:
As an application developer, I would prefer Option B, because it would allow our application to orthogonalize images (by resampling or refusing to read/write). As far as I know, this approach is used by VTK, too. An orthogonalization filter could be added to ITK (that would resample the sheared volume to have an orthogonal volume). In the future, it might be even implemented in image IO base class and performed automatically (could be enabled/disabled).
If there is an agreement that we should go towards option B, then a first step could be to remove orthogonalization during image writing for file formats that can store sheared volume and throw an error (instead of silent data corruption) in file formats that can only store orthogonal volumes.
@matt.mccormick @hjmjohnson @blowekamp what are your preferences? We should add an orthogonalization filter either way.
I’m not sure what the proposed desired behavior is here. The sheared image needs to be read and filter to have the ability to correct the volume is desired. Throwing an exception when reading a sheared volume would render the image file unusable!
In general, if an ImageIO does not support writing meta-data contained in an image, them meta-data is silently dropped. I’m thinking of many of the non-medical imaging file formats not support orientation correctly.
The functionality to do this is in the ResampleImageFilter. It should just be wrapping it in a mini-pipeline filter and determining the correct parameters needed. Likely the filter should handle other common “re-gridding” operations as well.
This is a difficult issue.
" The columns of the Direction matrix are expected to form an orthogonal right handed coordinate syste. But this is not checked nor enforced in itk::ImageBase."
I would recommend Option B for the most expedient default behavior. There should be other IO options added to support reading images that do not conform to the strict orthognalized representations.
THROW_ERROR – Default in almost all cases
RESAMPLE - Resample data to orthogonalized form in same grid as provided
EXTRACT_TRANSFORM - return image with identity direction cosign, and an ITK transform object as a separate element.
Hans
I completely agree with you, ITK IO classes should allow read/write of sheared volumes (sorry, I just mixed up the letters A/B).
I would not consider image geometry (image origin, spacing, axis directions) to be “metadata” but it is the data itself. It is not something that I would expect ITK to silently drop. I know that there are software applications/libraries out there that operate in voxel space or don’t care about image directions, but those should not be a reason to lower ITK’s standards.
This sounds perfect. It would ensure correct behavior and compatibility with various workflows (e.g., resample mode can be used in simple viewers, but if you need transform the loaded image then you can create a composite transform of the acquisition transform and the additional transform).
Edit:
In particular gantry tilt in DICOM is not related to “shear” in matrix (should be orthogonal), of course, just for clearness. Every slice position/orientation just have to be saved and the geometry processed later in application.
Gantry tilt is related to shear in matrix, because IJK to LPS matrix of such a volume contains shear (axes are not orthogonal).
Yes, it is true that in some rare cases we may need every slice position and orientation (for example in images with variable slice spacing or non-parallel slices), but most acquisition geometries can be described by a linear transformation.
Gantry tilt is related to shear in matrix, because IJK to LPS matrix of such a volume contains shear (axes are not orthogonal).
Basis should be orthogonal. In DICOM there are 6 values for direction cosines, for X and Y, Z is always cross-product of X and Y. Gantry tilt is shear in Z, it can not be expressed this way.
Equation C.7.6.2.1-1
