# Saving non-orthogonal volume in Nifti format

**URL:** <https://discourse.itk.org/t/saving-non-orthogonal-volume-in-nifti-format/2760>\
**Category:** Engineering\
**Created:** [February 25, 2020, 4:12am UTC](https://discourse.itk.org/t/saving-non-orthogonal-volume-in-nifti-format/2760 "2020-02-25T04:12:29Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![lassoan](https://discourse.itk.org/user_avatar/discourse.itk.org/lassoan/32/27_2.png) [@lassoan](https://discourse.itk.org/u/lassoan)\
**Post date:** [February 25, 2020, 4:12am UTC](https://discourse.itk.org/t/saving-non-orthogonal-volume-in-nifti-format/2760/1 "2020-02-25T04:12:29Z")

</div>

When writing a Nifti image, NiftiImageIO seems to [silently orthogonalize the IJK to LPS matrix that is written to the file](https://github.com/InsightSoftwareConsortium/ITK/blob/0676d81d927036c3666fdd12b7196aad849a3781/Modules/IO/NIFTI/src/itkNiftiImageIO.cxx#L1961-L1962).

Is this normalization really necessary? Can nifti store images with non-orthogonal axes?

(I’ve tried to decode this from nifti’s documentation, but I’ve found image orientation definition extremely complicated, with lots of ambiguities and redundancy. I hope that there is someone here who knows nifti format well.)

---

<div class="post-metadata">

**Author:** ![dzenanz](https://discourse.itk.org/user_avatar/discourse.itk.org/dzenanz/32/1093_2.png) [@dzenanz](https://discourse.itk.org/u/dzenanz)\
**Post date:** [February 25, 2020, 2:50pm UTC](https://discourse.itk.org/t/saving-non-orthogonal-volume-in-nifti-format/2760/2 "2020-02-25T14:50:19Z")

</div>

If I remember correctly, Nifti format uses a quaternion to store image orientation, so extra information in 4x4 matrix has to be discarded.

---

<div class="post-metadata">

**Author:** ![lassoan](https://discourse.itk.org/user_avatar/discourse.itk.org/lassoan/32/27_2.png) [@lassoan](https://discourse.itk.org/u/lassoan)\
**Post date:** [February 25, 2020, 3:01pm UTC](https://discourse.itk.org/t/saving-non-orthogonal-volume-in-nifti-format/2760/3 "2020-02-25T15:01:59Z")

</div>

I saw that there orientations can be defined multiple ways in nifti. While qform is indeed a quaternion, sform seems to be a matrix, so it might be possible to save a non-orthogonal matrix.

If nifti really cannot save images with non-orthogonal axes then ITK should throw an error (or at least should have an option for it) when writing of such an image is attempted, instead of silently modifying the image geometry. It leads to data corruption problems as reported by a user [here](https://discourse.slicer.org/t/nrrd-vs-nii-regarding-affine-data-and-transforms/1896/17).

Nifti format is causing us so much trouble… I understand that it is popular among neuroimaging people, but its usage as a general medical image file format should be discouraged - nrrd is simple and does not have any of the problems that nifti suffers from.

---

<div class="post-metadata">

**Author:** ![dzenanz](https://discourse.itk.org/user_avatar/discourse.itk.org/dzenanz/32/1093_2.png) [@dzenanz](https://discourse.itk.org/u/dzenanz)\
**Post date:** [February 25, 2020, 3:08pm UTC](https://discourse.itk.org/t/saving-non-orthogonal-volume-in-nifti-format/2760/4 "2020-02-25T15:08:02Z")

</div>

I would gladly review a PR, and @hjmjohnson would too, probably. But I don’t have enough time to look into it myself.

---

<div class="post-metadata">

**Author:** ![dzenanz](https://discourse.itk.org/user_avatar/discourse.itk.org/dzenanz/32/1093_2.png) [@dzenanz](https://discourse.itk.org/u/dzenanz)\
**Post date:** [February 25, 2020, 4:25pm UTC](https://discourse.itk.org/t/saving-non-orthogonal-volume-in-nifti-format/2760/5 "2020-02-25T16:25:38Z")

</div>

And there is a related issue in our bug tracker:

> <https://github.com/InsightSoftwareConsortium/ITK/issues/1042>
>
> Description
> Placeholder for possibly a longer bug report. But at least store the information for reproducing the issue.
> Steps to Reproduce
> So basically the...

---

<div class="post-metadata">

**Author:** ![hbraunDSP](https://discourse.itk.org/user_avatar/discourse.itk.org/hbraundsp/32/1261_2.png) [@hbraunDSP](https://discourse.itk.org/u/hbraunDSP)\
**Post date:** [February 25, 2020, 8:00pm UTC](https://discourse.itk.org/t/saving-non-orthogonal-volume-in-nifti-format/2760/6 "2020-02-25T20:00:58Z")

</div>

I’m the user in @lassoan’s link. My understanding is that as you’ve said, NIFTI headers contain both quaternions (`qform`) and an affine transformation matrix (`sform`). The sform can handle shears, while the qform cannot. However, there is also a `qform_code` field, which describes whether the qform is valid.

My preference as a user in these cases would be to set the `qform_code` to zero (`NIFTI_XFORM_UNKNOWN`) and store the full affine warp in the sform. This would preserve the full spatial orientation information and still be a completely correct NIFTI file.

Optionally, one could also store the approximate orthogonalized spatial orientation information in the qform. A user could enable the use of the approximate qform by altering the qform\_code if, for example, they needed to use it with software that couldn’t handle the sform and could tolerate the imperfect alignment.

---

<div class="post-metadata">

**Author:** ![spinicist](https://discourse.itk.org/user_avatar/discourse.itk.org/spinicist/32/183_2.png) [@spinicist](https://discourse.itk.org/u/spinicist)\
**Post date:** [February 25, 2020, 9:48pm UTC](https://discourse.itk.org/t/saving-non-orthogonal-volume-in-nifti-format/2760/7 "2020-02-25T21:48:48Z")

</div>

I’m not a big fan of Nifti either, but I think it’s worth noting:

1. Nifti was “proposed by the NIfTI DFWG as a short-term measure to facilitate _inter-operation_ of functional MRI data analysis software packages” (emphasis mine). It is a sad turn of history that it has also become the default format for neuro-imaging _processing_.
2. At least some of the recent problems that have been discussed here were specifically about still supporting ANALYZE, rather than Nifti 1.1
3. nrrd does suffer from one significant drawback compared to Nifti - to my knowledge, it is not supported by FSL, SPM, AFNI\*, ANTs\*\*, NiftyReg/NiftySeg, Conn, Freesurfer, or BIDS. In short, Nifti has utter dominance in the Neuroimaging world and it would be a big (but noble) struggle to convince people to change. I personally would love to see a different format adopted, but there’s only one of me.

\*AFNI does the correct thing and converts Nifti to AFNI format for processing  
\*\*ANTs is at least built on top of ITK so I think in theory could read nrrd? I’ve never tried

---

<div class="post-metadata">

**Author:** ![spinicist](https://discourse.itk.org/user_avatar/discourse.itk.org/spinicist/32/183_2.png) [@spinicist](https://discourse.itk.org/u/spinicist)\
**Post date:** [February 25, 2020, 9:59pm UTC](https://discourse.itk.org/t/saving-non-orthogonal-volume-in-nifti-format/2760/8 "2020-02-25T21:59:39Z")

</div>

Hello Henry,

There is a lot of debate about what the `qform`/`sform` codes should be, but a significant problem is that various pieces of software are inconsistent about how they handle them.

If you follow [https://nifti.nimh.nih.gov/nifti-1/documentation/nifti1fields/nifti1fields\_pages/qsform\_brief\_usage](https://nifti.nimh.nih.gov/nifti-1/documentation/nifti1fields/nifti1fields_pages/qsform_brief_usage), ideally what should happen is that the `qform` is not altered by a registration program, and remains set to `NIFTI_XFORM_SCANNER_ANAT`, while the `sform` would store the (sheared) transform and be set to `NIFTI_XFORM_ALIGNED_ANAT`.

The `qform` should not be set to `NIFTI_XFORM_UNKNOWN`, as if it is, the `qform` is essentially ignored completely (see method 0 documented here [https://nifti.nimh.nih.gov/nifti-1/documentation/nifti1fields/nifti1fields\_pages/qsform.html](https://nifti.nimh.nih.gov/nifti-1/documentation/nifti1fields/nifti1fields_pages/qsform.html)).

---

<div class="post-metadata">

**Author:** ![hbraunDSP](https://discourse.itk.org/user_avatar/discourse.itk.org/hbraundsp/32/1261_2.png) [@hbraunDSP](https://discourse.itk.org/u/hbraunDSP)\
**Post date:** [February 26, 2020, 12:05am UTC](https://discourse.itk.org/t/saving-non-orthogonal-volume-in-nifti-format/2760/9 "2020-02-26T00:05:58Z")

</div>

Good points, Tobias. I would ditch Nifti if I could, but it really is the only choice for neuroimaging. My rationale for setting `qform_code` to `NIFTI_XFORM_UNKNOWN` is that no truly correct qform exists for the file. I think leaving it as the original scanner coordinates is not widely practiced and likely to be misinterpreted, even though it is technically correct. Despite this, it is probably better than getting rid of the qform entirely or setting it to an approximation, so I agree with you.

My understanding is that if `sform_code` is set to a non-zero value, the sform should be used rather than the qform. Are you aware of any software that doesn’t do this?

---

<div class="post-metadata">

**Author:** ![spinicist](https://discourse.itk.org/user_avatar/discourse.itk.org/spinicist/32/183_2.png) [@spinicist](https://discourse.itk.org/u/spinicist)\
**Post date:** [February 26, 2020, 3:40pm UTC](https://discourse.itk.org/t/saving-non-orthogonal-volume-in-nifti-format/2760/10 "2020-02-26T15:40:43Z")

</div>

Hello,

My understanding is that your file is the result of a co-registration in Slicer? If that is correct, then there is a valid `qform` for the image - the one corresponding to the input file’s orientation. That’s the exact point of having `SCANNER_ANAT` and `ALIGNED_ANAT`.

To understand the behaviour, I think it’s again important to be aware of history. Originally a lot of Niftis came in two files - the `.hdr` and `.img`. The intent was that it should be possible to store the results of a co-registration without actually having to resample or otherwise touch the image data on disk - instead you could modify only the `.hdr` (or write a new one I guess?). This had the added bonus that only changes to the header would be detected by backup systems, saving storage space (which was important back in 2004/5. I have memories of an SPM mailing list post from John Ashburner explaining this). Cross-reference this to the current SPM behaviour, where for affine co-registrations a separate `.mat` file is written alongside the input `.nii` image - it’s the same principle of minimising re-samplings.

These days the `.hdr`/`.img` split has (THANKFULLY!) fallen out of common use, but this has also invalidated the above approach. Even if you want to store the co-registration transform in the `sform`, you are still going to write out a new `.nii` with a full copy of the input data (unless you are mad and overwriting your input files). And as a result, we are left with these two vestigial transforms in the header leading to huge amounts of confusion.

---

<div class="post-metadata">

**Author:** ![lassoan](https://discourse.itk.org/user_avatar/discourse.itk.org/lassoan/32/27_2.png) [@lassoan](https://discourse.itk.org/u/lassoan)\
**Post date:** [February 26, 2020, 5:09pm UTC](https://discourse.itk.org/t/saving-non-orthogonal-volume-in-nifti-format/2760/11 "2020-02-26T17:09:30Z")

</div>

> [@spinicist](#):
>
> My understanding is that your file is the result of a co-registration in Slicer?

It is not result of a registration, it is just a tilted-gantry CT acquisition. In such images, axes are not orthogonal.

The main question if there is a way to write a tilted gantry acquisition (non-orthotonal axes image) into a Nifti file that is interpreted correctly by all modern Nifti-capable applications, or it is safer for ITK to refuse writing of such images into Nifti.

Based on what I have learnt from the answers here, it is probably better if ITK’s Nifti IO refuses writing images with non-orthogonal axes.

---

<div class="post-metadata">

**Author:** ![spinicist](https://discourse.itk.org/user_avatar/discourse.itk.org/spinicist/32/183_2.png) [@spinicist](https://discourse.itk.org/u/spinicist)\
**Post date:** [February 26, 2020, 5:29pm UTC](https://discourse.itk.org/t/saving-non-orthogonal-volume-in-nifti-format/2760/12 "2020-02-26T17:29:44Z")

</div>

Ah, I misunderstood. Nifti was specifically introduced for fMRI, I doubt CT acquisitions like this were considered among the original use cases.

If the ITK Nifti writer was updated so that a non-orthogonal `sform` could be stored, and the `sform` and `qform` codes independently controlled by the user, I would now argue that setting the `qform` code to `UNKNOWN` and `sform` to `SCANNER_ANAT` would be semantically meaningful within the Nifti specification. But I’m probably in a minority on that.

More importantly:

1. That’s a lot of work
2. Applications would have to be aware of this and special-case writing Niftis
3. As you mention, I don’t think it would guarantee all applications would read the resulting file correctly anyway\*.

I am not sure if outright refusing to write Niftis with non-orthogonal axes is correct, because:

1. As I outlined above, non-orthogonal axes as the result of an affine transform are valid and indeed explicitly catered for in the spec.
2. As in @dzenanz’s linked bug, there are DICOMs in the wild that do not have perfectly orthogonal axes, but probably should do (i.e. the headers are slightly wrong). Maybe the strictly correct thing to do would be to fix the header information before writing to Nifti?

Sadly I am not sure there is a one-size-fits-all solution.

\*As an illustrative aside, FSL’s flirt tool predates the introduction of Nifti and by default _ignores orientation information entirely_, instead inventing a co-ordinate system (you can force using the `qform` with a command-line switch). This was a deliberate choice when they wrote it, because ANALYZE header information was notoriously unreliable. The flirt algorithm is robust enough that in most cases it will still find a correct result.

---

<div class="post-metadata">

**Author:** ![lassoan](https://discourse.itk.org/user_avatar/discourse.itk.org/lassoan/32/27_2.png) [@lassoan](https://discourse.itk.org/u/lassoan)\
**Post date:** [February 29, 2020, 8:23pm UTC](https://discourse.itk.org/t/saving-non-orthogonal-volume-in-nifti-format/2760/13 "2020-02-29T20:23:25Z")

</div>

I did some testing and it turns out that ITK allows reading sheared volumes (tested with nrrd and mha) but orthogonalizes axes when writing volumes (tested with mha, nrrd, nifti, mgz, mnc). This is a quite significant inconsistency.

Since ITK allows storage of sheared volumes in image data objects ([source](https://itk.org/Doxygen/html/classitk_1_1ImageBase.html#a23ebc76de3b60bb1eeabf66cd4dabc48)), we have two options:

- **Option A:** enable sheared volumes in ITK image IO class inputs and outputs
- **Option B:** not let ITK image IO classes to read/write sheared volumes (throw an error if such volumes are encountered)

As an application developer, I would prefer Option B, because it would allow our application to orthogonalize images (by resampling or refusing to read/write). As far as I know, this approach is used by VTK, too. An orthogonalization filter could be added to ITK (that would resample the sheared volume to have an orthogonal volume). In the future, it might be even implemented in image IO base class and performed automatically (could be enabled/disabled).

If there is an agreement that we should go towards option B, then a first step could be to remove orthogonalization during image writing for file formats that can store sheared volume and throw an error (instead of silent data corruption) in file formats that can only store orthogonal volumes.

---

<div class="post-metadata">

**Author:** ![dzenanz](https://discourse.itk.org/user_avatar/discourse.itk.org/dzenanz/32/1093_2.png) [@dzenanz](https://discourse.itk.org/u/dzenanz)\
**Post date:** [March 2, 2020, 3:33pm UTC](https://discourse.itk.org/t/saving-non-orthogonal-volume-in-nifti-format/2760/14 "2020-03-02T15:33:20Z")

</div>

@matt.mccormick @hjmjohnson @blowekamp what are your preferences? We should add an orthogonalization filter either way.

---

<div class="post-metadata">

**Author:** ![blowekamp](https://discourse.itk.org/user_avatar/discourse.itk.org/blowekamp/32/79_2.png) [@blowekamp](https://discourse.itk.org/u/blowekamp)\
**Post date:** [March 2, 2020, 3:50pm UTC](https://discourse.itk.org/t/saving-non-orthogonal-volume-in-nifti-format/2760/15 "2020-03-02T15:50:00Z")

</div>

> [@lassoan](#):
>
> **Option B:** not let ITK image IO classes to read/write sheared volumes (throw an error if such volumes are encountered)

I’m not sure what the proposed desired behavior is here. The sheared image needs to be _read_ and filter to have the ability to correct the volume is desired. Throwing an exception when reading a sheared volume would render the image file unusable!

> [@lassoan](#):
>
> If there is an agreement that we should go towards option B, then a first step could be to remove orthogonalization during image writing for file formats that can store sheared volume and throw an error (instead of silent data corruption) in file formats that can only store orthogonal volumes.

In general, if an ImageIO does not support writing meta-data contained in an image, them meta-data is silently dropped. I’m thinking of many of the non-medical imaging file formats not support orientation correctly.

> [@dzenanz](#):
>
> We should add an orthogonalization filter either way.

The functionality to do this is in the ResampleImageFilter. It should just be wrapping it in a mini-pipeline filter and determining the correct parameters needed. Likely the filter should handle other common “re-gridding” operations as well.

---

<div class="post-metadata">

**Author:** ![hjmjohnson](https://discourse.itk.org/user_avatar/discourse.itk.org/hjmjohnson/32/23_2.png) [@hjmjohnson](https://discourse.itk.org/u/hjmjohnson)\
**Post date:** [March 2, 2020, 4:00pm UTC](https://discourse.itk.org/t/saving-non-orthogonal-volume-in-nifti-format/2760/16 "2020-03-02T16:00:36Z")

</div>

This is a difficult issue.

" The columns of the Direction matrix are expected to form an orthogonal right handed coordinate syste. But this is not checked nor enforced in [itk::ImageBase](https://itk.org/Doxygen/html/classitk_1_1ImageBase.html)."

I would recommend Option B for the most expedient default behavior. There should be other IO options added to support reading images that do not conform to the strict orthognalized representations.  
THROW\_ERROR – Default in almost all cases  
RESAMPLE - Resample data to orthogonalized form in same grid as provided  
EXTRACT\_TRANSFORM - return image with identity direction cosign, and an ITK transform object as a separate element.

Hans

---

<div class="post-metadata">

**Author:** ![lassoan](https://discourse.itk.org/user_avatar/discourse.itk.org/lassoan/32/27_2.png) [@lassoan](https://discourse.itk.org/u/lassoan)\
**Post date:** [March 2, 2020, 5:37pm UTC](https://discourse.itk.org/t/saving-non-orthogonal-volume-in-nifti-format/2760/17 "2020-03-02T17:37:40Z")

</div>

> [@lassoan](#):
>
> > [@lassoan](#):
> >
> > **Option B:** not let ITK image IO classes to read/write sheared volumes (throw an error if such volumes are encountered)
> 
> I’m not sure what the proposed desired behavior is here. The sheared image needs to be _read_ and filter to have the ability to correct the volume is desired. Throwing an exception when reading a sheared volume would render the image file unusable!

I completely agree with you, ITK IO classes should allow read/write of sheared volumes (sorry, I just mixed up the letters A/B).

> [@blowekamp](#):
>
> In general, if an ImageIO does not support writing meta-data contained in an image, them meta-data is silently dropped. I’m thinking of many of the non-medical imaging file formats not support orientation correctly.

I would not consider image geometry (image origin, spacing, axis directions) to be “metadata” but it is the data itself. It is not something that I would expect ITK to silently drop. I know that there are software applications/libraries out there that operate in voxel space or don’t care about image directions, but those should not be a reason to lower ITK’s standards.

> [@hjmjohnson](#):
>
> There should be other IO options added to support reading images that do not conform to the strict orthognalized representations.  
> THROW\_ERROR – Default in almost all cases  
> RESAMPLE - Resample data to orthogonalized form in same grid as provided  
> EXTRACT\_TRANSFORM - return image with identity direction cosign, and an ITK transform object as a separate element.

This sounds perfect. It would ensure correct behavior and compatibility with various workflows (e.g., resample mode can be used in simple viewers, but if you need transform the loaded image then you can create a composite transform of the acquisition transform and the additional transform).

---

<div class="post-metadata">

**Author:** ![mihail.isakov](https://discourse.itk.org/letter_avatar_proxy/v4/letter/m/c4cdca/32.png) [@mihail.isakov](https://discourse.itk.org/u/mihail.isakov)\
**Post date:** [March 3, 2020, 4:27am UTC](https://discourse.itk.org/t/saving-non-orthogonal-volume-in-nifti-format/2760/18 "2020-03-03T04:27:22Z")

</div>

> [@blowekamp](#):
>
> In general, if an ImageIO does not support writing meta-data contained in an image, them meta-data is silently dropped.

👍

 ![20200303-051038](https://discourse.itk.org/uploads/default/original/2X/f/f9b2be15d82d70a111b090beb5bccc25e43c3e9b.png)

Edit:

In particular gantry tilt in DICOM is not related to “shear” in matrix (should be orthogonal), of course, just for clearness. Every slice position/orientation just have to be saved and the geometry processed later in application.

---

<div class="post-metadata">

**Author:** ![lassoan](https://discourse.itk.org/user_avatar/discourse.itk.org/lassoan/32/27_2.png) [@lassoan](https://discourse.itk.org/u/lassoan)\
**Post date:** [March 3, 2020, 5:22am UTC](https://discourse.itk.org/t/saving-non-orthogonal-volume-in-nifti-format/2760/19 "2020-03-03T05:22:20Z")

</div>

> [@mihail.isakov](#):
>
> In particular gantry tilt in DICOM is not related to possible “shear” in matrix, of course, just for clearness.

Gantry tilt is related to shear in matrix, because IJK to LPS matrix of such a volume contains shear (axes are not orthogonal).

> [@mihail.isakov](#):
>
> Every slice position/orientation just have to be saved and the geometry processed later in application.

Yes, it is true that in some rare cases we may need every slice position and orientation (for example in images with variable slice spacing or non-parallel slices), but most acquisition geometries can be described by a linear transformation.

---

<div class="post-metadata">

**Author:** ![mihail.isakov](https://discourse.itk.org/letter_avatar_proxy/v4/letter/m/c4cdca/32.png) [@mihail.isakov](https://discourse.itk.org/u/mihail.isakov)\
**Post date:** [March 3, 2020, 5:36am UTC](https://discourse.itk.org/t/saving-non-orthogonal-volume-in-nifti-format/2760/20 "2020-03-03T05:36:10Z")

</div>

> Gantry tilt is related to shear in matrix, because IJK to LPS matrix of such a volume contains shear (axes are not orthogonal).

Basis should be orthogonal. In DICOM there are 6 values for direction cosines, for X and Y, Z is always cross-product of X and Y. Gantry tilt is shear in Z, it can not be expressed this way.

[C.7.6.2 Image Plane Module](http://dicom.nema.org/medical/dicom/current/output/chtml/part03/sect_C.7.6.2.html)

**Equation C.7.6.2.1-1**

![part03_withmml_image_1](https://discourse.itk.org/uploads/default/original/2X/b/bbddcd81836c27fac9a80666f0634df2b8afd889.jpeg)

[Next page](https://discourse.itk.org/t/saving-non-orthogonal-volume-in-nifti-format/2760.md?page=2)